Re: How to make Python interpreter a little more strict?

[Steven D'Aprano]

On Mon, 28 Mar 2016 07:49 am, BartC wrote:

> On 27/03/2016 21:32, Tim Chase wrote:
>> On 2016-03-27 14:28, Steven D'Aprano wrote:
> 
>>> In this case, the two lines "fnc" and "next" simply look up the
>>> function names, but without actually calling them. They're not
>>> quite "no-ops", since they can fail and raise NameError if the name
>>> doesn't exist, but otherwise they might as well be no-ops.
>>
>> Which is actually useful.  I've got some 2.4 code that reads
>>
>>try:
>>  any
>>except NameError:
>>  def any(...):
>>...
>>
>> (with a similar block for all() )
>>
>> I don't want to call any() or all(), I simply want to test whether
>> they exist.
> 
> But would it have been much of an imposition to have typed:
> 
>  try:
>test = any
>  except NameError:
>def any(...):
>  ...
> 
> ? (Or any of the half dozen ways there must be to test the existence of
> a name.)

No, not much of an imposition. But then you might have your linter (if you
bother with one) complaining that you define "test" but never use it.

But regardless of whether it is an imposition or not, that's not how we do
it. It's a stylistic choice, that's all. We've explained the reasons for
it, but it ultimately comes down to a matter of taste.



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to make Python interpreter a little more strict?

[Nobody]

On Sat, 26 Mar 2016 23:30:30 +, John Pote wrote:

> So I have sympathy with the OP, I would expect the compiler to pick this
> up

Why? The code is valid, the compiler knows how to generate the
appropriate bytecode for it.

The compiler isn't "lint".

Reporting code which is actually invalid is fairly straightforward.
When the parser attempts to match the next token against a parse rule
and finds that nothing matches (e.g. the "fnc next" example), it just
needs to raise a SyntaxError. The point at which the exception needs to be
raised naturally exists in the code.

But to identify code which is perfectly valid yet is "probably" a mistake
first requires someone to identify such cases, then someone needs to start
adding the appropriate tests to the compiler to distinguish such code from
the rest.

> It would be all to easy to write a series of lines just calling
> functions and forget the () on one of them. Not fun programming. It's
> also a good reminder that the meaning of a keyword in language A is not
> necessarily the same in language B (ie 'next', Python)

"next" isn't a keyword, it's a built-in function. It's perfectly valid to
re-use that name for your own variables or functions.

> So on this last point is this behaviour of Python defined somewhere in
> the docs? 

What behaviour? Evaluating a name?

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to make Python interpreter a little more strict?

[Chris Angelico]

On Mon, Mar 28, 2016 at 7:49 AM, BartC  wrote:
> On 27/03/2016 21:32, Tim Chase wrote:
>>
>> On 2016-03-27 14:28, Steven D'Aprano wrote:
>
>
>>> In this case, the two lines "fnc" and "next" simply look up the
>>> function names, but without actually calling them. They're not
>>> quite "no-ops", since they can fail and raise NameError if the name
>>> doesn't exist, but otherwise they might as well be no-ops.
>>
>>
>> Which is actually useful.  I've got some 2.4 code that reads
>>
>>try:
>>  any
>>except NameError:
>>  def any(...):
>>...
>>
>> (with a similar block for all() )
>>
>> I don't want to call any() or all(), I simply want to test whether
>> they exist.
>
>
> But would it have been much of an imposition to have typed:
>
> try:
>   test = any
> except NameError:
>   def any(...):
> ...
>
> ? (Or any of the half dozen ways there must be to test the existence of a
> name.)

It shifts the uselessness to "why did you assign to that name?". While
that's not too bad (and nothing like as bad as a dummy function call),
it's generally reserved for the places where Python syntax mandates an
assignment:

for _ in range(4): next(iter) # discard the headers

If you're building a linter, I would expect "name assigned to and
never used" to be its own warning; but also, the try/except block
hints that this isn't useless. Generally, "dummy expressions" like
this are going to explicitly catch at least one exception that can be
raised only in that expression (ie a really small try block). That
pretty much counts as your language-level marker for conscious dummy
expression usage.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to make Python interpreter a little more strict?

[BartC]

On 27/03/2016 21:32, Tim Chase wrote:

On 2016-03-27 14:28, Steven D'Aprano wrote:



In this case, the two lines "fnc" and "next" simply look up the
function names, but without actually calling them. They're not
quite "no-ops", since they can fail and raise NameError if the name
doesn't exist, but otherwise they might as well be no-ops.


Which is actually useful.  I've got some 2.4 code that reads

   try:
 any
   except NameError:
 def any(...):
   ...

(with a similar block for all() )

I don't want to call any() or all(), I simply want to test whether
they exist.


But would it have been much of an imposition to have typed:

try:
  test = any
except NameError:
  def any(...):
...

? (Or any of the half dozen ways there must be to test the existence of 
a name.)


--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to make Python interpreter a little more strict?

[Tim Chase]

On 2016-03-27 14:28, Steven D'Aprano wrote:
> > So intrigued by this question I tried the following
> > def fnc( n ):
> >  print "fnc called with parameter '%d'" % n
> >  return n
> > 
> > for i in range(0,5):
> >  if i%2 == 0:
> >  fnc
> >  next
> >  print i
> > 
> > and got the same result as the OP  
> 
> In this case, the two lines "fnc" and "next" simply look up the
> function names, but without actually calling them. They're not
> quite "no-ops", since they can fail and raise NameError if the name
> doesn't exist, but otherwise they might as well be no-ops.

Which is actually useful.  I've got some 2.4 code that reads

  try:
any
  except NameError:
def any(...):
  ...

(with a similar block for all() )

I don't want to call any() or all(), I simply want to test whether
they exist.

-tkc



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to make Python interpreter a little more strict?

[Steven D'Aprano]

On Sun, 27 Mar 2016 10:30 am, John Pote wrote:

> So intrigued by this question I tried the following
> def fnc( n ):
>  print "fnc called with parameter '%d'" % n
>  return n
> 
> for i in range(0,5):
>  if i%2 == 0:
>  fnc
>  next
>  print i
> 
> and got the same result as the OP

In this case, the two lines "fnc" and "next" simply look up the function
names, but without actually calling them. They're not quite "no-ops", since
they can fail and raise NameError if the name doesn't exist, but otherwise
they might as well be no-ops.


> A couple of tests showed that the only important thing about the name in
> the if clause is that it is known at runtime and then it is silently
> ignored.

Right. What actually happens is that Python evaluates the expression,
generates the result, and then if that result isn't used, a microsecond
later the garbage collector deletes it. In this case, the expression
consists of only a single name: "fnc", or "next".

If you have a more complex expression, Python can do significant work before
throwing it away:

[x**3 for x in range(1)]


generates a list of cubed numbers [0, 1, 8, 27, ...]. Then the garbage
collector sees that nothing refers to that list, and it is available to be
deleted, so it deletes it.

Why does Python bother generating the list only to throw it away a
microsecond later? Because the interpreter can't easily tell if the
calculations will have any side-effects. It might turn out that something
in the expression ends up setting a global variable, or printing output, or
writing to a file, or who knows what?

Now, you and I can read the line and see that (assuming range hasn't been
overridden) there are no side-effects from calculating cubes of numbers.
But the Python interpreter is very simple-minded and not as smart as you or
I, so it can't tell, and so it plays it safe and does the calculations just
in case. Future versions of the interpreter may be smarter.

In cases where Python *can* tell that there are no side-effects, it may
ignore stand-alone expressions that don't do anything useful.


> However, if the name is not known/accessible at run time a 'NameError'
> is raised,
> NameError: name 'mynewfn123' is not defined

Correct.



> On the other hand the following if clause
>  if i%2 == 0:
>  fncnext
> 
> results in a compiler error,
> D:\projects\python
>  >>python next.py
>File "next.py", line 9
>  fnc next
> ^
> SyntaxError: invalid syntax
> 
> This is all for Python 2.7.9. (Don't know about Python 3).

Python 3 will be more-or-less the same. Python 3 might be a bit smarter
about recognising expressions that have no side-effects and that can be
ignored, but not much more.


> So I have sympathy with the OP, I would expect the compiler to pick this
> up - indeed it does so for two (or more ?) unused names on a single
> line. That is unless someone can give a useful use of this behaviour or
> is there something going on under the Python hood I'm not aware of?

In Python, variables AND FUNCTIONS are created dynamically, and can be
deleted or created as needed. So Python doesn't know if your function "fnc"
actually exists or not until runtime. Just because you defined it using def
doesn't mean that it will still be around later: maybe you have called:

del fnc

or possibly reassigned the variable:

fnc = "Surprise! Not a function any more!"

So the compiler can't tell whether 

fnc

is a legal line or not. Maybe fnc exists, maybe it doesn't, it will have to
actually evaluate the expression to find out, and that happens at runtime.

But the compiler can recognise syntax errors:

fnc next

is illegal syntax, as are:

1234fnc
fnc?
x = ) (

and for those, you will get an immediate SyntaxError before the code starts
running.


> It would be all to easy to write a series of lines just calling
> functions and forget the () on one of them. Not fun programming.

In theory, you are correct. But in practice, it's not really as big a
problem as you might think. Very, very few functions take no arguments.
There is this one:

import random
r = random.random  # oops, forgot to call the function

for example, but most functions take at least one argument:

r = random.randint(1, 6)

There are a few functions or methods that you call for their side-effects,
like Pascal "procedures" or C "void functions":

mylist.sort()

and yes, it is annoying when you forget the brackets (round brackets or
parentheses for any Americans reading) and the list doesn't sort, but
that's the exception rather than the rule. And if you're worried about
that, you can run a "linter" which check your source code for such
potential problems.

Google for PyLint, PyFlakes, PyChecker, Jedi, etc if you want more
information about linters, or just ask here. I can't tell you too much
about them, as I don't use them, but somebody will probably answer.





-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/

Re: How to make Python interpreter a little more strict?

[BartC]

On 26/03/2016 23:30, John Pote wrote:


So intrigued by this question I tried the following
def fnc( n ):
 print "fnc called with parameter '%d'" % n
 return n

for i in range(0,5):
 if i%2 == 0:
 fnc
 next
 print i

and got the same result as the OP



A couple of tests showed that the only important thing about the name in
the if clause is that it is known at runtime and then it is silently
ignored.
However, if the name is not known/accessible at run time a 'NameError'
is raised,
NameError: name 'mynewfn123' is not defined

On the other hand the following if clause
 if i%2 == 0:
 fncnext


The results aren't surprising once you know what's going on.

If you have:

   fnc

then this name is evaluated (it refers to a function object given your 
definition. Then that value is discarded. But this:


   kljgkjhgjk

is not defined anywhere, and it can't evaluate it, raising the name 
error. While in:


   fnc()

the 'fnc' is evaluated, and then it's called (when it should give a 
parameter error as your def expects one). But


   abc def

is just a syntax error as usually two identifiers can't be adjacent 
(AFAIK) except where the first is a reserved word.



So I have sympathy with the OP, I would expect the compiler to pick this
up - indeed it does so for two (or more ?) unused names on a single
line. That is unless someone can give a useful use of this behaviour or
is there something going on under the Python hood I'm not aware of?


This one of those features I think that do have the odd use but most of 
the time just result in perplexing results or hard-to-find bugs. It 
could have been eliminated from the language (especially as many people 
aren't even aware of the possibilities) with little loss.


Someone who desperately wants to evaluate a name or expression can 
always use something like:


 def test(x):pass

 test(fnc)

for the same purpose. But this time it's obvious. (Ideally it would be a 
built-in construct to avoid the call overhead.)



--
Bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: How to make Python interpreter a little more strict?

[John Pote]

On 26/03/2016 12:05, Chris Angelico wrote:

On Fri, Mar 25, 2016 at 11:06 PM, Aleksander Alekseev  wrote:

Recently I spend half an hour looking for a bug in code like this:

eax@fujitsu:~/temp$ cat ./t.py
#!/usr/bin/env python3

for x in range(0,5):
 if x % 2 == 0:
 next
 print(str(x))

eax@fujitsu:~/temp$ ./t.py
0
1
2
3
4

Is it possible to make python complain in this case? Or maybe solve
such an issue somehow else?

I think what you're looking for here is an acknowledgement that
evaluating the name "next" accomplishes nothing. That's not really
something the Python interpreter should be looking at (hey, you might
have good reason for doing that), but there are linters that can
detect this kind of dead code. Some of them tie into programmer's
editors, so you could get a nice little warning message right in the
window where you're typing your code. Look into some of the top-end
editors (free or commercial) and see what you think of them - they can
save you no end of time.

ChrisA

So intrigued by this question I tried the following
def fnc( n ):
print "fnc called with parameter '%d'" % n
return n

for i in range(0,5):
if i%2 == 0:
fnc
next
print i

and got the same result as the OP

D:\projects\python
>>python next.py
0
1
2
3
4
D:\projects\python
>>

A couple of tests showed that the only important thing about the name in 
the if clause is that it is known at runtime and then it is silently 
ignored.
However, if the name is not known/accessible at run time a 'NameError' 
is raised,

NameError: name 'mynewfn123' is not defined

On the other hand the following if clause
if i%2 == 0:
fncnext

results in a compiler error,
D:\projects\python
>>python next.py
  File "next.py", line 9
fnc next
   ^
SyntaxError: invalid syntax

This is all for Python 2.7.9. (Don't know about Python 3).

So I have sympathy with the OP, I would expect the compiler to pick this 
up - indeed it does so for two (or more ?) unused names on a single 
line. That is unless someone can give a useful use of this behaviour or 
is there something going on under the Python hood I'm not aware of?


It would be all to easy to write a series of lines just calling 
functions and forget the () on one of them. Not fun programming.
It's also a good reminder that the meaning of a keyword in language A is 
not necessarily the same in language B (ie 'next', Python)
So on this last point is this behaviour of Python defined somewhere in 
the docs?


Regards all,
John

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to make Python interpreter a little more strict?

[Chris Angelico]

On Fri, Mar 25, 2016 at 11:06 PM, Aleksander Alekseev  wrote:
> Recently I spend half an hour looking for a bug in code like this:
>
> eax@fujitsu:~/temp$ cat ./t.py
> #!/usr/bin/env python3
>
> for x in range(0,5):
> if x % 2 == 0:
> next
> print(str(x))
>
> eax@fujitsu:~/temp$ ./t.py
> 0
> 1
> 2
> 3
> 4
>
> Is it possible to make python complain in this case? Or maybe solve
> such an issue somehow else?

I think what you're looking for here is an acknowledgement that
evaluating the name "next" accomplishes nothing. That's not really
something the Python interpreter should be looking at (hey, you might
have good reason for doing that), but there are linters that can
detect this kind of dead code. Some of them tie into programmer's
editors, so you could get a nice little warning message right in the
window where you're typing your code. Look into some of the top-end
editors (free or commercial) and see what you think of them - they can
save you no end of time.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to make Python interpreter a little more strict?

[Steven D'Aprano]

On Fri, 25 Mar 2016 11:06 pm, Aleksander Alekseev wrote:

> Is it possible to make python complain in this case? Or maybe solve
> such an issue somehow else?

This is a job for a "linter", such as pychecker, pylint or pyflakes. Google
for more if you are interested.

A linter will check your code for things which are *legal* code, but might
not do what you expect, or are a sign of a potential error. For instance,
dead code that will never be reached, or variables that are defined and
then never used, or functions (such as next) which are named but not used.

Another thing you can do is use an editor which colours functions
like "next" differently from statements like "continue".



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to make Python interpreter a little more strict?

[Mark Lawrence]

On 25/03/2016 12:06, Aleksander Alekseev wrote:

Hello

Recently I spend half an hour looking for a bug in code like this:

eax@fujitsu:~/temp$ cat ./t.py
#!/usr/bin/env python3

for x in range(0,5):
 if x % 2 == 0:
 next
 print(str(x))

eax@fujitsu:~/temp$ ./t.py
0
1
2
3
4

Is it possible to make python complain in this case? Or maybe solve
such an issue somehow else?



How does the interpreter work out that you've typed 'next' instead of 
'continue'?


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to make Python interpreter a little more strict?

[Chris Warrick]

On 25 March 2016 at 13:06, Aleksander Alekseev  wrote:
> Hello
>
> Recently I spend half an hour looking for a bug in code like this:
>
> eax@fujitsu:~/temp$ cat ./t.py
> #!/usr/bin/env python3
>
> for x in range(0,5):
> if x % 2 == 0:
> next
> print(str(x))
>
> eax@fujitsu:~/temp$ ./t.py
> 0
> 1
> 2
> 3
> 4
>
> Is it possible to make python complain in this case? Or maybe solve
> such an issue somehow else?
>
> --
> Best regards,
> Aleksander Alekseev
> http://eax.me/
> --
> https://mail.python.org/mailman/listinfo/python-list

You were probably looking for `continue`. This is just bad memory on
your part, and Python can’t help. In the REPL, typing an object name
is legal and helpful. Other languages might crash, sure — but they
usually don’t have a REPL like Python does.

That said, this code is designed badly. Here’s a better idea (also,
note the fixed print statement):

for x in range(0, 5):
if x % 2 != 0:
print(x)

Or even with a more suitable range() that adds 2 instead of 1 in each step:

for x in range(1, 5, 2):
print(x)

-- 
Chris Warrick 
PGP: 5EAAEA16
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to make Python interpreter a little more strict?

[Nick Sarbicki]

On Sat, Mar 26, 2016 at 9:59 AM Aleksander Alekseev 
wrote:

> Hello
>
> Recently I spend half an hour looking for a bug in code like this:
>
> eax@fujitsu:~/temp$ cat ./t.py
> #!/usr/bin/env python3
>
> for x in range(0,5):
> if x % 2 == 0:
> next
> print(str(x))
>
> eax@fujitsu:~/temp$ ./t.py
> 0
> 1
> 2
> 3
> 4
>
> Is it possible to make python complain in this case? Or maybe solve
> such an issue somehow else?
>
> --
> Best regards,
> Aleksander Alekseev
> http://eax.me/
> --
> https://mail.python.org/mailman/listinfo/python-list


What would it be complaining about?

I don't see any issues, it does what I would expect.
-- 
https://mail.python.org/mailman/listinfo/python-list

How to make Python interpreter a little more strict?

[Aleksander Alekseev]

Hello

Recently I spend half an hour looking for a bug in code like this:

eax@fujitsu:~/temp$ cat ./t.py 
#!/usr/bin/env python3

for x in range(0,5):
if x % 2 == 0:
next
print(str(x))

eax@fujitsu:~/temp$ ./t.py 
0
1
2
3
4

Is it possible to make python complain in this case? Or maybe solve
such an issue somehow else?

-- 
Best regards,
Aleksander Alekseev
http://eax.me/
-- 
https://mail.python.org/mailman/listinfo/python-list