[Python-ideas] Re: Ensuring asserts are always on in a file

[Alex Hall]

On Mon, May 18, 2020 at 12:03 AM Richard Damon 
wrote:

> On 5/17/20 5:04 PM, Alex Hall wrote:
> > Some people (like myself, or the coworkers of [this person](
> https://mail.python.org/archives/list/python-ideas@python.org/thread/PLXOXKACKGXN4ZKISDVXLKMFIETWTF63/))
> just like to use asserts as a convenient way to check things. We don't want
> asserts to ever be turned off. Maybe there could be some kind of compiler
> directive which means "in this file, even with -O, keep the asserts". Maybe
> the line `assert __debug__`?
> >
> My answer to that is 'Your doing int wrong', that is like asking for an
> if statement to not bother checking some of its conditionl.
>
> the assert operation has, like forever, been a debugging aid to say this
> condition should ALWAYS/NEVER occur, check for it in Debug Builds, but
> not for production builds. In some cases (some languages) the compiler
> might even optimize to code based on the knowledge that the condition,
> even in code executed before the condition (if it can be sure the assert
> will be reached).
>

That bit about other languages is fascinating! And a bit scary...


> To quote the documentation:
>
> Assertions should *not* be used to test for failure cases that can occur
> because of bad user input or operating system/environment failures, such
> as a file not being found.
>

 I understand this, but I still don't understand in what situation people
use assertions 'correctly'. To me an assert implies:

1. If the condition is false: that's bad, and the code shouldn't keep
running.
2. I'm not 100% sure the condition is always true (a bug in my code is
always a possiblity) and I need code to check for me.

In that case I want the check there even more in production than in
development. It never makes sense to me for such checks to be turned off.

I understand the reason to turn them off is performance, but this always
seems like a minor optimisation. I don't want code to be significantly
slower without -O. For me it makes development and testing slow and
painful, for users it pushes them to use a global switch to solve a local
problem at the cost of safety. Really slow correctness checking is
generally reserved for continuous integration tests. I've never wanted to
put something significantly slow in an assert and I've never seen anyone
else do that, or write in their documentation "this code is best used with
-O". So the tradeoff always seems clear - accept a small performance hit
for better peace of mind of correctness. If I was really concerned about
speed, Python is already the wrong language.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7U6KB2CSNBKC2UTADWACDD4SOAGC5VNR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: [Python-Dev] Re: PEP 618: Add Optional Length-Checking To zip

[Rhodri James]

On 17/05/2020 19:43, David Mertz wrote:

I believe boolean mode switches are usually a bad design
for Python.  Not always, there are exceptions like open().


Actually, open() is a really bad example.  It does have a flag, 
"closefd" which if False and a file descriptor was passed in rather than 
a filename leaves the file descriptor open when the file object is 
closed.  The mode parameter than most people will be thinking about 
really is a mode parameter, not a flag; it folds together four basic 
opening modes (read, write, exclusive, append), an update flag and a 
text/binary flag.  The former universal newlines flag got separated out 
to be a mode parameter all its own when it turned out not to be a simple 
flag after all.  I seem to remember that separation being somewhat 
painful...


--
Rhodri James *-* Kynesim Ltd
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ML7BU75U6R36Y2MKDD3B4UYZWRZHAIFT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Ensuring asserts are always on in a file

[Richard Damon]

On 5/18/20 5:25 AM, Alex Hall wrote:
> On Mon, May 18, 2020 at 12:03 AM Richard Damon
> mailto:rich...@damon-family.org>> wrote:
>
> On 5/17/20 5:04 PM, Alex Hall wrote:
> > Some people (like myself, or the coworkers of [this
> 
> person](https://mail.python.org/archives/list/python-ideas@python.org/thread/PLXOXKACKGXN4ZKISDVXLKMFIETWTF63/))
> just like to use asserts as a convenient way to check things. We
> don't want asserts to ever be turned off. Maybe there could be
> some kind of compiler directive which means "in this file, even
> with -O, keep the asserts". Maybe the line `assert __debug__`?
> >
> My answer to that is 'Your doing int wrong', that is like asking
> for an
> if statement to not bother checking some of its conditionl.
>
> the assert operation has, like forever, been a debugging aid to
> say this
> condition should ALWAYS/NEVER occur, check for it in Debug Builds, but
> not for production builds. In some cases (some languages) the compiler
> might even optimize to code based on the knowledge that the condition,
> even in code executed before the condition (if it can be sure the
> assert
> will be reached).
>
>
> That bit about other languages is fascinating! And a bit scary...
>  
>
> To quote the documentation:
>
> Assertions should *not* be used to test for failure cases that can
> occur
> because of bad user input or operating system/environment
> failures, such
> as a file not being found.
>
>
>  I understand this, but I still don't understand in what situation
> people use assertions 'correctly'. To me an assert implies:
>
> 1. If the condition is false: that's bad, and the code shouldn't keep
> running.
> 2. I'm not 100% sure the condition is always true (a bug in my code is
> always a possiblity) and I need code to check for me.
>
> In that case I want the check there even more in production than in
> development. It never makes sense to me for such checks to be turned off.
>
> I understand the reason to turn them off is performance, but this
> always seems like a minor optimisation. I don't want code to be
> significantly slower without -O. For me it makes development and
> testing slow and painful, for users it pushes them to use a global
> switch to solve a local problem at the cost of safety. Really slow
> correctness checking is generally reserved for continuous integration
> tests. I've never wanted to put something significantly slow in an
> assert and I've never seen anyone else do that, or write in their
> documentation "this code is best used with -O". So the tradeoff always
> seems clear - accept a small performance hit for better peace of mind
> of correctness. If I was really concerned about speed, Python is
> already the wrong language.

The idea is that during debug, you have added sentinel checking the pre-
and post- conditions to routines. Pre-conditions are those requirements
that MUST be met to call the function, things that the caller was
supposed to make sure were true to begin with (either from other
routines post conditions or what it has checked itself. A failure of a
per-condition indicate a bug in the caller, and routines have these
asserts to help find errors in the caller. The post conditions are the
things the routine promises to deliver, a failure of a post condition
indicates a bug in this routine.

One part of asserts is there presence documents what the routines pre-
and post-conditions are (in addition to their debugging help)

Many of these tests are trivial (thing like simple tests on values) and
for these the removal may not matter that much (but sometimes a 1% gain
is important, maybe not so much in Python though). Some of the tests may
be very expensive, test like 'tree is balanced' or 'list has no
duplicate values'  may require more work to verify then the routine
itself, and these can be important to remove in production releases.

For the 'cheap' asserts, if you want to keep the test in the code
because you aren't sure you have debugged enough, you can replace the
assert with an if, with the assert statement in the conditional (or just
raise the exception yourself, or better use a custom problem report
routine). The normal reason to not want to do this is that the asserts
output isn't really friendly to the average user, but is designed for
the developer. In some cases, it might even reveal information that the
end user isn't really supposed to know.

-- 
Richard Damon
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Q4DHIKGZW6P6YVTONUAURYPM2H4VJMHT/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Optional keyword arguments

[James Lu]

"There should be one-- and preferably only one --obvious way to do it."
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/U6OFH6FEWT2274KSRMQXH75TMT653TNO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Optional keyword arguments

[Richard Damon]

On 5/18/20 9:06 AM, James Lu wrote:
> "There should be one-- and preferably only one --obvious way to do it."

*obvious*

multiple ways are allowed as long as there is one clear preference.

-- 
Richard Damon
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/PCAVU6BEI4KUYUUVL7F3CKV2EQ7ZPBPK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Optional keyword arguments

[Caleb Donovick]

Certainly the way default arguments work with mutable types is not the most
intuitive and I think your complaint has some merit.

However how would you define the following to work:

def foo():
cons = [set(), [], (),]
funs = []
for ds in cons:
def g(arg:=ds):
return arg
funs.append(g)
return funs

How would you evaluate "ds" in the context of the call?
If it were to have the same observable behavior as def g(arg=ds) except
that you would get "fresh" reference on each invocation you would get the
following:

assert [f() for f in foo()]  == [set(), [], ()]

Note it cannot be a simple syntactic transform because:

class _MISSING: pass
def foo():
cons = [set(), [], (),]
funs = []
for ds in cons:
def g(arg=_MISSING):
if arg is _MISSING:
arg = eval('ds') # equivalent to arg = ds so does not
produce a fresh reference
return arg
funs.append(g)
  return funs

assert [f() for f in foo()]  == [(), (), ()]

Granted the way closures work (especially in the context of loops) is also
a pretty unintuitive, but stands as a barrier to easily implementing your
desired behavior.
And even if that wasn't the case we still have the issue that eval('ds')
doesn't give you a fresh reference.

Would it implicitly deepcopy ds?  e.g.:

class _MISSING: pass
def build_g(default):
def g(arg=_MISSING):
if arg is _MISSING:
arg =  deepcopy(default)
return arg
return g

def foo():
cons = [set(), [], (),]
funs = []
for ds in cons:
g = build_g(ds)
funs.append(g)
  return funs

What if ds doesn't implement __deepcopy__?


On Mon, May 18, 2020 at 7:11 AM Richard Damon 
wrote:

> On 5/18/20 9:06 AM, James Lu wrote:
> > "There should be one-- and preferably only one --obvious way to do it."
>
> *obvious*
>
> multiple ways are allowed as long as there is one clear preference.
>
> --
> Richard Damon
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/PCAVU6BEI4KUYUUVL7F3CKV2EQ7ZPBPK/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YE77WSNCGMLNVCTTD472WFWAELURMHSF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Formally deprecate urllib.request.urlretrieve() and urllib.request.urlcleanup()

[remi . lapeyre]

Both urlretrieve() and urlcleanup() were kept for backward compatibility with 
Python2 but they were never deprecated like urllib.request.URLOpener and 
urllib.request.FancyURLOpener.

Now that Python2 has been definitely sunset, can we raise a deprecation warning 
so that they can be removed in the future?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BVLC52RM23DNTSVCYGSG4QK6KYTTFABJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Formally deprecate urllib.request.urlretrieve() and urllib.request.urlcleanup()

[Guido van Rossum]

Sounds eminently reasonable -- for this kind of stuff, why don't you open a
bpo issue?

On Mon, May 18, 2020 at 2:46 PM  wrote:

> Both urlretrieve() and urlcleanup() were kept for backward compatibility
> with Python2 but they were never deprecated like urllib.request.URLOpener
> and urllib.request.FancyURLOpener.
>
> Now that Python2 has been definitely sunset, can we raise a deprecation
> warning so that they can be removed in the future?
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/BVLC52RM23DNTSVCYGSG4QK6KYTTFABJ/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/E7C2QU3DSUYXCHHA2ZV6JQOPRFS2YVXK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Formally deprecate urllib.request.urlretrieve() and urllib.request.urlcleanup()

[remi . lapeyre]

Thanks!

I wasn't sure whether I was missing something as they were not deprecated at 
the same time as URLOpener so I thought I would ask here before opening a 
useless issue. Sorry for the noise if this not an appropriate usage of this 
list.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/FCKRN42DTGJH3WLJLIT72DSPOHNYPZSS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Formally deprecate urllib.request.urlretrieve() and urllib.request.urlcleanup()

[Guido van Rossum]

Well it’s always possible that you’ll get an answer saying these should not
be deprecated. How can you be sure they were only kept for Py 2
compatibility?

On Mon, May 18, 2020 at 15:15  wrote:

> Thanks!
>
> I wasn't sure whether I was missing something as they were not deprecated
> at the same time as URLOpener so I thought I would ask here before opening
> a useless issue. Sorry for the noise if this not an appropriate usage of
> this list.
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/FCKRN42DTGJH3WLJLIT72DSPOHNYPZSS/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-- 
--Guido (mobile)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/T53XPYTMXQCK5QWTU6QPTADNGAG3C6D6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Formally deprecate urllib.request.urlretrieve() and urllib.request.urlcleanup()

[remi . lapeyre]

The doc says "The following functions and classes are ported from the Python 2 
module urllib (as opposed to urllib2). They might become deprecated at some 
point in the future."
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WANLAKXVRY3NQDDEWMUL6GNVDSXRNLST/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Ensuring asserts are always on in a file

[Steven D'Aprano]

On Mon, May 18, 2020 at 11:25:39AM +0200, Alex Hall wrote:

>  I understand this, but I still don't understand in what situation people
> use assertions 'correctly'. 

https://import-that.dreamwidth.org/676.html


> To me an assert implies:
> 
> 1. If the condition is false: that's bad, and the code shouldn't keep
> running.
>
> 2. I'm not 100% sure the condition is always true (a bug in my code is
> always a possiblity) and I need code to check for me.
 
They apply to many other conditional checks apart from assertions.

The question should be, what distinguishes an *assertion* from other 
conditional checks followed by raise? Both trigger on errors; the 
difference is in the semantics: assert is intended for use 
during development.

If we are confident that there are no bugs (at least not in the areas 
checked by assertions), then the assertions are safe to disable in 
production.

If we're not confident, then we're running a system which is not ready 
for production, and we should be honest about that at least to 
ourselves even if we don't exactly advertise that fact :-)

And if removing the asserts could *cause* bugs, rather than reveal 
their existence, then they certainly shouldn't be asserts.


> In that case I want the check there even more in production than in
> development. It never makes sense to me for such checks to be turned off.

That opinion pretty much goes against the standard practice in every 
language that I know of that has assertions. "Assertions that you cannot 
turn off" are just regular error checking. The ability to disable 
assertions is a feature, not a bug, and is the primary feature that 
distinguishes asserts from `if... raise`.

---  -
assert   if...raise
---  -
raises if not condition  raises if condition
can be disabled  cannot be disabled
limited to AssertionErrorno limit to the exception type
---  -

If you expect the assertions to fail in production, why not fix the 
error conditions that could lead them to fail?

Probably no other language makes assertions a bigger part of the 
language than Eiffel. They have, by my count, at least five kinds of 
assertions:

* preconditions
* postconditions
* loop invariants
* class invariants
* checks

and they are so fundamental to Eiffel's programming model that they are 
part of the syntactic structure of classes and methods. But even 
Eiffel designed assertions to be disabled:

"During development and testing, assertion monitoring should be turned 
on at the highest possible level. [...] When releasing the final version 
of a system, it is usually appropriate to turn off assertion monitoring, 
or bring it down to the `require` level. The exact policy depends on the 
circumstances; it is a trade off between efficiency considerations, the 
potential cost of mistakes, and how much the developers and quality 
assurance team trust the product."

https://www.eiffel.org/doc/eiffel/ET-_Design_by_Contract_%28tm%29%2C_Assertions_and_Exceptions

Compared to Eiffel's fine-grained assertion system, Python's is pretty 
coarse, we don't have multiple levels of assertion testing, just All On 
and All Off. But the principle still applies. Assertions are intended to 
be capable of being disabled.


> I understand the reason to turn them off is performance, but this always
> seems like a minor optimisation.

Certainly in Python the optimizations tend to be small, but not always 
insignificant. And even small things can add up to make a significant 
difference.


> I don't want code to be significantly
> slower without -O.

So you want it to be equally slow with -O? *wink*

The aim isn't to intentionally slow down the unoptimised code, but to 
speed up the optimized code by trading off some "just in case" tests 
which should never fail for additional speed. It is the person running 
the code, not the developer, who decides whether to make that tradeoff 
or not.

(Unless the developer and the user are the same person, in which 
case why do you care? Just don't run your code with -O. If you are your 
application's sole user, then do whatever you like, it's okay.)

If you know a test should never fail, then it is safe to turn it off. If 
it's not safe to turn it off, then it's not an assertion.

(Safety, of course, is not an absolute. If it were absolute, we wouldn't 
bother running the assertions at all, ever, since we would know that 
they absolutely will never fail.)

> For me it makes development and testing slow and painful,

Sorry, are you saying that assertions make development and testing slow 
and painful? If that's not what you mean, I don't know what this means.

In pre-production and debugging, you run your code without -O. If the 
application is slow and painful, you have to solve that regardless of 
ass

[Python-ideas] Re: Optional keyword arguments

[Steven D'Aprano]

On Mon, May 18, 2020 at 01:06:22PM -, James Lu wrote:

> "There should be one-- and preferably only one --obvious way to do it."

Yes?

"There should be one" does not mean the same thing as "there shouldn't 
be two".

"There should be one qualified pilot in the cockpit of the plane at all 
times during the flight" is not the same as "There should not be two 
qualified pilots in the cockpit".



-- 
Steven
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IA32WCRDPACLI6R562GDPETV6WQRUBSV/
Code of Conduct: http://python.org/psf/codeofconduct/