Re: [Python-ideas] Changes to the existing optimization levels

[Diana Clarke]

Oh, I like this idea!

I had very briefly considered treating the existing flag as a
bitfield, but then promptly forgot to explore that line of thought
further.

I'll play with that approach next week, see where it takes me, and
then report back.

Thanks so much for taking the time to think this through with me –
much appreciated.

Cheers,

--diana

On Fri, Sep 29, 2017 at 1:33 AM, Nick Coghlan  wrote:
> 2. We could reinterpret "optimize" as a bitfield instead of a regular
> integer, special casing the already defined values:
>
> - all zero: no optimizations
> - sign bit set: negative -> use global settings
> - 0x0001: nodebug+noassert
> - 0x0002: nodebug+noassert+nodocstrings
> - 0x0004: nodebug
> - 0x0008: noassert
> - 0x0010: nodocstrings
>
> The "redefine optimizations as a bitfield" approach seems particularly
> promising to me - it's a full integer, so even with all negative
> numbers disallowed and the two low order bits reserved for the legacy
> combinations, that's still 29 different optimisation flags given
> 32-bit integers. We currently have 3, so that's room for an 866%
> increase in the number of defined flags :)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] allow overriding files used for the input builtin

[Wren Turkal]

So, I was just thinking, maybe we don't want an errfile arg, but an arg that is 
a sequence of file objects that need to be flushed before showing the prompt. 
That's certainly more complicated, but it seems more general if we want to 
cover this case.


Thoughts?


wt


From: Python-ideas  on behalf of 
Wren Turkal 
Sent: Friday, September 29, 2017 9:53:01 AM
To: Amit Green
Cc: python-ideas@python.org
Subject: [Potential Spoof] Re: [Python-ideas] allow overriding files used for 
the input builtin


Oh, I thought that stderr was unbuffered. Like the following C program:


#include 

int main() {
  fprintf(stdout, "stdout0");
  fprintf(stderr, "stderr0");
  fprintf(stdout, "stdout1");
  return 0;
}


This outputs:

stderr0stdout0stdout1


Turns out fprintf in glibc will also implicitly flush at newlines also. So, the 
following program:

#include 

int main() {
  fprintf(stdout, "stdout0\nstdout12");
  fprintf(stderr, "stderr0");
  fprintf(stdout, "stdout1");
  return 0;
}


outputs:

stdout0
stderr0stdout12stdout1


This is in line with python. The following program:

import sys

sys.stdout.write('stdout0\nstdout01')
sys.stderr.write('stderr0')
sys.stdout.write('stdout1')


outputs:

stdout0
stderr0stdout01stdout1

While the following program:

import sys

sys.stdout.write('stdout0')
sys.stderr.write('stderr0')
sys.stdout.write('stdout1')


outputs:

stderr0stdout0stdout1


So, I guess that the errfile is explicitly flushed due to the fact that stdout 
may or may not be flushed during output.


So, I have 2 questions: (1) do we need to keep the errfile param to make this 
useful (I think we do), and if #1 is yes, (2) what is an appropriate way to doc 
this?


wt


From: Amit Green 
Sent: Friday, September 29, 2017 9:17:39 AM
To: Wren Turkal
Cc: Steven D'Aprano; python-ideas@python.org
Subject: Re: [Python-ideas] allow overriding files used for the input builtin

Hmm, very good point.

Flushing standard error is essential:


  *   This is because output of standard output & standard error are often 
redirected to the same file -- frequently the terminal -- and its important to 
flush one of them before output to the other (since these are typically line 
buffered you only see this if you output partial lines).

  *   Here is an example of not flushing & the mess it causes:

>>> import sys
>>> sys.stdout.write('hi'); sys.stderr.write(' HMM '); 
>>> sys.stdout.write('there\n')
 HMM hithere

  *   Here is the same example with flushing:

>>> sys.stdout.write('hi'); sys.stdout.flush(); sys.stderr.write(' HMM '); 
>>> sys.stderr.flush(); sys.stdout.write('there\n')
hi HMM there

In fact, I think you need to improve your interface and documentation:

  *   If either of the parameters 'outfile', or 'errfile' are passed in & not 
the 'none' value then the following happens:
 *   Both sys.stdout & sys.stderr are flushed (in that order) before being 
redirected.  (NOTE: Both are flushed, even if only one is redirected).
 *   Before restoring sys.stdout & sys.stderr; then outfile & errfile are 
flushed (if both have been used then they are flushed in that order).

You would of course need to write the documentation clearer than I did above 
(writing documentation well is not my skill) -- I wrote it to convey exactly 
what has to happen.

On Fri, Sep 29, 2017 at 12:01 PM, Wren Turkal > 
wrote:

Steven and Amit,


I originally configured the mailing list for digest delivery and can't reply 
directly to his message. However, I now seen it, I will update the PR with the 
suggested name changes as soon as my "make test" finishes. FWIW, I've changed 
to direct message delivery.


With regard to ferr (the corresponding variable in the builtin_input c function 
before the change), I saw this code:


/* First of all, flush stderr */
tmp = _PyObject_CallMethodId(ferr, _flush, NULL);
if (tmp == NULL)
PyErr_Clear();
else
Py_DECREF(tmp);


And I assumed that it was important to be able to override a stderr as a result.


I think there are few options here to resolve that:

  1.  Remove the errfile parameter and just do what happened before.
  2.  Remove the errfile and the above code (FWIW, I am not sure I understand 
the importance of flushing stderr before taking input).
  3.  Document the explicit purpose of the errfile. Amit, you'd responded with 
something about this (again digest reply, sorry). I am not sure how to 
concisely do that given Amit's descriptions. 

I am honestly leaning toward 2 unless I can figure out why the flushing of 
stderr is actually needed.


wt



From: Amit Green >
Sent: Friday, September 29, 2017 8:13:21 AM

To: Wren Turkal
Cc: python-ideas@python.org
Subject: Re: 

Re: [Python-ideas] allow overriding files used for the input builtin

[Wren Turkal]

Oh, I thought that stderr was unbuffered. Like the following C program:


#include 

int main() {
  fprintf(stdout, "stdout0");
  fprintf(stderr, "stderr0");
  fprintf(stdout, "stdout1");
  return 0;
}


This outputs:

stderr0stdout0stdout1


Turns out fprintf in glibc will also implicitly flush at newlines also. So, the 
following program:

#include 

int main() {
  fprintf(stdout, "stdout0\nstdout12");
  fprintf(stderr, "stderr0");
  fprintf(stdout, "stdout1");
  return 0;
}


outputs:

stdout0
stderr0stdout12stdout1


This is in line with python. The following program:

import sys

sys.stdout.write('stdout0\nstdout01')
sys.stderr.write('stderr0')
sys.stdout.write('stdout1')


outputs:

stdout0
stderr0stdout01stdout1

While the following program:

import sys

sys.stdout.write('stdout0')
sys.stderr.write('stderr0')
sys.stdout.write('stdout1')


outputs:

stderr0stdout0stdout1


So, I guess that the errfile is explicitly flushed due to the fact that stdout 
may or may not be flushed during output.


So, I have 2 questions: (1) do we need to keep the errfile param to make this 
useful (I think we do), and if #1 is yes, (2) what is an appropriate way to doc 
this?


wt


From: Amit Green 
Sent: Friday, September 29, 2017 9:17:39 AM
To: Wren Turkal
Cc: Steven D'Aprano; python-ideas@python.org
Subject: Re: [Python-ideas] allow overriding files used for the input builtin

Hmm, very good point.

Flushing standard error is essential:


  *   This is because output of standard output & standard error are often 
redirected to the same file -- frequently the terminal -- and its important to 
flush one of them before output to the other (since these are typically line 
buffered you only see this if you output partial lines).

  *   Here is an example of not flushing & the mess it causes:

>>> import sys
>>> sys.stdout.write('hi'); sys.stderr.write(' HMM '); 
>>> sys.stdout.write('there\n')
 HMM hithere

  *   Here is the same example with flushing:

>>> sys.stdout.write('hi'); sys.stdout.flush(); sys.stderr.write(' HMM '); 
>>> sys.stderr.flush(); sys.stdout.write('there\n')
hi HMM there

In fact, I think you need to improve your interface and documentation:

  *   If either of the parameters 'outfile', or 'errfile' are passed in & not 
the 'none' value then the following happens:
 *   Both sys.stdout & sys.stderr are flushed (in that order) before being 
redirected.  (NOTE: Both are flushed, even if only one is redirected).
 *   Before restoring sys.stdout & sys.stderr; then outfile & errfile are 
flushed (if both have been used then they are flushed in that order).

You would of course need to write the documentation clearer than I did above 
(writing documentation well is not my skill) -- I wrote it to convey exactly 
what has to happen.

On Fri, Sep 29, 2017 at 12:01 PM, Wren Turkal > 
wrote:

Steven and Amit,


I originally configured the mailing list for digest delivery and can't reply 
directly to his message. However, I now seen it, I will update the PR with the 
suggested name changes as soon as my "make test" finishes. FWIW, I've changed 
to direct message delivery.


With regard to ferr (the corresponding variable in the builtin_input c function 
before the change), I saw this code:


/* First of all, flush stderr */
tmp = _PyObject_CallMethodId(ferr, _flush, NULL);
if (tmp == NULL)
PyErr_Clear();
else
Py_DECREF(tmp);


And I assumed that it was important to be able to override a stderr as a result.


I think there are few options here to resolve that:

  1.  Remove the errfile parameter and just do what happened before.
  2.  Remove the errfile and the above code (FWIW, I am not sure I understand 
the importance of flushing stderr before taking input).
  3.  Document the explicit purpose of the errfile. Amit, you'd responded with 
something about this (again digest reply, sorry). I am not sure how to 
concisely do that given Amit's descriptions. 

I am honestly leaning toward 2 unless I can figure out why the flushing of 
stderr is actually needed.


wt



From: Amit Green >
Sent: Friday, September 29, 2017 8:13:21 AM

To: Wren Turkal
Cc: python-ideas@python.org
Subject: Re: [Python-ideas] allow overriding files used for the input builtin

Yes, infile, outfile & errfile would be consistent with python naming 
convention (also mentioned by Steven D'Aprano above)

One of python's greatest strength is its library, the consistency of the 
library, and how well documented the library is (in fact, I think the library 
is a greater strength than even the very nice syntax of python in general).

By "consistency of the library" I mean: functions do pretty much what you 
expect, they use consistent error mechanism & the documentation pretty much 
accurately documents what the 

Re: [Python-ideas] allow overriding files used for the input builtin

[Wren Turkal]

Steven and Amit,


I originally configured the mailing list for digest delivery and can't reply 
directly to his message. However, I now seen it, I will update the PR with the 
suggested name changes as soon as my "make test" finishes. FWIW, I've changed 
to direct message delivery.


With regard to ferr (the corresponding variable in the builtin_input c function 
before the change), I saw this code:


/* First of all, flush stderr */
tmp = _PyObject_CallMethodId(ferr, _flush, NULL);
if (tmp == NULL)
PyErr_Clear();
else
Py_DECREF(tmp);


And I assumed that it was important to be able to override a stderr as a result.


I think there are few options here to resolve that:

  1.  Remove the errfile parameter and just do what happened before.
  2.  Remove the errfile and the above code (FWIW, I am not sure I understand 
the importance of flushing stderr before taking input).
  3.  Document the explicit purpose of the errfile. Amit, you'd responded with 
something about this (again digest reply, sorry). I am not sure how to 
concisely do that given Amit's descriptions. 

I am honestly leaning toward 2 unless I can figure out why the flushing of 
stderr is actually needed.


wt



From: Amit Green 
Sent: Friday, September 29, 2017 8:13:21 AM
To: Wren Turkal
Cc: python-ideas@python.org
Subject: Re: [Python-ideas] allow overriding files used for the input builtin

Yes, infile, outfile & errfile would be consistent with python naming 
convention (also mentioned by Steven D'Aprano above)

One of python's greatest strength is its library, the consistency of the 
library, and how well documented the library is (in fact, I think the library 
is a greater strength than even the very nice syntax of python in general).

By "consistency of the library" I mean: functions do pretty much what you 
expect, they use consistent error mechanism & the documentation pretty much 
accurately documents what the function does -- especially as to showing its 
results & how it handles errors.

Regarding this though, this then brings up the question (above from Steven 
D'Aprano) -- what would the the "errfile" parameter do?

  *   As a general principle of consistency python library functions, and 
python itself, do not output to errfile, but instead throw errors.
  *   (There are very minor exceptions such as exceptions thrown in __del__ 
functions; which are caught by python & then printed to standard error).

I would thus think you don't want the errfile parameter -- unless it would be 
for catching these __del__ method that get triggered by input failing (for 
example your 'infile' parameter when called, allocated an object, which gets 
deallocated & throws an exception inside of its __del__ method).

If this is the purpose, then (back to 'how well documented the library is') -- 
it should be documented this is the purpose of the "errfile" parameter ;-)

[A secondary reason you might want to redirect "errfile" is that the passed in 
input or output file's, themselves do output to standard error ...]


On Fri, Sep 29, 2017 at 10:50 AM, Wren Turkal > 
wrote:

I am happy to rename the args. What do you think about infile, outfile, and 
errfile?


FWIW, I did consider "in", "out", and "err", but "in" is a keyword, and I 
didn't think those quite captured the full meaning.


wt


From: Amit Green >
Sent: Thursday, September 28, 2017 11:18:18 PM
To: Wren Turkal
Cc: python-ideas@python.org
Subject: Re: [Python-ideas] allow overriding files used for the input builtin

I'm fine with the idea in general of extra keyword parameters to the input 
function.

A few points:

Your example code, needs try/catch to match what the input with parameters does 
-- and yes, its way nicer to be able to use it the example you have shown than 
play games with try/catch (Personally I also refuse to ever change sys.stdin, 
or sys.stdout, as I consider that a bad coding style).

Mostly though I would like to ask, please do not name keyword arguments with 
names like 'fin' & 'fout'.  This is almost unreadable and make's code almost 
indecipherable to others the first time they see the function & its keyword 
arguments (First impressions are very important).

Both a function name & its keyword parameters need to be as understandable as 
possible when a user encounters them for the first time.

On Fri, Sep 29, 2017 at 1:53 AM, Wren Turkal > 
wrote:

Hi there,


I have posted an idea for improvement with a PR of an implementation to 
https://bugs.python.org/issue31603.


The basic idea 

Re: [Python-ideas] PEP 560 (second post)

[Ivan Levkivskyi]

On 29 September 2017 at 08:57, Nick Coghlan  wrote:

> On 29 September 2017 at 08:04, Ivan Levkivskyi 
> wrote:
> >> How would you feel about calling it "__mro_entry__", as a mnemonic for
> >> "the substitute entry to use instead of this object when calculating a
> >> subclass MRO"?
> >
> > I don't have any preferences for the name, __mro_entry__ sounds equally
> OK
> > to me.
>
> I'd propose changing it then, as searching for "Python mro entry" is
> likely to get people to the right place faster than searching for
> "Python subclass base".
>
>
OK, will do.


> > I propose to update
> > ``type.__new__`` to just give
> > a better error message explaining this.
>
> +1 from me, since that avoids ever resolving the list of bases twice.
>
>
OK, I will update the reference implementation.


> > 3) Do we need to update types.new_class and types.prepare_class?
> > Here I am not sure. These functions are rather utility functions and are
> > designed to
> > mimic in Python what __build_class__ does in C. I think we might add
> > types._update_bases
> > that does the same as its C counterpart. Then we can update
> types.new_class
> > and types.prepare_class
> > like you proposed, this will preserve their current API while
> > types.new_class will match behaviour of __build_class__
>
> Your suggestion for `types.__new__` gave me a different idea: what if
> `types.prepare_class` *also* just raised an error when given a
> non-class as a nominal base class?
>
> Then if we added `types.resolve_bases` as a public API, a full
> reimplementation of `types.new_class` would now look like:
>
> resolved_bases = types.resolve_bases(bases)
> mcl, ns, updated_kwds = types.prepare_class(name, resolved_bases, kwds)
> exec_body(ns)
> ns["__orig_bases__"] = bases
> mcl(name, resolved_bases, ns, **updated_kwds)
>
> That way, `types.new_class` would transparently switch to the new
> behaviour, while clients of any other dynamic type creation API could
> do their own base class resolution, even if the type creation API they
> were using didn't implicitly support MRO entry resolution.
>
>
Yes, makes sense. I no one is against adding new public
``types.resolve_bases``
then I will add this to the PEP.

--
Ivan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 560 (second post)

[Ivan Levkivskyi]

On 29 September 2017 at 10:14, Victor Stinner 
wrote:

> > Title: Core support for generic types
>
> Would it be possible to mention "typing" somewhere in the title? If
> you don't know the context, it's hard to understand that the PEP is
> related to type annotation and type checks. At least just from the
> title.
>

What do you think about "Core support for typing module and generic types"?
Another option is "Runtime mechanism to improve generics and typing module".

--
Ivan
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] allow overriding files used for the input builtin

[Amit Green]

Yes, infile, outfile & errfile would be consistent with python naming
convention (also mentioned by Steven D'Aprano above)

One of python's greatest strength is its library, the consistency of the
library, and how well documented the library is (in fact, I think the
library is a greater strength than even the very nice syntax of python in
general).

By "consistency of the library" I mean: functions do pretty much what you
expect, they use consistent error mechanism & the documentation pretty much
accurately documents what the function does -- especially as to showing its
results & how it handles errors.

Regarding this though, this then brings up the question (above from Steven
D'Aprano) -- what would the the "errfile" parameter do?

   - As a general principle of consistency python library functions, and
   python itself, do not output to errfile, but instead throw errors.
   - (There are very minor exceptions such as exceptions thrown in __del__
   functions; which are caught by python & then printed to standard error).

I would thus think you don't want the errfile parameter -- unless it would
be for catching these __del__ method that get triggered by input failing
(for example your 'infile' parameter when called, allocated an object,
which gets deallocated & throws an exception inside of its __del__ method).

If this is the purpose, then (back to 'how well documented the library is')
-- it should be documented this is the purpose of the "errfile" parameter
;-)

[A secondary reason you might want to redirect "errfile" is that the passed
in input or output file's, themselves do output to standard error ...]



On Fri, Sep 29, 2017 at 10:50 AM, Wren Turkal  wrote:

> I am happy to rename the args. What do you think about infile, outfile,
> and errfile?
>
>
> FWIW, I did consider "in", "out", and "err", but "in" is a keyword, and I
> didn't think those quite captured the full meaning.
>
>
> wt
> --
> *From:* Amit Green 
> *Sent:* Thursday, September 28, 2017 11:18:18 PM
> *To:* Wren Turkal
> *Cc:* python-ideas@python.org
> *Subject:* Re: [Python-ideas] allow overriding files used for the input
> builtin
>
> I'm fine with the idea in general of extra keyword parameters to the input
> function.
>
> A few points:
>
> Your example code, needs try/catch to match what the input with parameters
> does -- and yes, its way nicer to be able to use it the example you have
> shown than play games with try/catch (Personally I also refuse to ever
> change sys.stdin, or sys.stdout, as I consider that a bad coding style).
>
> Mostly though I would like to ask, please do not name keyword arguments
> with names like 'fin' & 'fout'.  This is almost unreadable and make's code
> almost indecipherable to others the first time they see the function & its
> keyword arguments (First impressions are very important).
>
> Both a function name & its keyword parameters need to be as understandable
> as possible when a user encounters them for the first time.
>
> On Fri, Sep 29, 2017 at 1:53 AM, Wren Turkal  wrote:
>
>> Hi there,
>>
>>
>> I have posted an idea for improvement with a PR of an implementation to
>> https://bugs.python.org/issue31603
>> 
>> .
>>
>>
>> The basic idea is to add fin, fout, and ferr file object parameters and
>> default to using what is used today when the args are not specified. I
>> believe this would be useful to allow captures input and send output to
>> specific files when using input. The input builtin has some logic to use
>> readline if it's available. It would be nice to be able to use this same
>> logic no matter what files are being used for input/output.
>>
>>
>> This is meant to turn code like the following:
>>
>> orig_stdin = sys.stdin
>>
>> orig_stdout = sys.stdout
>>
>> with open('/dev/tty', 'r+') as f:
>>
>> sys.stdin = f
>>
>> sys.stdout = f
>>
>> name = input('Name? ')
>>
>> sys.stdin = orig_stdin
>>
>> sys.stdout = orig_stdout
>>
>> print(name)
>>
>>
>> into something more like this:
>>
>> with open('/dev/tty', 'r+') as f:
>>
>> name = input('Name? ', fin=f, fout=f)
>>
>> print(name)
>>
>>
>> It's nice that it makes the assignment to a global variable to change the
>> file used for input/output to no longer be needed.
>>
>>
>> I had this idea the other day, and I realized that it would be super easy
>> to implement, so I went ahead the threw up a PR also.
>>
>>
>> Would love to see if anyone else is interested in this. I think it's
>> pretty cool that the core logic really didn't need to be changed other than
>> plumbing in the new args.
>>
>>
>> FWIW, this change introduces no regressions and adds a few more tests to
>> test the new functionality. Honestly, I think this 

Re: [Python-ideas] allow overriding files used for the input builtin

[Wren Turkal]

I am happy to rename the args. What do you think about infile, outfile, and 
errfile?


FWIW, I did consider "in", "out", and "err", but "in" is a keyword, and I 
didn't think those quite captured the full meaning.


wt


From: Amit Green 
Sent: Thursday, September 28, 2017 11:18:18 PM
To: Wren Turkal
Cc: python-ideas@python.org
Subject: Re: [Python-ideas] allow overriding files used for the input builtin

I'm fine with the idea in general of extra keyword parameters to the input 
function.

A few points:

Your example code, needs try/catch to match what the input with parameters does 
-- and yes, its way nicer to be able to use it the example you have shown than 
play games with try/catch (Personally I also refuse to ever change sys.stdin, 
or sys.stdout, as I consider that a bad coding style).

Mostly though I would like to ask, please do not name keyword arguments with 
names like 'fin' & 'fout'.  This is almost unreadable and make's code almost 
indecipherable to others the first time they see the function & its keyword 
arguments (First impressions are very important).

Both a function name & its keyword parameters need to be as understandable as 
possible when a user encounters them for the first time.

On Fri, Sep 29, 2017 at 1:53 AM, Wren Turkal > 
wrote:

Hi there,


I have posted an idea for improvement with a PR of an implementation to 
https://bugs.python.org/issue31603.


The basic idea is to add fin, fout, and ferr file object parameters and default 
to using what is used today when the args are not specified. I believe this 
would be useful to allow captures input and send output to specific files when 
using input. The input builtin has some logic to use readline if it's 
available. It would be nice to be able to use this same logic no matter what 
files are being used for input/output.


This is meant to turn code like the following:

orig_stdin = sys.stdin

orig_stdout = sys.stdout

with open('/dev/tty', 'r+') as f:

sys.stdin = f

sys.stdout = f

name = input('Name? ')

sys.stdin = orig_stdin

sys.stdout = orig_stdout

print(name)


into something more like this:

with open('/dev/tty', 'r+') as f:

name = input('Name? ', fin=f, fout=f)

print(name)


It's nice that it makes the assignment to a global variable to change the file 
used for input/output to no longer be needed.


I had this idea the other day, and I realized that it would be super easy to 
implement, so I went ahead the threw up a PR also.


Would love to see if anyone else is interested in this. I think it's pretty 
cool that the core logic really didn't need to be changed other than plumbing 
in the new args.


FWIW, this change introduces no regressions and adds a few more tests to test 
the new functionality. Honestly, I think this functionality could probably be 
used to simplify some of the other tests as well, but I wanted to gauge what 
folks thought of the change before going farther.


Wren Turkal

Existential Production Engineer of the Ages

Facebook, Inc.

___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: 
http://python.org/psf/codeofconduct/


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Changes to the existing optimization levels

[Ned Batchelder]

On 9/28/17 2:48 PM, Diana Clarke wrote:

Hi folks:

I was recently looking for an entry-level cpython task to work on in
my spare time and plucked this off of someone's TODO list.

 "Make optimizations more fine-grained than just -O and -OO"

There are currently three supported optimization levels (0, 1, and 2).
Briefly summarized, they do the following.

 0: no optimizations
 1: remove assert statements and __debug__ blocks
 2: remove docstrings, assert statements, and __debug__ blocks




Don't forget that the current "no optimizations" setting actually does 
peephole optimizations.  Are we considering addressing 
https://bugs.python.org/issue2506 to make a really truly "no 
optimizations" option?


--Ned.
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] allow overriding files used for the input builtin

[Adam Bartoš]

Hello,

it seems that there are two independent issues – a way to temporarily
replace all sys.std* streams, and a way to use the special interactive
readline logic for arbitrary terminal-like file. I thought that OP's
concern was the latter. In that case shouldn't there just be a way to
produce an IO object f whose .readline() (or other special method) does
this and has an optional prompt argument? The OP's code would become

with open_terminal('/dev/tty') as f:
name = f.readline(prompt='Name? ')

It seems to me that changing the rather specialized input() function is
wrong in both cases. For me input() is just a short way to read from the
standard input, usually interactively – the same way print is a short way
to write to the standard output. IIRC there was even proposal to remove
input() in favor of direct use of sys.stdin.readline() at time of Python
3000 redesign.

Regards, Adam Bartoš
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] allow overriding files used for the input builtin

[Serhiy Storchaka]

29.09.17 08:53, Wren Turkal пише:

This is meant to turn code like the following:

orig_stdin = sys.stdin

orig_stdout = sys.stdout

with open('/dev/tty', 'r+') as f:

     sys.stdin = f

     sys.stdout = f

     name = input('Name? ')

sys.stdin = orig_stdin

sys.stdout = orig_stdout

print(name)


into something more like this:

with open('/dev/tty', 'r+') as f:

     name = input('Name? ', fin=f, fout=f)

print(name)


Why not use just the following two lines?

f.write('Name? ')
name = f.readline()

This falls to me in the category "not every two lines of the code should 
be added as a builtin".


___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] allow overriding files used for the input builtin

[Steven D'Aprano]

On Fri, Sep 29, 2017 at 05:53:58AM +, Wren Turkal wrote:

[...]
> The basic idea is to add fin, fout, and ferr file object parameters 
> and default to using what is used today when the args are not 
> specified. I believe this would be useful to allow captures input and 
> send output to specific files when using input. The input builtin has 
> some logic to use readline if it's available. It would be nice to be 
> able to use this same logic no matter what files are being used for 
> input/output.


I've done the whole "set stdout and stdin, call input, reset stdout and 
stdin" dance, and so I like this concept of being able to easily use 
input() non-interactively. I think your basic idea is a good one.

I wonder what you think the ferr parameter will do? As far as I know, 
input() doesn't use stderr.

I also don't think much of your parameter names. They strike me as very 
C-like, rather than Pythonic.

So my proposal is:

input(prompt, *, infile, outfile)

where infile must be a file-like object with a read() method, suitable 
for replacing stdin, and outfile must be a file-like object with a 
write() method suitable for replacing stdout.


> This is meant to turn code like the following:
> 
> orig_stdin = sys.stdin
> orig_stdout = sys.stdout
> with open('/dev/tty', 'r+') as f:
> sys.stdin = f
> sys.stdout = f
> name = input('Name? ')
> 
> sys.stdin = orig_stdin
> sys.stdout = orig_stdout
> print(name)

For production use, that should be wrapped in a try...finally:


try:
   ...
finally:
sys.stdin = orig_stdin
sys.stdout = orig_stdout


> into something more like this:
> 
> with open('/dev/tty', 'r+') as f:
> name = input('Name? ', fin=f, fout=f)
> 
> print(name)

I like it very much.

But as an alternative, perhaps all we really need is a context manager 
to set the std* files:

with open('/dev/tty', 'r+') as f:
with stdio(stdin=f, stdout=f):
name = input('Name? ')

print(name)


That's nearly as nice, and is possibly useful in more situations. Or 
maybe we should have both?


[...]
> Would love to see if anyone else is interested in this. I think it's 
> pretty cool that the core logic really didn't need to be changed other 
> than plumbing in the new args.

Definitely interested!



-- 
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 560 (second post)

[Victor Stinner]

> Title: Core support for generic types

Would it be possible to mention "typing" somewhere in the title? If
you don't know the context, it's hard to understand that the PEP is
related to type annotation and type checks. At least just from the
title.

Victor
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Changes to the existing optimization levels

[Nick Coghlan]

On 29 September 2017 at 05:02, Antoine Pitrou  wrote:
> On Thu, 28 Sep 2017 12:48:15 -0600
> Diana Clarke
>  wrote:
>>
>> 2) Added a new command line option N that allows you to specify
>> any number of individual optimization flags.
>>
>> For example:
>>
>> python -N nodebug -N noassert -N nodocstring
>
> We could instead reuse the existing -X option, which allows for
> free-form implementation-specific flags.

And declaring named optimisation flags to be implementation dependent
is likely a good way to go.

The one downside is that it would mean there was no formally
interpreter independent way of requesting the "nodebug,nodocstring"
configuration, but informal conventions around particular uses of "-X"
may be sufficient for that purpose.

>> I'm also not certain if the various compile signatures are even open
>> for change (int optimize => PyObject *optimizations), or if that's a
>> no-no.
>
> You probably want to keep the existing signatures for compatibility:
> - in C, add new APIs with the new convention
> - in Python, add a new (optional) function argument for the new
>   convention

This approach should also reduce the overall amount of code churn,
since any CPython (or external) code currently passing "optimize=-1"
won't need to change at all: that already says "get the optimization
settings from the interpreter state", so it will pick up any changes
to how that configuration works "for free".

That said, we may also want to consider a couple of other options
related to changing the meaning of *existing* parameters to these
APIs:

1. We have the PyCompilerFlags struct that's currently only used to
pass around feature flags for the __future__ module. It could gain a
second bitfield for optimisation options
2. We could reinterpret "optimize" as a bitfield instead of a regular
integer, special casing the already defined values:

- all zero: no optimizations
- sign bit set: negative -> use global settings
- 0x0001: nodebug+noassert
- 0x0002: nodebug+noassert+nodocstrings
- 0x0004: nodebug
- 0x0008: noassert
- 0x0010: nodocstrings

The "redefine optimizations as a bitfield" approach seems particularly
promising to me - it's a full integer, so even with all negative
numbers disallowed and the two low order bits reserved for the legacy
combinations, that's still 29 different optimisation flags given
32-bit integers. We currently have 3, so that's room for an 866%
increase in the number of defined flags :)

The opt-N values in pyc files would be somewhat cryptic-to-humans, but
still relatively easy to translate back to readable strings given the
bitfield values, and common patterns (like 0x14 -> 20 for
nodebug+nodocstrings) would likely become familiar pretty quickly.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] PEP 560 (second post)

[Nick Coghlan]

On 29 September 2017 at 08:04, Ivan Levkivskyi  wrote:
> On 28 September 2017 at 08:27, Nick Coghlan  wrote:
>>
>> On 27 September 2017 at 19:28, Ivan Levkivskyi 
>> wrote:
>> > If an object that is not a class object appears in the bases of a class
>> > definition, the ``__subclass_base__`` is searched on it. If found,
>> > it is called with the original tuple of bases as an argument. If the
>> > result
>> > of the call is not ``None``, then it is substituted instead of this
>> > object.
>> > Otherwise (if the result is ``None``), the base is just removed. This is
>> > necessary to avoid inconsistent MRO errors, that are currently prevented
>> > by
>> > manipulations in ``GenericMeta.__new__``. After creating the class,
>> > the original bases are saved in ``__orig_bases__`` (currently this is
>> > also
>> > done by the metaclass).
>>
>> How would you feel about calling it "__mro_entry__", as a mnemonic for
>> "the substitute entry to use instead of this object when calculating a
>> subclass MRO"?
>
> I don't have any preferences for the name, __mro_entry__ sounds equally OK
> to me.

I'd propose changing it then, as searching for "Python mro entry" is
likely to get people to the right place faster than searching for
"Python subclass base".

>> I think the other thing that needs to be clarified is whether or not
>> the actual metaclass can expect to receive an already-resolved
>> sequence of MRO entries as its list of bases, or if it will need to
>> repeat the base resolution process executed while figuring out the
>> metaclass.
>>
>
> There are three points for discussion here:
>
> 1) It is necessary to make the bases resolution soon, before the metaclass
> is calculated. This is why I do this at the beginning of __build_class__ in
> the
> reference implementation.

Indeed.

> 2) Do we need to update type.__new__ to be able to accept non-classes as
> bases?
> I think no. One might be a bit surprised that
>
> class C(Iterable[int]):
> pass
>
> works, but
>
> type('C', (Iterable[int],), {})
>
> fails with a metaclass conflict, but I think it is natural that static
> typing and dynamic
> class creation should not be used together. I propose to update
> ``type.__new__`` to just give
> a better error message explaining this.

+1 from me, since that avoids ever resolving the list of bases twice.

> 3) Do we need to update types.new_class and types.prepare_class?
> Here I am not sure. These functions are rather utility functions and are
> designed to
> mimic in Python what __build_class__ does in C. I think we might add
> types._update_bases
> that does the same as its C counterpart. Then we can update types.new_class
> and types.prepare_class
> like you proposed, this will preserve their current API while
> types.new_class will match behaviour of __build_class__

Your suggestion for `types.__new__` gave me a different idea: what if
`types.prepare_class` *also* just raised an error when given a
non-class as a nominal base class?

Then if we added `types.resolve_bases` as a public API, a full
reimplementation of `types.new_class` would now look like:

resolved_bases = types.resolve_bases(bases)
mcl, ns, updated_kwds = types.prepare_class(name, resolved_bases, kwds)
exec_body(ns)
ns["__orig_bases__"] = bases
mcl(name, resolved_bases, ns, **updated_kwds)

That way, `types.new_class` would transparently switch to the new
behaviour, while clients of any other dynamic type creation API could
do their own base class resolution, even if the type creation API they
were using didn't implicitly support MRO entry resolution.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] allow overriding files used for the input builtin

[Amit Green]

I'm fine with the idea in general of extra keyword parameters to the input
function.

A few points:

Your example code, needs try/catch to match what the input with parameters
does -- and yes, its way nicer to be able to use it the example you have
shown than play games with try/catch (Personally I also refuse to ever
change sys.stdin, or sys.stdout, as I consider that a bad coding style).

Mostly though I would like to ask, please do not name keyword arguments
with names like 'fin' & 'fout'.  This is almost unreadable and make's code
almost indecipherable to others the first time they see the function & its
keyword arguments (First impressions are very important).

Both a function name & its keyword parameters need to be as understandable
as possible when a user encounters them for the first time.

On Fri, Sep 29, 2017 at 1:53 AM, Wren Turkal  wrote:

> Hi there,
>
>
> I have posted an idea for improvement with a PR of an implementation to
> https://bugs.python.org/issue31603.
>
>
> The basic idea is to add fin, fout, and ferr file object parameters and
> default to using what is used today when the args are not specified. I
> believe this would be useful to allow captures input and send output to
> specific files when using input. The input builtin has some logic to use
> readline if it's available. It would be nice to be able to use this same
> logic no matter what files are being used for input/output.
>
>
> This is meant to turn code like the following:
>
> orig_stdin = sys.stdin
>
> orig_stdout = sys.stdout
>
> with open('/dev/tty', 'r+') as f:
>
> sys.stdin = f
>
> sys.stdout = f
>
> name = input('Name? ')
>
> sys.stdin = orig_stdin
>
> sys.stdout = orig_stdout
>
> print(name)
>
>
> into something more like this:
>
> with open('/dev/tty', 'r+') as f:
>
> name = input('Name? ', fin=f, fout=f)
>
> print(name)
>
>
> It's nice that it makes the assignment to a global variable to change the
> file used for input/output to no longer be needed.
>
>
> I had this idea the other day, and I realized that it would be super easy
> to implement, so I went ahead the threw up a PR also.
>
>
> Would love to see if anyone else is interested in this. I think it's
> pretty cool that the core logic really didn't need to be changed other than
> plumbing in the new args.
>
>
> FWIW, this change introduces no regressions and adds a few more tests to
> test the new functionality. Honestly, I think this functionality could
> probably be used to simplify some of the other tests as well, but I wanted
> to gauge what folks thought of the change before going farther.
>
>
> Wren Turkal
>
> Existential Production Engineer of the Ages
>
> Facebook, Inc.
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/