Re: [Python-ideas] Trial balloon: adding variable type declarations in support of PEP 484

With PEP 520 accepted, would it be possible to iterate over __definition_order__? class PrototypeNamedTuple: cache = {} def __new__(cls, *args): P = PrototypeNamedTuple if cls not in P.cache: P.cache[cls] = typing.NamedTuple(cls.__name__, [(definition, cls.__

Re: [Python-ideas] PEP 525: Asynchronous Generators

On 09.08.2016 05:23, Nick Coghlan wrote: On 9 August 2016 at 08:37, Sven R. Kunze > wrote: From what I've heard in the wild, that most if not all pieces of async are mirroring existing Python features. So, building async basically builds a parallel structure i

Re: [Python-ideas] PEP 525: Asynchronous Generators

Hi Brett, On 2016-08-10 12:27 PM, Brett Cannon wrote: [..] According to the inspect module that's a coroutine function that creates a coroutine/awaitable (and a function w/ @types.coroutine is just an awaitable when it contains a `yield`). Now the existence of `yield` in a function causing i

Re: [Python-ideas] Trial balloon: adding variable type declarations in support of PEP 484

Sounds like you're thinking with your runtime hat on, not your type checker hat. :-) On Tue, Aug 9, 2016 at 9:46 PM, Neil Girdhar wrote: > With PEP 520 accepted, would it be possible to iterate over > __definition_order__? > > class PrototypeNamedTuple: > cache = {} > def __new__(cls, *a

[Python-ideas] Fix default encodings on Windows

I suspect there's a lot of discussion to be had around this topic, so I want to get it started. There are some fairly drastic ideas here and I need help figuring out whether the impact outweighs the value. Some background: within the Windows API, the preferred encoding is UTF-16. This is a 16-

Re: [Python-ideas] Fix default encodings on Windows

On 10 August 2016 at 19:10, Steve Dower wrote: > To summarise the proposals (remembering that these would only affect Python > 3.6 on Windows): > > * change sys.getfilesystemencoding() to return 'utf-8' > * automatically decode byte paths assuming they are utf-8 > * remove the deprecation warning

Re: [Python-ideas] Fix default encodings on Windows

On Wed, Aug 10, 2016, at 14:10, Steve Dower wrote: > To summarise the proposals (remembering that these would only affect > Python 3.6 on Windows): > > * change sys.getfilesystemencoding() to return 'utf-8' > * automatically decode byte paths assuming they are utf-8 > * remove the deprecation war

Re: [Python-ideas] Fix default encodings on Windows

On 10Aug2016 1144, Paul Moore wrote: I presume you'd be targeting 3.7 for this change. Does 3.6 seem too aggressive? I think I have time to implement the changes before beta 1, as it's mostly changing default values and mopping up resulting breaks. (Doing something like reimplementing files

Re: [Python-ideas] Fix default encodings on Windows

On 10Aug2016 1146, Random832 wrote: On Wed, Aug 10, 2016, at 14:10, Steve Dower wrote: To summarise the proposals (remembering that these would only affect Python 3.6 on Windows): * change sys.getfilesystemencoding() to return 'utf-8' * automatically decode byte paths assuming they are utf-8 *

Re: [Python-ideas] Fix default encodings on Windows

On 10 August 2016 at 20:08, Steve Dower wrote: > On 10Aug2016 1144, Paul Moore wrote: >> >> I presume you'd be targeting 3.7 for this change. > > Does 3.6 seem too aggressive? I think I have time to implement the changes > before beta 1, as it's mostly changing default values and mopping up > resu

Re: [Python-ideas] Fix default encodings on Windows

On Wed, Aug 10, 2016, at 15:08, Steve Dower wrote: > Testing with obscure filenames and strings is where help will be needed > most :) How about filenames with invalid surrogates? For added fun, consider that the file system encoding is normally used with surrogateescape.

Re: [Python-ideas] Fix default encodings on Windows

On 10Aug2016 1226, Random832 wrote: On Wed, Aug 10, 2016, at 15:08, Steve Dower wrote: Testing with obscure filenames and strings is where help will be needed most :) How about filenames with invalid surrogates? For added fun, consider that the file system encoding is normally used with surrog

Re: [Python-ideas] Fix default encodings on Windows

On Wed, Aug 10, 2016, at 15:22, Steve Dower wrote: > > Why? What's the use case? [byte paths] > > Allowing library developers who support POSIX and Windows to just use > bytes everywhere to represent paths. Okay, how is that use case impacted by it being mbcs instead of utf-8? What about only d

Re: [Python-ideas] Fix default encodings on Windows

On Wed, Aug 10, 2016 at 6:10 PM, Steve Dower wrote: > Similarly, locale.getpreferredencoding() on Windows returns a legacy value - > the user's active code page - which should generally not be used for any > reason. The one exception is as a default encoding for opening files when no > other infor

Re: [Python-ideas] Fix default encodings on Windows

On 10 August 2016 at 21:16, eryk sun wrote: > IMO, Python needs a C implementation of the win_unicode_console > module, using the wide-character APIs ReadConsoleW and WriteConsoleW. > Note that this sets sys.std*.encoding as UTF-8 and transcodes, so > Python code never has to work directly with UT

Re: [Python-ideas] Fix default encodings on Windows

On Thu, Aug 11, 2016 at 6:09 AM, Random832 wrote: > On Wed, Aug 10, 2016, at 15:22, Steve Dower wrote: >> > Why? What's the use case? [byte paths] >> >> Allowing library developers who support POSIX and Windows to just use >> bytes everywhere to represent paths. > > Okay, how is that use case impa

Re: [Python-ideas] PEP 525: Asynchronous Generators

On Thu, Aug 11, 2016 at 3:17 AM, Sven R. Kunze wrote: > Lately, I talked to friend of mine about async and his initial reaction was > like "hmm that reminds me of early programming days, where you have to > explicitly tell the scheduler to get control back". He's much older than me, > so I think i

Re: [Python-ideas] PEP 525: Asynchronous Generators

I think over on async-sig my confusion over how this is for async generators and not coroutines came up. Now I think I know where my confusion stems from. I don't think this will change anything, but I wanted to get it out there as it might influence how we communicate things. Take Yury's simple e

Re: [Python-ideas] Fix default encodings on Windows

On Wed, 10 Aug 2016 at 11:16 Steve Dower wrote: > [SNIP] > > Finally, the encoding of stdin, stdout and stderr are currently > (correctly) inferred from the encoding of the console window that Python > is attached to. However, this is typically a codepage that is different > from the system codep

Re: [Python-ideas] Trial balloon: adding variable type declarations in support of PEP 484

On Wed, Aug 10, 2016 at 11:56 AM, Guido van Rossum wrote: > Sounds like you're thinking with your runtime hat on, not your type checker > hat. :-) > > On Tue, Aug 9, 2016 at 9:46 PM, Neil Girdhar wrote: >> >> With PEP 520 accepted, would it be possible to iterate over >> __definition_order__? St

Re: [Python-ideas] Fix default encodings on Windows

On Wed, Aug 10, 2016 at 8:09 PM, Random832 wrote: > On Wed, Aug 10, 2016, at 15:22, Steve Dower wrote: >> >> Allowing library developers who support POSIX and Windows to just use >> bytes everywhere to represent paths. > > Okay, how is that use case impacted by it being mbcs instead of utf-8? Usi

Re: [Python-ideas] Trial balloon: adding variable type declarations in support of PEP 484

On Wed, Aug 10, 2016 at 4:04 PM, Eric Snow wrote: [...] > > Still, it would be really nice to be able to introspect a class's > instance attributes at run-time. A stdlib helper for that would be > great, e.g. "inspect.get_inst_attrs(cls)". At one point a few years > back I wrote something like t

Re: [Python-ideas] Fix default encodings on Windows

On Wed, Aug 10, 2016, at 19:04, eryk sun wrote: > Using 'mbcs' doesn't work reliably with arbitrary bytes paths in > locales that use a DBCS codepage such as 932. Er... utf-8 doesn't work reliably with arbitrary bytes paths either, unless you intend to use surrogateescape (which you could also do

Re: [Python-ideas] Fix default encodings on Windows

On 10Aug2016 1431, Chris Angelico wrote: On Thu, Aug 11, 2016 at 6:09 AM, Random832 wrote: On Wed, Aug 10, 2016, at 15:22, Steve Dower wrote: Why? What's the use case? [byte paths] Allowing library developers who support POSIX and Windows to just use bytes everywhere to represent paths. Ok

Re: [Python-ideas] Fix default encodings on Windows

On 10Aug2016 1630, Random832 wrote: On Wed, Aug 10, 2016, at 19:04, eryk sun wrote: Using 'mbcs' doesn't work reliably with arbitrary bytes paths in locales that use a DBCS codepage such as 932. Er... utf-8 doesn't work reliably with arbitrary bytes paths either, unless you intend to use surro

Re: [Python-ideas] Fix default encodings on Windows

On Wed, Aug 10, 2016 at 11:30 PM, Random832 wrote: > Er... utf-8 doesn't work reliably with arbitrary bytes paths either, > unless you intend to use surrogateescape (which you could also do with > mbcs). > > Is there any particular reason to expect all bytes paths in this > scenario to be valid UT

Re: [Python-ideas] Fix default encodings on Windows

On Thu, Aug 11, 2016 at 9:40 AM, Steve Dower wrote: > On 10Aug2016 1431, Chris Angelico wrote: >> I'd rather a single consistent default encoding. > > I'm proposing to make that single consistent default encoding utf-8. It > sounds like we're in agreement? Yes, we are. I was disagreeing with Rand

Re: [Python-ideas] Fix default encodings on Windows

On Wed, Aug 10, 2016 at 11:40 PM, Steve Dower wrote: > It sounds like we'll have to look into effectively merging that package into > the core. I'm afraid that'll come with a much longer tail of bugs (and will > quite likely break code that expects to use file descriptors to access > stdin/out), b

Re: [Python-ideas] Fix default encodings on Windows

On Wed, Aug 10, 2016 at 04:40:31PM -0700, Steve Dower wrote: > On 10Aug2016 1431, Chris Angelico wrote: > >>* make the default open() encoding check for a BOM or else use utf-8 > > > >-0.5. Is there any precedent for this kind of data-based detection > >being the default? There is precedent: the

Re: [Python-ideas] PEP 525: Asynchronous Generators

On 11 August 2016 at 03:17, Sven R. Kunze wrote: > On 09.08.2016 05:23, Nick Coghlan wrote: > > On 9 August 2016 at 08:37, Sven R. Kunze wrote: > >> From what I've heard in the wild, that most if not all pieces of async >> are mirroring existing Python features. So, building async basically buil

Re: [Python-ideas] Fix default encodings on Windows

On 11 August 2016 at 04:10, Steve Dower wrote: > > I suspect there's a lot of discussion to be had around this topic, so I want > to get it started. There are some fairly drastic ideas here and I need help > figuring out whether the impact outweighs the value. My main reaction would be that if

Re: [Python-ideas] Fix default encodings on Windows

On 11 August 2016 at 13:26, Nick Coghlan wrote: > On 11 August 2016 at 04:10, Steve Dower wrote: >> >> I suspect there's a lot of discussion to be had around this topic, so I want >> to get it started. There are some fairly drastic ideas here and I need help >> figuring out whether the impact o

Re: [Python-ideas] PEP 525: Asynchronous Generators

On Thu, Aug 11, 2016 at 1:17 PM, Nick Coghlan wrote: >> Lately, I talked to friend of mine about async and his initial reaction >> was like "hmm that reminds me of early programming days, where you have to >> explicitly tell the scheduler to get control back". He's much older than me, >> so I thin

Re: [Python-ideas] Fix default encodings on Windows

On Thu, Aug 11, 2016 at 1:14 PM, Steven D'Aprano wrote: > On Wed, Aug 10, 2016 at 04:40:31PM -0700, Steve Dower wrote: > >> On 10Aug2016 1431, Chris Angelico wrote: >> >>* make the default open() encoding check for a BOM or else use utf-8 >> > >> >-0.5. Is there any precedent for this kind of data

Re: [Python-ideas] Fix default encodings on Windows

On Wed, Aug 10, 2016, at 17:31, Chris Angelico wrote: > AIUI, the data flow would be: Python bytes object Nothing _starts_ as a Python bytes object. It has to be read from somewhere or encoded in the source code as a literal. The scenario is very different for "defined internally within the progr