[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On 2/11/21, M.-A. Lemburg wrote: > On 11.02.2021 13:49, Eryk Sun wrote: > >> Currently, locale.getpreferredencoding(False) is implemented as >> locale._get_locale_encoding(). This ultimately calls >> _Py_GetLocaleEncoding(), defined in "Python/fileutils.c". >> TextIOWrapper() calls this C

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On 11.02.2021 13:49, Eryk Sun wrote: > On 2/11/21, M.-A. Lemburg wrote: > >> I think the main problem here is that open() doesn't use >> locale.getlocale()[1] as default for the encoding parameter, >> but instead locale.getpreferredencoding(False). > > Currently,

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On 2/11/21, M.-A. Lemburg wrote: > I think the main problem here is that open() doesn't use > locale.getlocale()[1] as default for the encoding parameter, > but instead locale.getpreferredencoding(False). Currently, locale.getpreferredencoding(False) is implemented as

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On 10.02.2021 23:10, Eryk Sun wrote: > On 2/10/21, M.-A. Lemburg wrote: >> >> setx PYTHONUTF8 1 >> >> does the trick in an admin command shell on Windows globally. > > The above command sets the variable only for the current user, which > I'd recommend anyway. It does not require administrator

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

I looked some Python courses for children. They won't use venvs. For example, they put .py file in a specified directory, then run it in the Minecraft or other graphical applications. Now I think we should promote putting PYTHONUTF8=1 in user environment before thinking about complex per-site

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On 2/11/21, Christopher Barker wrote: > On Wed, Feb 10, 2021 at 12:33 AM Paul Moore wrote: > >> So get PYTHONUTF8 added to the environment activate script. That's a >> simple change to venv. And virtualenv, and conda > > That's probably a good solution for venv and virtualenv -- essentially add

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Wed, Feb 10, 2021 at 12:33 AM Paul Moore wrote: > So get PYTHONUTF8 added to the environment activate script. That's a > simple change to venv. And virtualenv, and conda That's probably a good solution for venv and virtualenv -- essentially add it as another environment creation option.

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On 2/10/21, M.-A. Lemburg wrote: > > setx PYTHONUTF8 1 > > does the trick in an admin command shell on Windows globally. The above command sets the variable only for the current user, which I'd recommend anyway. It does not require administrator access. To set a machine value, run `setx /M

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Wed, 10 Feb 2021 at 13:31, Inada Naoki wrote: > > I'm sorry about it. I have not chose actual implementation yet so I > can not write concrete PEP yet. It's not a problem. I appreciate all of the time you're putting into considering the responses and keeping the discussion going. (And please

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On 10.02.2021 08:15, Christopher Barker wrote: > And all the examples given here of how to set environment variables and > shortcuts, etc on Windows is EXACTLY the kind of information  I don't want to > have to provide for my students :-( -- I'm teaching Python, not Windows > administration.

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Wed, Feb 10, 2021 at 8:39 PM Paul Moore wrote: > > On Wed, 10 Feb 2021 at 11:01, Inada Naoki wrote: > > > > On Wed, Feb 10, 2021 at 5:33 PM Paul Moore wrote: > > > > > > So get PYTHONUTF8 added to the environment activate script. That's a > > > simple change to venv. And virtualenv, and

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Wed, 10 Feb 2021 at 11:01, Inada Naoki wrote: > > On Wed, Feb 10, 2021 at 5:33 PM Paul Moore wrote: > > > > So get PYTHONUTF8 added to the environment activate script. That's a > > simple change to venv. And virtualenv, and conda - yes, it need to > > happen in multiple places, but that's

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Wed, Feb 10, 2021 at 5:33 PM Paul Moore wrote: > > So get PYTHONUTF8 added to the environment activate script. That's a > simple change to venv. And virtualenv, and conda - yes, it need to > happen in multiple places, but that's still easier IMO than proposing > a change to Python's already

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Wed, 10 Feb 2021 at 07:14, Christopher Barker wrote: > > On Tue, Feb 9, 2021 at 1:04 PM Paul Moore wrote: >> >> Why is that an issue? In the first instance, do the sorts of >> "beginner" we're discussing here have multiple python installs? Would >> they need per-interpreter configuration of

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Tue, Feb 9, 2021 at 1:04 PM Paul Moore wrote: > Why is that an issue? In the first instance, do the sorts of > "beginner" we're discussing here have multiple python installs? Would > they need per-interpreter configuration of UTF-8 mode? > yes -- many, many tutorials, particularly about web

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Wed, Feb 10, 2021 at 6:02 AM Paul Moore wrote: > > On Tue, 9 Feb 2021 at 17:32, Inada Naoki wrote: > > > > On Tue, Feb 9, 2021 at 7:42 PM M.-A. Lemburg wrote: > > > > > > Here's a good blog post about setting env vars on Windows: > > > > > >

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On 2/9/21, Inada Naoki wrote: > On Tue, Feb 9, 2021 at 7:42 PM M.-A. Lemburg wrote: > > But it affects to all Python installs. Can teachers recommend to set > PYTHONUTF8 environment variable for students? Users can simply create a shortcut that targets `cmd /k set PYTHONUTF8=1`. Optionally

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Tue, 9 Feb 2021 at 17:32, Inada Naoki wrote: > > On Tue, Feb 9, 2021 at 7:42 PM M.-A. Lemburg wrote: > > > > Here's a good blog post about setting env vars on Windows: > > > >

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Tue, Feb 9, 2021 at 12:28 AM Inada Naoki wrote: > It makes problem too hard, complex. It leads we can not fix anything > at all by Python 3.10. > We can add Unix support later if it is really worth enough. It is not > backward incompatible change. > yes and no -- if we don't anticipate

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Tue, Feb 9, 2021 at 7:42 PM M.-A. Lemburg wrote: > > Here's a good blog post about setting env vars on Windows: > > https://www.dowdandassociates.com/blog/content/howto-set-an-environment-variable-in-windows-command-line-and-registry/ > > It's not really much harder than on Unix platforms. >

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

> > > It's not really much harder than on Unix platforms. > > The only catch is that Windows users will often not know about such > env vars or how to use them, because on Windows you typically set up > your configuration via the application and using the registry. > Exactly -- it's not so much

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

Here's a good blog post about setting env vars on Windows: https://www.dowdandassociates.com/blog/content/howto-set-an-environment-variable-in-windows-command-line-and-registry/ It's not really much harder than on Unix platforms. The only catch is that Windows users will often not know about

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Tue, Feb 9, 2021 at 4:53 PM Christopher Barker wrote: > >> >> because UTF-8 mode helps many Windows users but it is not accessible >> enough for Windows users. > > > It's not just accessibility, but discoverability -- Windows users -- and even > more so developers that don't generally use

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Mon, Feb 8, 2021 at 10:49 PM Inada Naoki wrote: > I think it's MUCH better to have ONE way to do something that works, for > Python, on all platforms. That way people that only know one platform can > still write and document code that can work on all platforms. > > This thread is only for

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Tue, Feb 9, 2021 at 3:37 PM Christopher Barker wrote: > > On Mon, Feb 8, 2021 at 6:11 PM Inada Naoki wrote: >> >> > >> Unlike Windows, environment variables work very fine for such use cases. > > > Windows has environment variables, doesn't it? > But it isn't works well for Windows users.

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Mon, Feb 8, 2021 at 6:11 PM Inada Naoki wrote: > > > Unlike Windows, environment variables work very fine for such use cases. > Windows has environment variables, doesn't it? > On Unix, direnv, dotenv, and maybe more tools are there. I've been around Python for decades, and have never

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Tue, Feb 9, 2021 at 2:28 AM Christopher Barker wrote: > >> >> And beginners should use a UTF-8 locale. >> > Beginners may not know how to do that / have a choice. >> > >> > This is a question I still don't know the answer to -- I think that most >> > (all?) non Windows platforms currently

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Sun, Feb 7, 2021 at 11:19 PM Inada Naoki wrote: > > maybe, but we have many of the same issues -- we want the configuration > tied to the environment, not to the user and all environments. And I'd > rather have things done the same way on all platforms, rather than the > native way on each

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Mon, Feb 8, 2021 at 3:58 PM Christopher Barker wrote: > >> Should we support it in Unix? I don't think so. >> Command-line and environment variables are easy to use on Unix. > > > maybe, but we have many of the same issues -- we want the configuration tied > to the environment, not to the

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Sun, Feb 7, 2021 at 3:58 PM Inada Naoki wrote: > > Chiefly, I don't want to overload "pyvenv.cfg" with new behavior > > that's unrelated to virtual environments. > This is my point -- this is NOT unrelated to virtual environments -- UTF-8 mode, and other configuration parameters are very

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Sun, Feb 7, 2021 at 4:16 PM Eryk Sun wrote: > > On 2/6/21, Christopher Barker wrote: > > On Sat, Feb 6, 2021 at 11:47 AM Eryk Sun wrote: > > > >> Relative to the installation, "python.cfg" should only be found in the > >> same directory as the base executable, not its parent directory. > > >

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On 2/6/21, Christopher Barker wrote: > On Sat, Feb 6, 2021 at 11:47 AM Eryk Sun wrote: > >> Relative to the installation, "python.cfg" should only be found in the >> same directory as the base executable, not its parent directory. > > OK, my mistake — I thought that was already the case with

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On 2/6/21, Christopher Barker wrote: > On Fri, Feb 5, 2021 at 12:59 PM Eryk Sun wrote: > > But why limit it to that? If there are more things to configure in an > environment-specific way — why not put it in this existing location? I'd rather not limit the capability to just virtual

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Fri, Feb 5, 2021 at 12:59 PM Eryk Sun wrote: > I don't like extending "pyvenv.cfg" with generic settings. This is a > file to configure a virtual environment Yes indeed. in terms of finding the > standard library and packages. But why limit it to that? If there are more things to

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On 2/6/21, Inada Naoki wrote: > > If adding option to pyvenv.cfg is not make sense, we can add > `python.ini` to same place pyvenv.cfg. i.e., directory containing > python.exe, or one above directory. I'd rather look for "python.cfg" in the directory of the base executable (e.g. "C:\Program

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Sat, Feb 6, 2021 at 5:59 AM Eryk Sun wrote: > > > I would have preferred for the py launcher to read and merge settings > for all existing configuration files in the order of > "%ProgramData%\Python\py.ini" (all installations), > "%__AppDir__%\py.ini" (particular installation), and >

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On 2/5/21, Barry Scott wrote: >> On 5 Feb 2021, at 11:06, Inada Naoki wrote: > >> python.exe lookup pyvenv.cfg even outside of venv. >> So we can write utf8mode=1 in pyvenv.cfg even outside of venv. I don't like extending "pyvenv.cfg" with generic settings. This is a file to configure a virtual

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

> I'm talking about the user's override of the system default. This is indeed a limitation, but I’m not sure that’s a bad thing. As I think I’ve said before, utf-8 mode is probably not a feature you want turned on for ALL instances of Python, even all instances for a single user. So if a user

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

> On 5 Feb 2021, at 11:49, Inada Naoki wrote: > > On Fri, Feb 5, 2021 at 8:15 PM Barry Scott wrote: >> >>> >>> The main limitation is that users can not write config file in install >>> location when Python is installed for system, not for user. >> >> This is the problem that I was

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Fri, Feb 5, 2021 at 8:15 PM Barry Scott wrote: > > > > > The main limitation is that users can not write config file in install > > location when Python is installed for system, not for user. > > This is the problem that I was thinking about when I proposed using > a py.ini like solution where

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

> On 5 Feb 2021, at 11:06, Inada Naoki wrote: > > On Fri, Feb 5, 2021 at 7:59 PM Barry Scott wrote: >> >> I'm under the impression that new users will not create a venv. >> Indeed I run a lot of python scripts outside of venv world. >> I only use venv as part of my development pipe lines.

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Fri, Feb 5, 2021 at 7:59 PM Barry Scott wrote: > > I'm under the impression that new users will not create a venv. > Indeed I run a lot of python scripts outside of venv world. > I only use venv as part of my development pipe lines. > > I not sure that a venv cfg file would not help. > But a

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

> On 5 Feb 2021, at 03:11, Inada Naoki wrote: > > On Fri, Feb 5, 2021 at 6:17 AM Barry Scott wrote: >> >> Rather than reply point by point I will summarise my input. >> >> I think that utf-8 mode is a great idea. >> >> I think that an .INI file in the style that py.exe uses is better then

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Thu, Feb 4, 2021 at 7:12 PM Inada Naoki wrote: > I 100% agree with you. And pyvenv.cfg satisfies all your needs. > oops, sorry i missed this -- maybe pyvenv.cfg will do the job. Though I'm a bit confused about how it might work outside of venv itself, which usually creates the pyvenv.cfg

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Thu, Feb 4, 2021 at 1:17 PM Barry Scott wrote: > Rather than reply point by point I will summarise my input. > Thanks! > I think that utf-8 mode is a great idea. > agreed. > I'm hoping that the solution implemented allows new users to get a great > experience and also that advanced

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Fri, Feb 5, 2021 at 6:17 AM Barry Scott wrote: > > Rather than reply point by point I will summarise my input. > > I think that utf-8 mode is a great idea. > > I think that an .INI file in the style that py.exe uses is better then env > var. > > Env var on WIndows could be used but there can

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

Although a file adds I/O slowdown to startup (which is already slow) while an envvar doesn’t. On Thu, Feb 4, 2021 at 13:19 Barry Scott wrote: > > > > On 3 Feb 2021, at 02:49, Christopher Barker wrote: > > > > > Rather than reply point by point I will summarise my input. > > I think that utf-8

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

> On 3 Feb 2021, at 02:49, Christopher Barker wrote: > Rather than reply point by point I will summarise my input. I think that utf-8 mode is a great idea. I think that an .INI file in the style that py.exe uses is better then env var. Env var on WIndows could be used but there can be

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

> On 3 Feb 2021, at 02:49, Christopher Barker wrote: > > Aside: HTML 5 even has a encoding rule that acknowledges that web pages marked > utf-8 are really windows USA code page and show how to fall back! > > But that doesn't depend ina. system setting does it? So I don't get your > point:

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On 2/2/21, Christopher Barker wrote: > > In the common case, folks have their environment variables set in an > initialization file (or the registry? I've lost track of what Windows does > these days) It hasn't fundamentally changed since the mid 1990s. Configurable system variables are set in

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Tue, Feb 2, 2021 at 1:37 PM Barry Scott wrote: > 2) perhaps UTF-8 mode isn't the only use-case for this -- it would be good > to have a way to have Python startup parameters that are installation / > environment specific -- that could help with other issues with "global" > configuration:

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

> On 2 Feb 2021, at 19:45, Christopher Barker wrote: > > On Tue, Feb 2, 2021 at 11:12 AM Barry Scott > wrote: Where would Python look for a "configuration file like `pyvenv.cfg`" ? I am not a Windows expert so I am not sure. But I think it should

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Tue, Feb 2, 2021 at 11:12 AM Barry Scott wrote: > Where would Python look for a "configuration file like `pyvenv.cfg`" ? > > I am not a Windows expert so I am not sure. But I think it should be > the same directory where `python.exe` is in. > > A small note here -- ideally there would be

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

> On 2 Feb 2021, at 00:22, Inada Naoki wrote: > > On Tue, Feb 2, 2021 at 6:31 AM Barry Scott > wrote: >> >>> On 30 Jan 2021, at 12:05, Inada Naoki wrote: >>> >>> Where would Python look for a "configuration file like `pyvenv.cfg`" ? >>> >>> I am not a

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Tue, Feb 2, 2021 at 6:31 AM Barry Scott wrote: > >> On 30 Jan 2021, at 12:05, Inada Naoki wrote: >> >> Where would Python look for a "configuration file like `pyvenv.cfg`" ? >> >> I am not a Windows expert so I am not sure. But I think it should be >> the same directory where `python.exe` is

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

> On 30 Jan 2021, at 12:05, Inada Naoki wrote: > >> Where would Python look for a "configuration file like `pyvenv.cfg`" ? > > I am not a Windows expert so I am not sure. But I think it should be > the same directory where `python.exe` is in. You can put the system default there but each

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Sat, Jan 30, 2021 at 4:05 AM Inada Naoki wrote: > Sorry for the delay. I want to do it in Python 3.10, but I am not sure > the PEP is accepted. > I updated the PEP today and working on reference implementation now. > great, thanks! let us know if there's anything else we can do to help that

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Sat, Jan 30, 2021 at 3:45 PM Christopher Barker wrote: > > On Thu, Jan 28, 2021 at 4:25 PM Inada Naoki wrote: >> >> > The "real" solution is to change the defaults not to use the system >> > encoding at all -- which, of course, we are moving towards with PEP 597. >> > So first a plug to do

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Thu, Jan 28, 2021 at 4:25 PM Inada Naoki wrote: > > The "real" solution is to change the defaults not to use the system > encoding at all -- which, of course, we are moving towards with PEP 597. So > first a plug to do that as fast as possible! I myself would love to see PEP > 597 implemented

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Fri, Jan 29, 2021 at 12:54 PM Ben Rudiak-Gould wrote: > > On Wed, Jan 27, 2021 at 11:36 PM Inada Naoki wrote: >> >> * UnicodeDecodeError is raised when trying to open a text file written in >> UTF-8, such as JSON. >> * UnicodeEncodeError is raised when trying to save text data retrieved from

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Wed, Jan 27, 2021 at 11:36 PM Inada Naoki wrote: > * UnicodeDecodeError is raised when trying to open a text file written in > UTF-8, such as JSON. > * UnicodeEncodeError is raised when trying to save text data retrieved > from the web, etc. > * User run `pip install` and `setup.py` reads

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

On Fri, Jan 29, 2021 at 4:00 AM Christopher Barker wrote: > > The "real" solution is to change the defaults not to use the system encoding > at all -- which, of course, we are moving towards with PEP 597. So first a > plug to do that as fast as possible! I myself would love to see PEP 597 >

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

Thanks for working so hard to move this forward! The "real" solution is to change the defaults not to use the system encoding at all -- which, of course, we are moving towards with PEP 597. So first a plug to do that as fast as possible! I myself would love to see PEP 597 implemented tomorrow --