[Python-ideas] Re: Have del return a value

[Barry Scott]

> On 6 Sep 2023, at 03:47, Daniel Walker  wrote:
> 
> I use foo to instantiate another object:
> 
> bar = Bar(foo)
> 
> bar is free to manipulate foo however it wants and even del it if necessary. 

foo is a reference to an object, it is not an object in its own right.
You can say that bar is free to not keep a reference to the object that foo 
points to.
So no, bar cannot manipulate foo.

> However, since the foo name is still around, those resources won't get 
> cleaned up even if bar is done with them.  The simple solution is
> 
> bar = Bar(foo)
> del foo
> bar.do_stuff()

In depends on the scope that foo is in. If its a function then when you exit 
the function the ref-count
on the object that foo will be decremented for you.

Only if foo not going out of scope will you need the explitit del, for example 
in foo is defined at module level, etc.

I see no need for del to return anything, you already have the reference in foo.
The times that foo is dropped at module level are rare enough to not need 
special syntax.


Barry

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2GWNITBYV4GMPGWOHJVAN6L7LSQXXKXI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Proposal for get_or function in Python dictionaries

[Barry]

On 15 Jul 2023, at 23:14, Dom Grigonis  wrote:I am well aware of that.I just argue that it is not very pleasant to read.Personally, I am not a big user of it.C’s although maybe doesn’t have words to read, but is much more pleasant in both conciseness and logical sequence.Maybe it is not a big issue for others, but if C’s style conditional _expression_ existed, it would significantly change the way I write things.I agree with you and avoid the python version, and refactor it out of code.As you say ii also find it hard to read, and familiarity is not helping.The C and rust versions I do use.BarrySo just checking. :)On 16 Jul 2023, at 00:06, Paul Moore  wrote:On Sat, 15 Jul 2023 at 21:09, Dom Grigonis  wrote:So I would vote for something similar to:result = bar is None ? default : barresult = default if bar is None else barPython has a conditional _expression_ already.Paul 
___Python-ideas mailing list -- python-ideas@python.orgTo unsubscribe send an email to python-ideas-le...@python.orghttps://mail.python.org/mailman3/lists/python-ideas.python.org/Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/QARSVXWITOLFAXZZJLIEBSPFO35PRDDS/Code of Conduct: http://python.org/psf/codeofconduct/___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/QUUEWVIEBABWDJCCPCSQYNIGDS2MYGC4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: dict method to retrieve a key from a value

[Barry Scott]

> On 30 Jun 2023, at 02:50, Andre Delfino  wrote:
> 
> A dict method to retrieve the key of a value from a bijective dict would have 
> come in handy to me in several occasions:
> 
>>>> names = {'one': 1}
>>>> names.inverse()[1]
> 'one'
>>>> names = {'one': 1, 'uno': 1}
>>>> names.inverse()[1]
> ValueError: dict is not bijective
> 
> My usual use case is when both keys and values have simple types like 
> int/str, like:
> 
> {
> 'account-a': 123,
> 'account-b': 456,
> }
> 
> Users may enter the account name, or the ID. The system works with the 
> account ID (translating from the account name if needed), but when it needs 
> to mention the account to the user, it shows the account name instead.
> 
> I do understand that retrieval wouldn't be O(1).

In cases like this I have two dict one for each look up direction.

by_name = {}
by_id = {}

def add(name, id):
by_name[name] = id
by_id[id] = name

etc.

Barry


> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/BWMIASE5YCDETFSZYAUG7TVWTMBFHKQW/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NG6QFBDGHCJXKUFHCONO2U545WLI6N5F/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Warn when iterating over an already exhausted generator

[Barry]

> On 12 Jun 2023, at 16:55, BoppreH via Python-ideas  
> wrote:
> 
> Then the empty list creates hard-to-track bugs. I'm familiar with the 
> iterator protocol and why the behavior above happens, but couldn't it be 
> prevented?

I don’t think so. It is not always a bug that an iterator is empty.
For example this pattern:

args = iter(sys.argv)
progname = next(args)
mandatory_arg = next(args)
for arg in args:
 print(‘optional arg’, next(arg))

Your proposal will traceback for the for loop if there are no optional args.

Barry



___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/K4K32L226IAC22NOJLLWIHKC3WWF7SPG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add a .whitespace property to module unicodedata

[Barry]

> On 1 Jun 2023, at 19:10, David Mertz, Ph.D.  wrote:
> 
> %time unicode_whitespace = [chr(c) for c in range(0x) if 
> unicodedata.category(chr(c)) == "Zs"]

Try 0x10 to get all of unicode. 

Barry
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HMB57XHRGWRUJE4ZMULNBOGKD3ILH2DY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Mark a set of keys of a TypedDict as incompatible

[Barry]

On 17 May 2023, at 14:16, Sync In  wrote:I have a couple of {{{TypedDict}}}s for use in type hinting that are supposed to have some sets of incompatible keys.For example, if I need to specify that a particular {{{dict}}} can have {{{bar}}} or {{{baz}}} but not both of them, I can write two {{{TypedDict}}}s:{{{#!highlight pythonfrom typing import TypedDictclass Foo1(TypedDict):  bar: strclass Foo2(TypedDict):  baz: intFoo = Foo1 | Foo2foo_instance_1: Foo = {  # Works fine  'bar': 'foobar'}foo_instance_2: Foo = {  # Also fine  'baz': 42}foo_instance_3: Foo = {  # Warning: Expected type 'Foo1 | Foo2', got 'dict[str, str | int]' instead  'bar': 'foobar',  'baz': 42However, this doesn't scale very well if I were to have multiple sets; for example, a simple multiplication shows that three sets of 2, 3 and 4 keys each will result in 24 {{{TypedDict}}}s. A solution would be:{{{#!highlight pythonclass Foo(TypedDict):  bar: IncompatibleWith('baz', 'qux')[str]  baz: IncompatibleWith('bar', 'qux')[int]  qux: IncompatibleWith('bar', 'baz')[bool]}}}...or, with a decorator somewhat similar to {{{@overload}}}:{{{#!highlight python@incompatible('bar', 'baz', 'qux')# ...class Foo(TypedDict):  bar: str  baz: int  qux: bool}}}I asked this question on StackOverflow and was informed that such a feature doesn't exist. How about adding it to Python then?You may get a reply here but all the python type discussions take place on https://discuss.python.org these days. Suggest you ask there.Barry___Python-ideas mailing list -- python-ideas@python.orgTo unsubscribe send an email to python-ideas-le...@python.orghttps://mail.python.org/mailman3/lists/python-ideas.python.org/Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/TPILQMOHCWAD7GKDCREONOHZTFY5Z7YW/Code of Conduct: http://python.org/psf/codeofconduct/___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KXX2FASV4CXN2PSMC2RPX2NL5YH25WZC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Thoughts on allowing Path objects in shlex.join

[Barry]

> On 12 May 2023, at 18:54, mitchell.negus...@gmail.com wrote:
> 
> At the moment, `shlex.join` raises 
> 
> ```
> TypeError: expected string or bytes-like object
> ```
> 
> when given `pathlib.Path` objects. Since `subprocess.run` and related 
> subprocess commands allow `Path` since Python 3.6 and 3.8 for Windows, it 
> seems intuitive to me that `shlex.join` be able to handle paths as well. This 
> has been briefly discussed before on the bug tracker (issue 89293, link 
> below), but with the caveat that it was the wrong venue, and should be 
> brought up on this list instead. That said, I can't find any record that it 
> ever made it over here, but I am curious to get this community's perspective. 
> 
> My understanding of arguments against in that discussion were that 
> `shlex.join` should not implicitly convert its arguments to strings, and that 
> `Path` objects were not special enough for an exception. However (and as 
> noted over there), `subprocess` takes the exact opposite approach, and has an 
> explicit `isinstance` check to allow and convert `os.PathLike` objects. I'm 
> assuming that some of the reasoning behind allow path-to-string conversions 
> there has to do with the prevalence of using paths as command line arguments, 
> and that seems to me like it could be rationale enough for why `Path` objects 
> warrant an exception to the "don't-convert-inputs-to-strings" unwritten rule 
> for `shlex.join`. 
> 
> Without getting into the specific details of implementation, I'd imagine that 
> changing this to just convert `os.PathLike` to strings in `shlex.join` would 
> not break much existing code. I find it rather unlikely—though I'll admit I 
> have no supporting evidence—that anyone is relying on `shlex.join` to catch 
> path objects that should be strings; I'd bet instead anyone using paths 
> already is just converting them to strings "manually" before passing them to 
> `shlex.join`.
> 
> I'm curious to hear more opinions on this though, since I'm guessing there 
> may be very valid reasons that others have for not wanting to move in this 
> direction that I'm totally oblivious to.
> 
> Thanks all!
> 
> 
> Issue 89283: https://github.com/python/cpython/issues/89293

You may find that you need to raise on as an idea on 
https://discuss.python.org/ as that is where the python code dev are more 
likely to read this idea.

Barry
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/UJYYYI3VQKXN46X5QQTAAS5RORIXJSMF/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4Q4DSM2IXXAH5ITF3J5ZFP34RMPCODZI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: XDG Directory Support for PDB

[Barry]

On 20 Apr 2023, at 16:52, Gustaf Waldemarson  wrote:> Oh i see you read all the files.> Usually you need to stop looking once you find an rc file.I guess that really depends on the program, but since pdb is modelled ongdb, I figured it made sense to do something similar :)Turns out the gdbinit behaviour is complex.2.1.4.3 Home directory initialization fileAfter loading the system wide initialization files GDB will look for an initialization file in the users home directory2. There are a number of locations that GDB will search in the home directory, these locations are searched in order and GDB will load the first file that it finds, and subsequent locations will not be checked.On non-Apple hosts the locations searched are:$XDG_CONFIG_HOME/gdb/gdbinit$HOME/.config/gdb/gdbinit$HOME/.gdbinitThen ./.gdbinit is loaded if . Is not ~/And you need different logic for macOS.Anyways, I'll prep a pull-request for the cpython repo, should be interestingto see what comes up.Best regards,GustafDen tors 20 apr. 2023 kl 17:05 skrev Barry <ba...@barrys-emacs.org>:On 20 Apr 2023, at 14:24, Gustaf Waldemarson <gustaf.waldemar...@gmail.com> wrote:Interesting, are you both sure ./.pdbrc should be the first file? I was expectingthe order to always go from "most general" to "most specific", i.e.,"system" -> "home" -> "current". This way, it would be straightforwardto have general configuration in a "system"/"home" configuration which canbe overwritten in a more specialized one ("current").Oh i see you read all the files.Usually you need to stop looking once you find an rc file.BarryCoincidentally, this is also the order GDB is using:   https://sourceware.org/gdb/current/onlinedocs/gdb#StartupAny particular reason you guys prefer a different order?
> Not your job, but I bet you get "are there other stdlib applications 
> that would benefit from this change?" as feedback.  Eg, IDLE.Probably, but lets start the discussion somewhere!Best regards,GustafDen ons 19 apr. 2023 kl 17:11 skrev turnbull <turnb...@sk.tsukuba.ac.jp>:I apologize for the top-post and awkward trimming, my employer just 
switched to o365 and won't accomodate my existing mail client so I'm 
stuck with a crappy webmail client on this address. :-p

I agree with Barry on both points: seems like a good idea, but the 
search priority should be
./.pdbrc first.

Not your job, but I bet you get "are there other stdlib applications 
that would benefit from this change?" as feedback.  Eg, IDLE.

2023-04-19 02:50 に Barry さんは書きました:

>>    +            for rcFileName in ('${XDG_CONFIG_HOME}/python/pdbrc',
>>    +                               '~/.config/python/pdbrc',
>>    +                               '~/.pdbrc',
>>    +                               '.pdbrc'):
> I would expect .pdbrc to be first.
> 
>>    +                rcPath = 
>> os.path.expanduser(os.path.expandvars(rcFileName))
>>    +                try:
>>    +                    with open(rcPath, encoding='utf-8') as rcFile:
>>    +                        self.rcLines.extend(rcFile)
>>    +                except OSError:
>>    +                    pass
>> 
>>             self.commands = {} # associates a command list to 
>> breakpoint numbers
>>             self.commands_doprompt = {} # for each bp num, tells if 
>> the prompt
> 
> Try raising a python bug and a PR with this code in it.
> Seems reasonable change to support.
> 
> Barry


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZWIKD7QFIWTWTFPB6RAFJTZEYAHQZCU4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: XDG Directory Support for PDB

[Barry]

On 20 Apr 2023, at 16:52, Gustaf Waldemarson  wrote:> Oh i see you read all the files.> Usually you need to stop looking once you find an rc file.I guess that really depends on the program, but since pdb is modelled ongdb, I figured it made sense to do something similar :)Turns out the gdbinit behaviour is complex.From https://sourceware.org/gdb/onlinedocs/gdb/Initialization-Files.html#Initialization-Files2.1.4.3 Home directory initialization fileAfter loading the system wide initialization files GDB will look for an initialization file in the users home directory2. There are a number of locations that GDB will search in the home directory, these locations are searched in order and GDB will load the first file that it finds, and subsequent locations will not be checked.On non-Apple hosts the locations searched are:$XDG_CONFIG_HOME/gdb/gdbinit$HOME/.config/gdb/gdbinit$HOME/.gdbinitThen ./.gdbinit is loaded if . Is not ~/And you need different logic for macOS.Anyways, I'll prep a pull-request for the cpython repo, should be interestingto see what comes up.Best regards,GustafDen tors 20 apr. 2023 kl 17:05 skrev Barry <ba...@barrys-emacs.org>:On 20 Apr 2023, at 14:24, Gustaf Waldemarson <gustaf.waldemar...@gmail.com> wrote:Interesting, are you both sure ./.pdbrc should be the first file? I was expectingthe order to always go from "most general" to "most specific", i.e.,"system" -> "home" -> "current". This way, it would be straightforwardto have general configuration in a "system"/"home" configuration which canbe overwritten in a more specialized one ("current").Oh i see you read all the files.Usually you need to stop looking once you find an rc file.BarryCoincidentally, this is also the order GDB is using:   https://sourceware.org/gdb/current/onlinedocs/gdb#StartupAny particular reason you guys prefer a different order?
> Not your job, but I bet you get "are there other stdlib applications 
> that would benefit from this change?" as feedback.  Eg, IDLE.Probably, but lets start the discussion somewhere!Best regards,GustafDen ons 19 apr. 2023 kl 17:11 skrev turnbull <turnb...@sk.tsukuba.ac.jp>:I apologize for the top-post and awkward trimming, my employer just 
switched to o365 and won't accomodate my existing mail client so I'm 
stuck with a crappy webmail client on this address. :-p

I agree with Barry on both points: seems like a good idea, but the 
search priority should be
./.pdbrc first.

Not your job, but I bet you get "are there other stdlib applications 
that would benefit from this change?" as feedback.  Eg, IDLE.

2023-04-19 02:50 に Barry さんは書きました:

>>    +            for rcFileName in ('${XDG_CONFIG_HOME}/python/pdbrc',
>>    +                               '~/.config/python/pdbrc',
>>    +                               '~/.pdbrc',
>>    +                               '.pdbrc'):
> I would expect .pdbrc to be first.
> 
>>    +                rcPath = 
>> os.path.expanduser(os.path.expandvars(rcFileName))
>>    +                try:
>>    +                    with open(rcPath, encoding='utf-8') as rcFile:
>>    +                        self.rcLines.extend(rcFile)
>>    +                except OSError:
>>    +                    pass
>> 
>>             self.commands = {} # associates a command list to 
>> breakpoint numbers
>>             self.commands_doprompt = {} # for each bp num, tells if 
>> the prompt
> 
> Try raising a python bug and a PR with this code in it.
> Seems reasonable change to support.
> 
> Barry


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GSAXO2JZK4EAWOLTM5SLDYJKQSBB4KJW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: XDG Directory Support for PDB

[Barry]

On 20 Apr 2023, at 14:24, Gustaf Waldemarson  wrote:Interesting, are you both sure ./.pdbrc should be the first file? I was expectingthe order to always go from "most general" to "most specific", i.e.,"system" -> "home" -> "current". This way, it would be straightforwardto have general configuration in a "system"/"home" configuration which canbe overwritten in a more specialized one ("current").Oh i see you read all the files.Usually you need to stop looking once you find an rc file.BarryCoincidentally, this is also the order GDB is using:   https://sourceware.org/gdb/current/onlinedocs/gdb#StartupAny particular reason you guys prefer a different order?
> Not your job, but I bet you get "are there other stdlib applications 
> that would benefit from this change?" as feedback.  Eg, IDLE.Probably, but lets start the discussion somewhere!Best regards,GustafDen ons 19 apr. 2023 kl 17:11 skrev turnbull <turnb...@sk.tsukuba.ac.jp>:I apologize for the top-post and awkward trimming, my employer just 
switched to o365 and won't accomodate my existing mail client so I'm 
stuck with a crappy webmail client on this address. :-p

I agree with Barry on both points: seems like a good idea, but the 
search priority should be
./.pdbrc first.

Not your job, but I bet you get "are there other stdlib applications 
that would benefit from this change?" as feedback.  Eg, IDLE.

2023-04-19 02:50 に Barry さんは書きました:

>>    +            for rcFileName in ('${XDG_CONFIG_HOME}/python/pdbrc',
>>    +                               '~/.config/python/pdbrc',
>>    +                               '~/.pdbrc',
>>    +                               '.pdbrc'):
> I would expect .pdbrc to be first.
> 
>>    +                rcPath = 
>> os.path.expanduser(os.path.expandvars(rcFileName))
>>    +                try:
>>    +                    with open(rcPath, encoding='utf-8') as rcFile:
>>    +                        self.rcLines.extend(rcFile)
>>    +                except OSError:
>>    +                    pass
>> 
>>             self.commands = {} # associates a command list to 
>> breakpoint numbers
>>             self.commands_doprompt = {} # for each bp num, tells if 
>> the prompt
> 
> Try raising a python bug and a PR with this code in it.
> Seems reasonable change to support.
> 
> Barry

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/45VRIN5V7ZJLG7HZC5P57VHQVMQ3JWCM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: XDG Directory Support for PDB

[Barry]

> On 18 Apr 2023, at 14:20, Gustaf Waldemarson  
> wrote:
> 
> Hello,
> 
> I don't really use the Python debugger all that much, but recently I used it
> quite a bit for a small side-project. That said, it kept annoying me that 
> there
> were no history associated with the prompt, so I added a few lines to the
> `pdbrc` config file to store that for me. However, it seems like this file 
> must
> either exist in the user home directory, or the current working directory,
> neither of which were very convenient for me.
> 
> Additionally, I strongly dislike polluting my home directory with dotfiles, 
> thus
> I did a bit of researching to see if this could be fixed. Thus, below is a 
> small
> patch to PDB that adds support for **also** reading from configuration files 
> in
> some XDG directories (currently ${XDG_CONFIG_HOME}/python/pdbrc).
> 
> So I guess the question is, would anyone else find this useful enough to the
> point were I should write a PEP for it? As far as I can tell, Python haven't
> been using the XDG directories, so I suppose the naming of the subdirectory
> is really subject to a larger discussion.
> 
>From d4ea8fbba3ba6c7538ccecad2a32e6b3b2059e5d Mon Sep 17 00:00:00 2001
>From: Gustaf Waldemarson 
>Date: Thu, 13 Apr 2023 11:39:00 +0200
>Subject: [PATCH] Allow users to store PDB configuration in XDG directories.
> 
>---
> Lib/pdb.py | 22 +++---
> 1 file changed, 11 insertions(+), 11 deletions(-)
> 
>diff --git a/Lib/pdb.py b/Lib/pdb.py
>index a3553b345a..3f4ffd7719 100755
>--- a/Lib/pdb.py
>+++ b/Lib/pdb.py
>@@ -231,19 +231,19 @@ def __init__(self, completekey='tab', stdin=None, 
> stdout=None, skip=None,
> self.allow_kbdint = False
> self.nosigint = nosigint
> 
>-# Read ~/.pdbrc and ./.pdbrc
>+# Read user configuration from a set of paths.
> self.rcLines = []
> if readrc:
>-try:
>-with open(os.path.expanduser('~/.pdbrc'), 
> encoding='utf-8') as rcFile:
>-self.rcLines.extend(rcFile)
>-except OSError:
>-pass
>-try:
>-with open(".pdbrc", encoding='utf-8') as rcFile:
>-self.rcLines.extend(rcFile)
>-except OSError:
>-pass
>+for rcFileName in ('${XDG_CONFIG_HOME}/python/pdbrc',
>+   '~/.config/python/pdbrc',
>+   '~/.pdbrc',
>+   '.pdbrc'):
I would expect .pdbrc to be first.

>+rcPath = 
> os.path.expanduser(os.path.expandvars(rcFileName))
>+try:
>+with open(rcPath, encoding='utf-8') as rcFile:
>+self.rcLines.extend(rcFile)
>+    except OSError:
>+pass
> 
> self.commands = {} # associates a command list to breakpoint 
> numbers
> self.commands_doprompt = {} # for each bp num, tells if the prompt

Try raising a python bug and a PR with this code in it.
Seems reasonable change to support.

Barry

>-- 
>2.25.1
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/HWWWESO3KUCU54OU5FISD3RKLEZMKBP7/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WCO4GTZUYTHQ4CVS72OWEPEP22RO7PB7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Auto dedent -c arguments.

[Barry]

On 10 Apr 2023, at 02:18, Jonathan Crall  wrote:There's no question that there are lots of ways you can work around the issue (I think the `if 1:` method is the easiest and most portable, but it still is boilerplate which can be confusing if you don't know why it's there). The question is: would enough people benefit from this to add the feature to CPython?A PR that has an implementation might push this forward.I wonder if in the C code of -c treating a string that starts with \n could trigger the dedent?BarryOn Sun, Apr 9, 2023 at 4:09 PM Barry Scott <ba...@barrys-emacs.org> wrote:
  

  
  


On 04/04/2023 15:18, Jonathan Crall
  wrote:


  
  I have what I think is a fairly low impact quality
of life improvement to suggest for the python CLI.


When I'm not working in Python I tend to be working in
  bash. But often I want to break out and do something quick in
  Python. I find the `python -c ` CLI very useful for this. For
  one liners it's perfect. E.g.



NEW_VAR=$(python -c "import pathlib;
print(pathlib.Path('$MYVAR').parent.parent)")


And even if I want to do something multi-line it's pretty
  easy


NEW_VAR=$(python -c "
import pathlib
for _ in range(10):
    print('this is a demo, bear with
me')
")


But the problem is when I'm writing bash inside a function
  or some other nested code, I would like to have nice
  indentation in my bash file, but if I write something like
  this:

  
mybashfunc(){
    python -c "
    import pathlib
    for _ in range(10):
        print('this is a demo, bear with me')
    "
}



I get `IndentationError: unexpected indent`. 


This means I have to write the function ugly like this:
  
  mybashfunc(){
    python -c "
import pathlib
for _ in range(10):
    print('this is a demo, bear with me')
"
}



Or use a helper function like this:
  
  codeblock()
{
    __doc__='
    copy-pastable implementation
    Prevents indentation errors in bash
    '
    echo "$1" | python -c "import sys; from textwrap import
dedent; print(dedent(sys.stdin.read()).strip('\n'))"
}
  

  
mybashfunc(){
    python -c $(codeblock "
    import pathlib
    for _ in range(10):
        print('this is a demo, bear with me')
    ")
}



Or more recently I found that this is a low-impact
  workaround: 
  
mybashfunc(){
    python -c "if 1:
    import pathlib
    for _ in range(10):
        print('this is a demo, bear with me')
    "
}



But as a certain Python dev may say: "There must be a
  better way."


Would there be any downside to the Python CLI automatically
  dedenting the input string given to -c? I can't think of any
  case off the top of my head where it would make a previously
  valid program invalid. Unless I'm missing something this would
  strictly make previously invalid strings valid.


Thoughts?
  

You can solve this with a small module. I named mine run_dedent
mkdir -p run_dedent
  echo pass >run_dedent/__init__.py
  cat <run_dedent/__main__.py
  import sys
  import textwrap
  cmd = textwrap.dedent(sys.argv[1])
  exec(cmd)
  EOF
    python3 -m run_dedent "
      import sys
          print(sys.version_info)
          "
Barry



  


-- 
  

  

  -Dr. Jon Crall (him)

  

  

  
  
  
  ___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/

[Python-ideas] Re: Auto dedent -c arguments.

[Barry Scott]

On 04/04/2023 15:18, Jonathan Crall wrote:
I have what I think is a fairly low impact quality of life improvement 
to suggest for the python CLI.


When I'm not working in Python I tend to be working in bash. But often 
I want to break out and do something quick in Python. I find the 
`python -c ` CLI very useful for this. For one liners it's perfect. E.g.


NEW_VAR=$(python -c "import pathlib; 
print(pathlib.Path('$MYVAR').parent.parent)")


And even if I want to do something multi-line it's pretty easy

NEW_VAR=$(python -c "
import pathlib
for _ in range(10):
    print('this is a demo, bear with me')
")

But the problem is when I'm writing bash inside a function or some 
other nested code, I would like to have nice indentation in my bash 
file, but if I write something like this:


mybashfunc(){
    python -c "
    import pathlib
    for _ in range(10):
        print('this is a demo, bear with me')
    "
}

I get `IndentationError: unexpected indent`.

This means I have to write the function ugly like this:

mybashfunc(){
    python -c "
import pathlib
for _ in range(10):
    print('this is a demo, bear with me')
"
}

Or use a helper function like this:

codeblock()
{
    __doc__='
    copy-pastable implementation
    Prevents indentation errors in bash
    '
    echo "$1" | python -c "import sys; from textwrap import dedent; 
print(dedent(sys.stdin.read()).strip('\n'))"

}

mybashfunc(){
    python -c $(codeblock "
    import pathlib
    for _ in range(10):
        print('this is a demo, bear with me')
    ")
}

Or more recently I found that this is a low-impact workaround:

mybashfunc(){
    python -c "if 1:
    import pathlib
    for _ in range(10):
        print('this is a demo, bear with me')
    "
}

But as a certain Python dev may say: "There must be a better way."

Would there be any downside to the Python CLI automatically 
dedenting the input string given to -c? I can't think of any case off 
the top of my head where it would make a previously valid program 
invalid. Unless I'm missing something this would strictly make 
previously invalid strings valid.


Thoughts?


You can solve this with a small module. I named mine run_dedent

mkdir -p run_dedent
echo pass >run_dedent/__init__.py
cat <run_dedent/__main__.py
import sys
import textwrap
cmd = textwrap.dedent(sys.argv[1])
exec(cmd)
EOF

    python3 -m run_dedent "
    import sys
        print(sys.version_info)
        "

Barry




--
-Dr. Jon Crall (him)

___
Python-ideas mailing list --python-ideas@python.org
To unsubscribe send an email topython-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived 
athttps://mail.python.org/archives/list/python-ideas@python.org/message/6SYN2MQIP5DTF6INZ4SP2YKLF6P4VJOW/
Code of Conduct:http://python.org/psf/codeofconduct/___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/E2XIQO2KVLARXF6LUUCOELH56I56F6DW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Make decorator-based context managers reentrant

[Barry]

> On 30 Mar 2023, at 08:10, Tobias Bengfort  wrote:
> 
> Hi,
> 
> I was wondering why decorator-based context managers are not reentrant or 
> even reusable.

Do you mean this use of reentrant? 
https://en.wikipedia.org/wiki/Reentrancy_(computing)


> 
> They could be made reusable by initializing the generator in __enter__() 
> instead of __init__(). They could be made reentrant by storing generatos in a 
> stack. A similar thing has been implemented for redirect_stdout in 
> 8e113b418df7d0c8480e1e2de29a385e1f31b15b.
> 
> I understand that not all generator-based context managers can be reentrant 
> by their nature. But currently it is impossible to make them reentrant, which 
> IMHO is an unnecessary restriction.
> 
> I could find very little discussion on this. Are there any strong reasons why 
> it was implemented this way or should I create a pull request to change it?
> 
> (My specific use case is a TUI application where I want to make sure that the 
> terminal is restored to a usable state on exit, but also need to step in and 
> out of the TUI on SIGTSTP/SIGCONT.)
> 
> thanks,
> tobias
> 
> https://docs.python.org/3/library/contextlib.html#single-use-reusable-and-reentrant-context-managers
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/K7M62JLBCR5USTJV5IRILJNQEYSJQ4QY/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/C7DQF5ZXLFVLVOJUCEPLHJ4MH4J5I2MV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: join() could add separators at start or end

[Barry]

> On 9 Mar 2023, at 00:37, Rob Cliffe via Python-ideas 
>  wrote:
> 
> Having had my last proposal shot down in flames, up I bob with another. 😁

See this discussion that has a nice solution proposed with the concat function.

Barry

> It seems to me that it would be useful to be able to make the str.join() 
> function put separators, not only between the items of its operand, but also 
> optionally at the beginning or end.
> E.g. '|'.join(('Spam', 'Ham', 'Eggs')) returns
> 'Spam|Ham|Eggs'
> but it might be useful to make it return one of
> '|Spam|Ham|Eggs'
> 'Spam|Ham|Eggs|'
> '|Spam|Ham|Eggs|'
> Again, I suggest that this apply to byte strings as well as strings.
> Going through the 3.8.3 stdlib I have found
> 24 examples where the separator needs to be added at the beginning
> 52 where the separator needs to be added at the end
>  4 where the separator needs to be added at the both ends
> I list these examples below.  Apologies if there are any mistakes.
> 
> While guessing is no substitute for measurement, it seems plausible that 
> using this feature
> where appropriate would increase runtime performance by avoiding 1 (or 2) 
> calls of str.__add__.
> This is perhaps more relevant when the separator is not a short constant 
> string,
> as in this example:
> Lib\email\_header_value_parser.py:2854:return 
> policy.linesep.join(lines) + policy.linesep
> Note also this example:
> Lib\site-packages\setuptools\command\build_ext.py:221: pkg = 
> '.'.join(ext._full_name.split('.')[:-1] + [''])
> where the author has used the unintuitive device of appending an empty string
> to a list to force join() to add an extra final dot, thereby avoiding 1 call 
> of str.__add__
> at the cost of 1 call of list.append.
> 
> What I cannot decide is what the best API would be.
> str.join() currently takes only 1 parameter, so it would be possible to add 
> an extra parameter or two.
> One scheme would be to have an atEnds parameter which could take values such 
> as
> 0=default behaviour  1=add sep at start  2=add sep at end  3=add sep at 
> both ends
> or
> 's'=add sep at start  'e'=add sep at end  'b'=add sep at both ends  
> (some) other=default behaviour
> Another would be to have 2 parameters, atStart and atEnd, which would both 
> default to False or 0.  E.g.
> '|'.join(('Spam', 'Ham', 'Eggs'), 1)== '|Spam|Ham|Eggs'
> '|'.join(('Spam', 'Ham', 'Eggs'), 0, 1) == 'Spam|Ham|Eggs|'
> Neither scheme results in particularly transparent usage, though no worse than
> s.splitlines(True) # What on earth is this parameter???
> 
> Corner case:
> What if join() is passed an empty sequence?  This is debatable,
> but I think it should return the separator if requested to add it
> at the beginning or end, and double it up if both are requested.
> This would preserve identities such as
> sep.join(seq, ) == sep + sep.join(seq)
> 
> Best wishes
> Rob Cliffe
> 
> EXAMPLES WHERE SEPARATOR ADDED AT START:
> 
> Lib\http\server.py:933:splitpath = ('/' + '/'.join(head_parts), tail_part)
> Lib\site-packages\numpy\ctypeslib.py:333:name += "_"+"_".join(flags)
> Lib\site-packages\numpy\testing\_private\utils.py:842: err_msg += '\n' + 
> '\n'.join(remarks)
> Lib\site-packages\pip\_vendor\pyparsing\core.py:2092-2095:
> out = [
> "\n" + "\n".join(comments) if comments else "",
> pyparsing_test.with_line_numbers(t) if with_line_numbers else 
> t,
> ]
> Lib\site-packages\pip\_vendor\requests\status_codes.py:121-125:
> __doc__ = (
> __doc__ + "\n" + "\n".join(doc(code) for code in sorted(_codes))
> if __doc__ is not None
> else None
> )
> Lib\site-packages\reportlab\lib\utils.py:1093: self._writeln(' '+' 
> '.join(A.__self__))
> Lib\site-packages\reportlab\platypus\flowables.py:708:L = 
> "\n"+"\n".join(L)
> Lib\site-packages\twisted\mail\smtp.py:1647: r.append(c + b' ' +  b' 
> '.join(v))
> Lib\site-packages\twisted\protocols\ftp.py:1203:return (PWD_REPLY, 
> '/' + '/'.join(self.workingDirectory))
> Lib\site-packages\twisted\runner\procmon.py:424-426:
> return ('<' + self.__class__.__name__ + ' '
> + ' '.join(l)
>

[Python-ideas] Re: Deprecate misleading escapes in strings

[Barry]

On 16 Feb 2023, at 19:45, Ben Rudiak-Gould  wrote:On Thu, Feb 16, 2023 at 8:09 AM Barry <ba...@barrys-emacs.org> wrote:This is valid and does not match your rules. ‘\x9b’ that is the ANSI CSI in 8-bit.
In 7-bit it is ‘\x1b[‘.Shouldn't that be b‘\x9b’?
No. I can work in unicode internally and only encode to bytes for output.Barry___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DL73X7VOXRSREL4MAKA3TMYDW647IJEG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Deprecate misleading escapes in strings

[Barry]

> On 16 Feb 2023, at 14:57, Arusekk  wrote:
> 
> Hi all!
> 
> I was writing a tutorial on the distinction between bytes and strings
> and why it is important, when I saw the root cause.  People coming from
> C, Perl, Python 2 and similar languages tend to misinterpret "\x90" for
> b"\x90" often.  My idea is that Python could deprecate string literals
> containing any non-ASCII codepoints specified in any way different from
> unicode or unicode escapes (\u, \U, \N).
> 
> (Actually I found that I started having the idea already back in 2021 on
> StackOverflow[1].  The question is an excellent example of what I mean.)
> 
> I would not go so far to follow JSON (disallowing \x11 and \222 escapes
> completely), but while writing "\x00" or "\0" is useful and widely used,
> "\x99" (and especially "\777"!) is probably marginal and definitely less
> explicit than "\u0099" (in the Zen of explicit better than implicit).
> Byte strings do not treat b"\u00ff" as b"\xff".
> 
> In the first part of implementing it, Python could raise a SyntaxWarning
> (or should it be DeprecationWarning? BytesWarning?), suggesting "\x99"
> to either become b"\u0099" or b"\x99", eventually promoting it to some
> equally helpful SyntaxError.  All of it could be hidden behind a feature
> like from __future__ import backslashes (one nice name I can think of).
> 
> The new regular expression for octals would be \\[01]?[0-7]{1,2} and
> \\x[0-7][0-9A-Fa-f] for hexadecimals, hopefully not confusing anyone,
> and not much more complex than the old ones.
> 
> In the meantime, probably between introducing a warning and changing it
> to become an error (the most reasonable timeline I can think of now),
> the default ascii() representation should eventually use the \u0099 form
> for all such codepoints, to keep the invariant of eval(ascii(x)) == x
> without syntax warnings.  repr() is also affected, but it is fortunately
> limited to the [\x80-\xa0\xad] range.  I mean [\u0080-\u00a0\u00ad] :-)
> 
> Another timeline would be to change the repr first, initially hidden
> under an interpreter flag or environment variable, then officially
> deprecate it in the documentation, then introduce the error guarded by
> from __future__ import backslashes or another flag, then make the repr
> use \u by default, then add the warning and finally make it always raise
> an error.
> As a precedent, breaking repr() was not a dealbreaker when introducing
> randomized seeds (even repr({"a", "b"}) is now unpredictable).
> 
> This would be of course a breaking change for a lot of unit tests, and
> stuff like pickle should probably support old syntax, delaying any such
> change until a new protocol comes (if it applies to the newest one ---
> quite sure it does not).  Such a breaking change must be used wisely.
> Other changes to octal escapes could be sneaked in, based on conclusions
> from the 2018 'Python octal escape character encoding "wats"' thread[2]
> (I like writing "\0" and "\4" though, just to make my opinion clear).
> If going the whole hog, the 2015 'Make non-meaningful backslashes
> illegal in string literals' thread[3] could be revived as well, maybe
> even with "\f\v" deprecated, "\e" = "\33" introduced and such.
> 
> Please let me know what you think, what else could break, and is it
> useful anywhere else apart from my use case, and what similar problems
> you have.

-1 i think you will break too much valid code.

This is valid and does not match your rules. ‘\x9b’ that is the ANSI CSI in 
8-bit.
In 7-bit it is ‘\x1b[‘.

Barry

> 
> Cheers,
> Arusekk
> 
> [1]: https://stackoverflow.com/q/64832281/3869724
> [2]: 
> https://mail.python.org/archives/list/python-ideas@python.org/thread/ARBCIPEQB32XBS7T3JMKUDIZ7BZGFTL6/
> [3]: 
> https://mail.python.org/archives/list/python-ideas@python.org/message/PJXKDJQT4XW6ZSMIIK7KAZ4OCDAO6DUT/
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/ITBFU4GPJJVXTHT57WNLASXKL4R4MPF5/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6Z62MV6LN3HIJ2UEPHMGCM25COWOB35U/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Proposal: -X importcache to supplement -X importtime for loaded modules

[Barry]

Try raising on https://discuss.python.org/ where you are more likely to have a 
python core dev comment on this idea.

Barry

> On 14 Feb 2023, at 15:12, anthony.flury via Python-ideas 
>  wrote:
> 
> 
> Personally,
> I am -1 on the name - I think '-X importtrace' or similar would be better I 
> think.
> 
> I am +100 on the functionality - Unless you put in log messages, tracking 
> imports is complex - so something like this I can see being very useful.
> 
> I know I don't make the decisions, but it seems if this really is only a 
> couple of lines, and all the test cases pass this (or something very similar) 
> should be a considered for 3.12.
> 
> Thank you Noah,
> 
> 
> 
> -- Original Message --
> From: "Noah Kim" 
> To: python-ideas@python.org
> Sent: Saturday, 11 Feb, 23 At 00:07
> Subject: [Python-ideas] Proposal: -X importcache to supplement -X importtime 
> for loaded modules
> 
> All,
> 
> I'm writing to propose an adjacent interpreter flag to `-X importtime`: `-X 
> importcache` (open to alternative naming). 
> 
> While `-X importtime` is incredibly useful for analyzing module import times, 
> by design, it doesn't log anything if an imported module has already been 
> loaded. `-X importcache` would provide additional output for every module 
> that's already been loaded: 
> 
> ```
> >>> import uuid
> import time: cached| cached |   _io
> import time: cached| cached |   _io
> import time: cached| cached |   os
> import time: cached| cached |   sys
> import time: cached| cached |   enum
> import time: cached| cached | _io
> import time: cached| cached | _io
> import time: cached| cached | collections
> import time: cached| cached | os
> import time: cached| cached | re
> import time: cached| cached | sys
> import time: cached| cached | functools
> import time: cached| cached | itertools
> import time:   151 |151 | _wmi
> import time: 18290 |  18440 |   platform
> import time:   372 |372 |   _uuid
> import time: 10955 |  29766 | uuid
> ```
> 
> In codebases with convoluted/poorly managed import graphs (and consequently, 
> workloads that suffer from long import times), the ability to record all 
> paths to an expensive dependency--not just the first-imported--can help 
> expedite refactoring (and help scale identification of this type of issue). 
> More generally, this flag would provide a more efficient path to tracking 
> runtime dependencies.
> 
> As a proof of concept, I was able to hack this functionality into `-X 
> importtime` by adding a couple lines to `import_ensure_initialized` in 
> `Python/import.c` (hence the output above). A separate flag is probably 
> desirable to preserve backwards compatibility. 
> 
> Looking forward to your feedback,
> Noah
> 
>  
> ___
> 
> Python-ideas mailing list -- python-ideas@python.org
> 
> To unsubscribe send an email to python-ideas-le...@python.org
> 
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> 
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/GEISYQ5BXWGKT33RWF77EOSOMMMFUBUS/
> 
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/ENFJY2RP73IYPUSRVGHBPNELS5JVQLAA/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/W3KUOJXPIZKEAKO5NRDGFY25WNOVAMWZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Multiple arguments to str.partition and bytes.partition

[Barry Scott]

On 08/01/2023 17:06, James Addison wrote:

On Sun, 8 Jan 2023 at 13:20, Barry Scott  wrote:

Maybe combine the ideas by allowing a tuple where a string is used.
'a=b;c'.partition('=', (':',';')) => ('a', '=', b, ';', 'c')

I like that idea - and it seems to fit neatly with the existing
partition contract: the inclusion of the matched-delimiter as an item
in the result-tuple would reduce match ambiguity (rationale: imagining
a use case where a caller would like to provide multiple delimiters
during method invocation, and then subsequently check which delimiter
was used at a given match position).

Since it's both orthogonal to (does not depend-on, nor is it a
dependency-of) multiple-partition-arguments and composible with (can
be combined with) multiple-partition-arguments, perhaps it should be
advocated-for as a separate proposal?  (rationale: that would allow
either proposal to advance without delaying the other -- bearing in
mind a hopefully-unlikely chance of merge conflicts if they reach
release-readiness implementation status in parallel)


I'd suggest going with the tuples and extra args in one proposal.

It possible that its more likely to be loved with the tuple as that 
removes an objection.


Barry



___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4QZ53QOATFDYCUVWZW4GUUF6WCN5SOAG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Multiple arguments to str.partition and bytes.partition

[Barry Scott]

> On 8 Jan 2023, at 10:10, James Addison via Python-ideas 
>  wrote:
> 
> On Sun, 8 Jan 2023 at 03:44, Steven D'Aprano  wrote:
>> 
>> Keep it nice and simple: provided with multiple separators, `partition`
>> will split the string on the first separator found in the source string.
>> 
>> In other words, `source.partition(a, b, c, d)` will split on a /or/ b
>> /or/ c /or/ d, whichever comes first on the left.
> 
> Thanks - that's a valid and similar proposal -- partition-by-any-of --
> although it's not the suggestion that I had in mind.
> 
> Roughly speaking, the goal I had in mind was to take an input that
> contains well-defined delimiters in a known order and to produce a
> sequence of partitions (and separating delimiters) from that input.
> 
> (you and dn have also indirectly highlighted a potential problem with
> the partitioning algorithm: how would "foo?a=b&c" partition when using
> 'str.partition("?", "#")'?  would it return a tuple of length five?)

Maybe combine the ideas by allowing a tuple where a string is used.

'a=b'.partition(('=', ':')) => ('a', '=', 'b')
'a:b'.partition(('=', ':')) => ('a', ':', 'b')

'a=b:c'.partition('=', (':',';')) => ('a', '=', b, ':', 'c')
'a=b;c'.partition('=', (':',';')) => ('a', '=', b, ';', 'c')

Barry


> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/6DNT7S46SZCQNFKXXBKD5PQ557KQRKEP/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7GRVMPHLIKH26ZIHMODADHJ3DXZWZXJN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Idea: Tagged strings in python

[Barry Scott]

 this? Do you see a value in adding tagged strings 
> to python? Are there other use-cases where this would be useful? Does the 
> suggestion need to support calling tags as functions like in javascript to be 
> interesting?
> 
> (I'm new to python-ideas, so I hope I haven't broken some unspoken rule with 
> this suggestion.)

I think this has been discussed before and rejected.

Your need 2 things to happen
(1) a syntax change in python that is acceptable
(2) a significant editor to support syntax highlighting for that python change.
(3) someone willing to write and support the feature in the python code base

Will you write and support the code?

If the tags are called as functions then you can do it today with this:

def html(s):
return s

HEAD = html('')


Barry

> 
> -- 
> Emil Stenström
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/OXHQHMV2JC2PY7K63VNIMSTP5T46LPKT/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/E27OL43KVTWNH7CDJ7Q7AAHF5UACMWEL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Nonuniform PRNG?

[Barry]

On 7 Dec 2022, at 19:45, James Johnson  wrote:How do I join the “regular” Python list? I’m not academically qualified for python ideasTry out https://discuss.python.org that has lots going on.BarryOn Wed, Dec 7, 2022 at 10:05 AM David Lowry-Duda  wrote:I'm very sorry, I didn't meant to post this to python-ideas. I didn't 
pay enough attention. I'm sending the same thing to the regular python 
list.

On Wed, Dec 07, 2022 at 11:03:24AM -0500, David Lowry-Duda wrote:
>Inspired by the recent thread on PRNG, I began to wonder: suppose that 
>I had a pseudorandom number generator that attempted to generate a 
>nonuniform distribution. Suppose for instance that it was to generate 
>a 0 bit 2/3 of the time, and a 1 bit 1/3 of the time.
>
>How would one go about testing this PRNG against an idealized 
>(similarly biased) PRNG?
>
>- DLD
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/5C6XQAKD2KLJNYQT6QUIDCJLDSY3ECPL/
Code of Conduct: http://python.org/psf/codeofconduct/
-- Truth causes consequences; consequences bring pain; pain exorcises guilt!
___Python-ideas mailing list -- python-ideas@python.orgTo unsubscribe send an email to python-ideas-le...@python.orghttps://mail.python.org/mailman3/lists/python-ideas.python.org/Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/HLSE3UTNQTG7ML3ZVBJAX7QCGPOFGZV7/Code of Conduct: http://python.org/psf/codeofconduct/___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HYAF6PMFRNNVYS3SQHFM4DM3EEEYWD6N/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Enhancing variable scope control

[Barry]

> On 29 Nov 2022, at 14:34, Anony Mous  wrote:
> 
> 
> As it stands now, to create a local scope, you must define a function.
> 
> However, there are many cases where various things can reasonably be done 
> inline. Preparatory code is often done this way.
> 
> Good coding practice is generally accepted to be that variables are local if 
> at all possible. However, in direct, inline Python code, we're inherently 
> creating variables with a global scope.
> 
> We don't actually need a function for this kind of code; but a local scope 
> would often be valuable (as we see with lambda.) Moving things off into a 
> function can create a circumstance where we have to go looking for the 
> function. When something is a "one-shot" as in global preparatory code, 
> that's doing more work, and creating more indirection, than needs actually be 
> done. But global operations have their own pitfalls, one of which is variable 
> collisions. So Python sort of drives us to make "one-use" functions where 
> lambdas are insufficient to the case in order to control locality.
> 
> You can end up writing things like...
> 
> def PrepOne():
> for MyCount in range(0,10):
> pass
> 
> PrepOne()
> 
> ...which achieves the locality desired for variable MyCount, but is clunky. 
> It also clutters the global namespace with prepOne, which has no good reason 
> to exist.
> 
> So I'm thinking of something along these lines:
> 
> local:# <== or whatever syntax makes sense
> for MyCount in range(0,10):
> pass
> 
> So in that example, the code is inline - runs as part of the global, 
> sequential code execution - but the variable MyCount is local and goes "poof" 
> once the local: scope is exited.
> 
> This is the scoping equivalent of a pair of curly braces in c and c++. I 
> would like to see it be nestable, as scopes are in c/c++:
> 
> local:# <== or whatever syntax makes sense
> for MyCount in range(0,10):
> local:
> for YourCount in range(0,MyCount+1)
> pass
> 
> There, MyCount is available in both scopes (the first local: is an enclosing 
> scope) and YourCount is available only in the second local: scope.
> 
> Or, local: could be very strict, and require further syntax to incorporate a 
> previous scope, something like this:
> 
> local:# <== or whatever syntax makes sense
> for MyCount in range(0,10):
> local:
> incorporate MyCount # <== or whatever syntax makes sense
> for YourCount in range(0,MyCount+1)
> pass
> 
> Lastly, I suggest that the global keyword be used to expose global scope 
> variables, just as it is within def functions:
> 
> TheirCount = 10
> local:# <== or whatever syntax makes sense
> global TheirCount
> for MyCount in range(0,TheirCount):
> local:
> for YourCount in range(0,TheirCount)
> pass
> 
> This would also be useful within normal functions, and again allows the 
> program flow to be linear rather than calling a function-within-a-function.

This has been discussed here before and rejected.
The ways scoping works in python is very hard to change as you are asking for i 
think was a major reason to reject.

Barry


> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/3ZEM5FPYWI24G3EOM3HA2I4RKWCMR3J5/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3AUAAUN5E6CZDWPSUKKVUGFSGMF7WASB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Allow more flexibility for describing bytes objects.

[Barry Scott]

> On 23 Nov 2022, at 12:32, Ronald Hoogenboom via Python-ideas 
>  wrote:
> 
> Refer to PEP 3137 and PEP 358.
>  
> Bytes objects are for conveying binary data (or encoded strings). Such binary 
> data is customary specified in hex-dump or base64 format in source files.
> It would be nice to introduce a way in python to do that ‘natively’ (at 
> lexical analysis time) using x-strings and y-strings (in lieu of f-strings).
>  
> x-strings:
>  
> x”0123456789abcdef”  (x-prefixed quoted string of 
> even-numbered amount of characters of the set [0-9a-fA-F], whitespace ignored)
> equivalent to:
> b”\x01\x23\x45\x67\x89\xab\xcd\xef”  (but more readable)
>  
> y-strings:
>  
> y”ASNFZ4mrze8=” (y-prefixed quoted string of valid base64, 
> whitespace ignored)
> equivalent to:
> b”\x01\x23\x45\x67\x89\xab\xcd\xef”  (but shorter)
>  
> This is not a replacement of the hex/base64 encoding, binascii packages etc. 
> It just gives the programmer more freedom to specify literal bytes objects in 
> the source code.

How often would this be useful? If its rare and only nice to have I would doubt 
it would be added to python.

I have written a lot of low level protocol and IOCTL calls that cannot think of 
a time this would have helped me.
struct is often a mean to create blocks of binary data.

Long strings of binary data would be a maintenance issue. What do all the bits 
mean?

Barry



>  
> ___
> Python-ideas mailing list -- python-ideas@python.org 
> <mailto:python-ideas@python.org>
> To unsubscribe send an email to python-ideas-le...@python.org 
> <mailto:python-ideas-le...@python.org>
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/VGXARYOPWYXUVGF6FA4DPMHCKIQVQF6L/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XOZQHUZGRVH62HHRGMJU2ZOHWD5LGJBX/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Better (?) PRNG

[Barry]

> On 14 Nov 2022, at 14:31, James Johnson  wrote:
> 
> 
> I wrote the attached python (3) code to improve on existing prng functions. I 
> used the time module for one method, which resulted in disproportionate odd 
> values, but agreeable means.
> 
> I used the hashlib module for the second. It is evident that the code is 
> amateur, but the program might result in better PRN generation. 
> 
> The "app" lends itself to checking, using statistical tools (see comments.)

Have you used any cryptographic tools to prove the quality of your PRNG?
What results did you get?
How does your PRNG compare to what python already has?

Without that this analysis this will be unlikely to be considered as a 
candidate for python stdlib.

Barry

> 
> I remain a fan,
> 
> James Johnson
> 
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/VENAT2YTVYVHTBSEVHHMIURCU6MG2CEO/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HWQV4AKQAUM7CY4NWS2IRIVLRAYMKR3V/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add mechanism to check if a path is a junction (for Windows)

[Barry]

On 8 Nov 2022, at 17:56, Charles Machalow  wrote:I would argue that just because it was easy for one to implement doesn't mean it's easy for others. That is true of so many things in life!I would have had no idea how to implement this without extra Googling and confusion.Having the abstraction makes it easier for others.But anyone that is suitably motivated can implement this.Barry- Charlie Scott Machalow
On Tue, Nov 8, 2022 at 1:12 AM Eryk Sun  wrote:On 11/8/22, Charles Machalow  wrote:
>
> Funny enough in PowerShell, for prints an "l" for both symlinks and
> junctions.. so it kind of thinks of it as a link of some sort too I guess.

As does Python already in many cases. For example, os.lstat() doesn't
traverse a mount point (junction). On Windows, symlinks and mount
points are in a general category of name-surrogate reparse points.
os.lstat() doesn't traverse them.

If Python supported copying a mount point via
os.symlink(os.readlink(src), dst), I'd be reluctantly in favor of just
letting ntpath.islink() return true for a mount point, as a practical
measure for seamless cross-platform implementations of functions like
rmtree() and copytree(). In terms of POSIX that's nonsense, but not
really on Windows.

> Is it that much of a waste to just return False on posix? I mean it's a
> couple lines and just maintains api.. and in theory can be more clear to
> some.

I'm just thinking this through in terms of conceptual cost and
usefulness in the standard library relative to how easy it is to
implement one's own isjunction() or is_name_surrogate() test. Of
course, a lot of the os.path tests have simple implementations, such
as exists(), isdir() and isfile(). They're in the standard library
because they're commonly needed. The question is whether isjunction()
is needed enough generally to justify adding it.

___Python-ideas mailing list -- python-ideas@python.orgTo unsubscribe send an email to python-ideas-le...@python.orghttps://mail.python.org/mailman3/lists/python-ideas.python.org/Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/MOY5NNFQK5PMD5GQBADZE3COVT7ONA62/Code of Conduct: http://python.org/psf/codeofconduct/___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WKA3ZPPLHTY3K3CCADUYC3KZI76NYTTA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Add copy to pathlib

[Barry]

> On 18 Oct 2022, at 21:59, Chris Angelico  wrote:
> 
> On Wed, 19 Oct 2022 at 06:50, Todd  wrote:
>> 
>> Currently, pathlib supports pretty much all common filesystem operations. 
>> You can create, move, and delete both files and directories. One big 
>> omission is copying. You need shutil for that.
>> 
>> Whatever the original intent might have been behind pathlib, it is now 
>> effectively the standard tool for filesystem operations. As such, excluding 
>> copying alone from basic operations creates an unnecessary hurdle for users. 
>> It also increases the difficulty of doing such operations since there is no 
>> hint within the pathlib documentation on how to copy, new users basically 
>> need to google it to find out.  That is fine for less common operations, but 
>> far from ideal from a basic one like copying.
>> 
> 
> Ah. I would look at solving this the other way: since this really
> isn't a path operation (in the same sense that moving/renaming and
> deleting are), keep it in shutil, but improve discoverability with a
> docs reference.

I tend to think that if move is in pathlib then copy makes sense.

As with all things pathlib it is a reasonable question to ask where should 
additions stop.

Barry

> 
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/DD36UOFGIBHY63YDAVYKTR5CFMMD3GUC/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/Z5YOAHHLNZ5BPC3VUBVQNZIBEWWUUVKI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Use 'bin' in virtual environments on Windows

[Barry Scott]

> On 21 Jul 2022, at 00:46, Svein Seldal  wrote:
> 
> 
> Using py in the three OS-es (unix-like, Mac and Win) has become very similar 
> in the last years, which is great. Making portable py code has become very 
> easy. However, there is one peculiarity in Win which I find annoying: virtual 
> environments insists on placing executables in the `Scripts` directory, while 
> the other platforms use `bin`. This forces portable scripts and programs on 
> the outside of py to be OS-aware in order to interact with a venv on a 
> Win-system vs other systems.
> 
> I would like to proposing the ability to configure virtual environments to 
> use `bin` directory for executable on Windows. Personally I think it would be 
> smart move if it were changed to `bin` as default, making all py platform act 
> consistently. However, I do understand that there might be existing code out 
> there that depend on using `Scripts`.

In my experience I have to write Windows specific scripts and code so dealing 
with "Scripts" is not an issue.
Yes I would have preferred it to be called bin, but we are where we are.

Barry

> 
> Best regards,
> Svein
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/5OFYP734PMXBVIXEW444IU4CPUI7KTIB/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/EV4OTJN34SCXDJ26QOC4VCQMVNU7ADW5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Use 'bin' in virtual environments on Windows

[Barry Scott]

> On 21 Jul 2022, at 16:42, Christopher Barker  wrote:
> 
> I still am dumbfounded that this wasn’t platform I dependent in the first 
> place, but you know what essay about hindsight.
> 
> However, I’m no Windows expert, but I *think* the modern Windows file 
> system(s?) support something like symlinks. It’s an under-the-hood feature, 
> but maybe it’s possible to add a symlink for bin. 

It has symlinks but only available if you are administrator.

> 
> Maybe I’m wrong, and/or it’s not possible on all file systems Python needs to 
> support, in which case *nix systems do support linking, so we could support 
> “Scripts” on all systems.

-1

Barry

> 
> Just a thought. 
> 
> -CHB
> 
> On Thu, Jul 21, 2022 at 8:26 AM Simão Afonso 
> mailto:simao.afo...@powertools-tech.com>> 
> wrote:
> On 2022-07-21 17:04:16, Svein Seldal wrote:
> > https://github.com/pypa/pip/bob/main/src/pip/_internal/locations/_distutils.py#L145-L148
> >  
> > <https://github.com/pypa/pip/blob/main/src/pip/_internal/locations/_distutils.py#L145-L148>
> 
> This actually works great! If the "Scripts" folder does not exist, it
> uses "bin".
> 
> > https://github.com/pypa/pip/blob/main/src/pip/_internal/utils/entrypoints.py#L48
> >  
> > <https://github.com/pypa/pip/blob/main/src/pip/_internal/utils/entrypoints.py#L48>
> 
> This is a just a small patch anyway.
> ___
> Python-ideas mailing list -- python-ideas@python.org 
> <mailto:python-ideas@python.org>
> To unsubscribe send an email to python-ideas-le...@python.org 
> <mailto:python-ideas-le...@python.org>
> https://mail.python.org/mailman3/lists/python-ideas.python.org/ 
> <https://mail.python.org/mailman3/lists/python-ideas.python.org/>
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/JZZ6QJVEJBC4TZF6SUBWD3TUK2RHIRTW/
>  
> <https://mail.python.org/archives/list/python-ideas@python.org/message/JZZ6QJVEJBC4TZF6SUBWD3TUK2RHIRTW/>
> Code of Conduct: http://python.org/psf/codeofconduct/ 
> <http://python.org/psf/codeofconduct/>
> -- 
> Christopher Barker, PhD (Chris)
> 
> Python Language Consulting
>   - Teaching
>   - Scientific Software Development
>   - Desktop GUI and Web Development
>   - wxPython, numpy, scipy, Cython
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/S6DMH3SXK53ZLG5DRFIITVAZFVUNW53Y/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2B54IABNSFMMBR7OJ4XGUKFDMPSG2TFO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Make dataclass aware that it might be used with Enum

[Barry Scott]

> On 9 Jul 2022, at 22:53, Steve Jorgensen  wrote:
> 
> I don't think that dataclasses have the limited set of intended uses that you 
> are interpreting them as having. To me, the fact that they can be frozen 
> makes them a good fit with Enum.

Please quote the email that you are replying to.

It is usually considered a code smell to have a class that is two or more 
things.
This seems to be what you are trying to do.

How can one class be a set of fields and also the enum for one of its own 
fields?
I do not understand why this is resonable.

Barry



> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/V6U7UMQRTLDZ2W6SWREL472L6ZH7MHB5/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HZFZE3YMCMT6CF5VZHNM6ZJVDAX6LZXK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Make dataclass aware that it might be used with Enum

[Barry Scott]

> On 8 Jul 2022, at 02:22, Steve Jorgensen  wrote:
> 
> After some playing around, I figured out a pattern that works without any 
> changes to the implementations of `dataclass` or `Enum`, and I like this 
> because it keeps the 2 kinds of concern separate. Maybe I'll try submitting 
> an MR to add an example like this to the documentation for `Enum`.
> 
> In [1]: from dataclasses import dataclass
> 
> In [2]: from enum import Enum
> 
> In [3]: @dataclass(frozen=True)
>   ...: class CreatureDataMixin:
>   ...: size: str
>   ...: legs: int
>   ...: 
> 
> In [4]: class Creature(CreatureDataMixin, Enum):
>   ...: BEETLE = ('small', 6)
>   ...: DOG = ('medium', 4)
>   ...: 
> 
> In [5]: Creature.DOG
> Out[5]: Creature(size='medium', legs=4)

Can't you define the type of size as an enum?
Using multiple inheritance seems like the wrong way to go.
What if you are 10 fields in the dataclass that are all enums?
That could get messy.

Disclaimer I have not used dataclass. Just thinking from OOD point of view.

Barry

> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/G2VALQ4RIVFKIOKVW4XZAHZMLSZWL2XS/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CBH24S2XG5EF2CQWER6BWLBSZEGK4Q7O/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Generalized deferred computation in Python

[Barry]

> On 23 Jun 2022, at 08:27, Stephen J. Turnbull  
> wrote:
> 
> Barry Scott writes:
> 
>> I can think of ways to implement evaluation-on-reference, but they
>> all have the effect of making python slower.
> 
> Probably.
> 
>> The simple
>> 
>>a = b
>> 
>> will need to slow down so that the object in b can checked to see
>> if it need evaluating.
> 
> No, it doesn't.  Binding a name is special in many ways, why not this
> one too?  Or "a = a" could be the idiom for "resolve a deferred now",
> which would require the check for __evaluate_me_now__ as you say.  But
> such simple "a = b" assignments are not so common that they would be a
> major slowdown.  I would think the real problem would be the "oops" of
> doing "a = b" and evaluating a deferred you don't want to evaluate.
> But this isn't a completely new problem, it's similar to a = b = []
> and expecting a is not b.

Interest idea that ref does not auto evaluate in all cases.
I was wondering about what the compile/runtime can do it avoid the costs
of checking for an evaluation.
> 
> Now consider a = b + 0.  b.__add__ will be invoked in the usual way.
> Only if b is a deferred will evaluation take place.

But the act of checking if b is deferred is a cost I am concerned about.

> 
> So I don't really see the rest of Python slowing down much.

Once we have the PEP address it’s semantics in detail we can estimate the costs.

I would think that it’s not that hard to add the expected check into the python 
ceval.c
And benchmark the impact of the checks. This would not need a full 
implementation
of the deferred mechanism.

Barry
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/DEBOPY6OIRRLLCO2SQDYXXM7UMXZYRMZ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Generalized deferred computation in Python

[Barry Scott]

> On 22 Jun 2022, at 19:09, Paul Moore  wrote:
> 
> I suspect that you consider evaluation-on-reference as an important
> feature of your proposal, but could you consider explicit evaluation
> as an alternative? Or at the very least address in the PEP the fact
> that this would close the door on future explicit evaluation models?

I can think of ways to implement evaluation-on-reference, but they all have the 
effect of
making python slower.

The simple

a = b

will need to slow down so that the object in b can checked to see if it need 
evaluating.

How will you avoid making python slower with this feature?

Barry

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/QBL6JFDKDKFBURKEXOHEOBCK6WULEEQR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: default as a keyword argument for dict.get and dict.pop

[Barry]

> On 8 Jun 2022, at 16:09, Rob Cliffe via Python-ideas 
>  wrote:
> 
> 
> 
>> On 08/06/2022 15:40, Eric V. Smith via Python-ideas wrote:
>> 
>>> On 6/7/2022 4:59 PM, Chris Angelico wrote:
>>> On Wed, 8 Jun 2022 at 00:36,  wrote:
>>>> Hello!
>>>> 
>>>> Do you know if there has been discussions around why is the default 
>>>> argument is positional only in the dict methods get and pop?
>>>> 
>>>> I think
>>>> 
>>>> ```
>>>> d.get(key, default=3)
>>>> ```
>>>> 
>>>> way more readable than
>>>> 
>>>> ```
>>>> d.get(key, 3)
>>>> ```
>>>> 
>>>> specially since max and min builtin functions use default as a keyword 
>>>> argument.
>>> With min and max, it MUST be a keyword argument, because positional
>>> arguments are the values to be compared. So I think the main reason is
>>> "because nobody ever bothered to do it". If there's enough value in
>>> it, that could probably be changed, although mere consistency alone
>>> isn't a very strong argument.
>> 
>> I suspect it's been this way because the API is so old. Now that we have 
>> Argument Clinic it would be easier to implement as a keyword argument. But 
>> there may also be a performance issue with keyword arguments vs. positional. 
>> People are touchy when it comes to dicts!
>> 
>> Eric
> True (and rightly so).  But all that is required is that there is no 
> significant slow-down when the positional argument is used.
> Does that sound feasible?
> People can always choose not to use the keyword form.


Won’t the code have to check for a keyword even if a positional is used and 
therefore it must be slower. Need a benchmark to know by how much.

I’m one of those people that notice uS slow does in access to some APIs.

Barry

> Best wishes
> Rob Cliffe
>> 
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at 
>> https://mail.python.org/archives/list/python-ideas@python.org/message/73OOS3YPUDTZVWFJ5V44X66XZCNXRQV6/
>> Code of Conduct: http://python.org/psf/codeofconduct/
> 
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/LMVZXUZQAJ4PRKWGTQDP45FS5YDRKRZG/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/SF2TJG6XL37UMZOQCPGZDV5XUDKCKETW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: TextIOBase: Make tell() and seek() pythonic

[Barry Scott]

> On 25 May 2022, at 10:16, Stephen J. Turnbull  
> wrote:
> 
>  This is much
> better than the pre-PEP-393 situation (where the unicode type was
> UTF-16, 

Only on Windows right? mac and Linux did not do this, at least with the bullds
I have used for python 2.7.

Barry

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/I5L4SZDIZATQDGLXRZVSUSQWBQJ2UUTJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Less is more? Smaller code and data to fit more into the CPU cache?

[Barry Scott]

> On 27 Mar 2022, at 18:16, Jonathan Fine  wrote:
> 
> Hi
> 
> Thank you Inada for your prompt and helpful reply. Here's a link for cached 
> hash in bytes object: https://bugs.python.org/issue46864 
> <https://bugs.python.org/issue46864>
> 
> What I have in mind is making selected objects smaller, for example by using 
> smaller pointers. But how to know the performance benefit this will give?

That will limit python to 2GiB or maybe 4GiB of memory - I routinely run beyond 
that size in production systems.

There is a memory model that GCC supports that is 32bit pointers and 64bit ints.
I do not recall the performance comparisons, buts it is not used very much.

> 
> I think it would be helpful to know how much SLOWER things are when we make 
> Python objects say 8 or 16 bytes LARGER. This would give an estimate of the 
> improvement from making all Python objects smaller.
> 
> I've not done much performance testing before. Anyone here interested in 
> doing it, or helping me do it? (Warning - I've never built Python before.)

Performance tests is hard to get right. I do this as my day job for a big Cloud 
app written mostly in Python.

There is an excellent book on performance measurement by Brendan Greg "Systems 
Performance; Enterprise and the Cloud".

Barry


> 
> with best regards
> 
> Jonathan
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/PMDNJFXMWXU3LQH5KXO4MM5SCGSP2J4P/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5MXT6UFAKAT3X56NGGSQ6S3NS5A6IHUG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Less is more? Smaller code and data to fit more into the CPU cache?

[Barry Scott]

> On 22 Mar 2022, at 15:57, Jonathan Fine  wrote:
> 
> Hi
> 
> As you may have seen, AMD has recently announced CPUs that have much larger 
> L3 caches. Does anyone know of any work that's been done to research or make 
> critical Python code and data smaller so that more of it fits in the CPU 
> cache? I'm particularly interested in measured benefits.

I few years ago (5? 10?) there was a blog about making the python eval loop fit 
into L1 cache.
The author gave up on the work as he claimed it was too hard to contribute any 
changes to python at the time.
I have not kept a link to the blog post sadly.

What I recall is that the author found that GCC was producing far more code 
then was required to implement sections of ceval.c.
Fixing that would shrink the ceval code by 50% I recall was the claim. He had a 
PoC that showed the improvements.

Then there was research on the opcodes and eliminating unnecessary code in 
there implementation,
but I do not trust that I remember the details of this, its been too long since 
I read the blog.

Barry


> 
> This search
>   https://www.google.com/search?q=python+performance+CPU+cache+size 
> <https://www.google.com/search?q=python+performance+CPU+cache+size>
> provides two relevant links
>   
> https://www.oreilly.com/library/view/high-performance-python/9781449361747/ch01.html
>  
> <https://www.oreilly.com/library/view/high-performance-python/9781449361747/ch01.html>
>   
> https://www.dlr.de/sc/Portaldata/15/Resources/dokumente/pyhpc2016/slides/PyHPC_2016_talk_14.pdf
>  
> <https://www.dlr.de/sc/Portaldata/15/Resources/dokumente/pyhpc2016/slides/PyHPC_2016_talk_14.pdf>
> but not much else I found relevant.
> 
> AnandTech writes about the chips with triple the L3 cache:
>   
> https://www.anandtech.com/show/17323/amd-releases-milan-x-cpus-with-3d-vcache-epyc-7003
>  
> <https://www.anandtech.com/show/17323/amd-releases-milan-x-cpus-with-3d-vcache-epyc-7003>
> "As with other chips that incorporate larger caches, the greatest benefits 
> are going to be found in workloads that spill out of contemporary-sized 
> caches, but will neatly fit into the larger cache."
> 
> And also:
>   
> https://www.anandtech.com/show/17313/ryzen-7-5800x3d-launches-april-20th-plus-6-new-low-mid-range-ryzen-chips
>  
> <https://www.anandtech.com/show/17313/ryzen-7-5800x3d-launches-april-20th-plus-6-new-low-mid-range-ryzen-chips>
> " As detailed by the company back at CES 2022 and reiterated in today’s 
> announcement, AMD has found that the chip is 15% faster at gaming than their 
> Ryzen 9 5900X."
> 
> I already know that using Non Uniform Memory Access (NUMA) raises the 
> difficult problem of cache coherence.
>   https://en.wikipedia.org/wiki/Non-uniform_memory_access 
> <https://en.wikipedia.org/wiki/Non-uniform_memory_access>
>   https://en.wikipedia.org/wiki/Cache_coherence 
> <https://en.wikipedia.org/wiki/Cache_coherence>
> 
> -- 
> Jonathan
> 
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/CUKUKYAFDMX6NTXPK6JQGHHDUSXICH3V/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YEYPDO5N6QRJ26Y2SDGOUWPA4J5PI27T/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: mro and super don't feel so pythonic

[Barry Scott]

> On 26 Mar 2022, at 18:15, malmiteria  wrote:
> 
> the alternative to super really is not the important part of my proposal, 
> it's the alternative to MRO.
> 
> An example of a case where i genuinly believe my solution adds value is this :
> 
> ```
> class A:
>def method(self):
>print("A")
> 
> class B:
>def method(self):
>print("B")
> 
> class C(A, B):
>pass
> ```
> 
> Today, a code such as ```C().method()``` works without any problems
> except of course when the method you wanted to refer to was the method from B.
> If class A and B both come from libraries you don't own, and for some reason 
> have each a method named the same (named run, for example)
> the run method of C is silently ignoring the run method of B.
> 
> I believe it would make the programmers life easier to have an error at this 
> time, or anything tbh, to make explicit this resolution, because it really is 
> not explicit from most programmers perspective
> 
> 
> my alternative to super comes to fit the alternative to mro, i think it 
> stills matter to have a dedicated feature instead of simply calling 
> class.method, this allows for more error handling and stuff like that, in 
> case for example you're calling super on a class that's not a parent for 
> example, since super really is for accessing parents context.
> This is not so much about final code, but more about making it easier *when* 
> writing code. But the final code would definitely not look that much prettier 
> / ugglier, I agree
> 
> And finally, super is not a proxy to parents, even plural, it's a proxy to 
> the next in mro order.
> 
> in this case :
> class Top:
>def method(self):
>print('Top')
> class Left(Top):
>def method(self):
>print('Left')
>super().method()
> class Right(Top):
>def method(self):
>print('Right')
>super().method()
> class Bottom(Left, Right):
>def method(self):
>print('Bottom')
>super().method()
> 
> Bottom().super() would print "Bottom", "Left", "Right", "Top".
> super, in Left, reffers to Right, which is not a parent of Left (and is 
> completely absent of it's definition)

My feeling is that if I was doing a code review of this sort of code I would be 
raising two possible issues.
1) You have a class that does more than 1 thing. That is an OO code smell. Use 
composition to avoid.
2) If you have a parent class that has a method that conflicts then I would say 
that you need to
add a facade to allow one of the conflicting method names to be accessed via 
another name.

For example

Class FacadeB(B):
def method_in_B(self, *args, **kwds):
B.method(self, *args, **kwds)

Barry


> 
> 
> 
> Anyways, i'm called to a pub, i'll talk to you guys more tomorrow, have a 
> great night
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/EWMRM6HIDK4XJPEV7R3WETVP72SUT4RA/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OB5KXXGVCQABFVL242EMB7RV4GAHTPEO/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: mro and super don't feel so pythonic

[Barry]

> On 26 Mar 2022, at 20:19, Christopher Barker  wrote:
> 
> Also, super() actually calls the method on all the  superclasses (but not the 
> same one twice) -- so that right to left thing doesn't matter.

Super() calls the method on only one class. Not all.
It finds that one class by using the mro to find the one class.

Barry
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/N7PUDFLIFYS5VH4XPWLKFDRAFR6JND2R/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: suprocess.check_call and friends include STDERR in error message

[Barry]

> On 9 Mar 2022, at 14:42, mangelo...@gmail.com wrote:
> 
> I often wish to perform the action of `subprocess.check_output`, i.e. print 
> STDOUT live, and check the return code is 0 on exit. However  if there is an 
> error, I would like the STDERR to be printed. When looking at log files and 
> just seeing the return code is often not very beneficial.
> 
> E.g.:
> 
> ```py
> subprocess.check_call(['ping', '-c', '1', 'github.com2'])
> ```
> 
> Will generate this error:
> ```
> subprocess.CalledProcessError: Command '['ping', '-c', '1', 'github.com2']' 
> returned non-zero exit status 2.
> ```
> 
> Here we are left wondering why it failed, on real world examples it becomes 
> critical to be able to see what the actual error is. So I find I always have 
> to wrap such calls in something like this to be able to see STDERR:
> 
> ```
> import subprocess
> cmds = ['ping', '-c', '1', 'github.com2']
> result = subprocess.run(cmds, stderr=subprocess.PIPE)
> if result.returncode != 0:
>msg = result.stderr.decode().strip()
>raise subprocess.CalledProcessError(f"CALLED SUBPROCESS ERROR: Command: {' 
> '.join(cmds)}\nReturn code {result.returncode}\nSTDERR: {msg}\n")
> ```
> 
> With the above one gets to see that STDERR is:
> ```
> ping: github.com2: Temporary failure in name resolution
> ```

I think you are saying you can do what you want with subprocess.run().
What did you need a change to support?

I often wrap code around subprocess.run() to get the exact behaviour that
I need. Seems to work well.

Barry


> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/7PVHSJGKFUIV4ZJFWG7EU3DR6XAGAKQW/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7F6HKX3FRG6QWEJM57BTCKHKUE2T7BXG/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Proposal to associate thread-local data/context with a faulthandler traceback

[Barry Scott]

> On 6 Mar 2022, at 07:19, t...@tomforb.es wrote:
> 
> For reference, this request comes from running Dask[1] jobs. Dask handles 
> retrying and tracking tasks across machines but if you're dealing with a 
> batch of inputs that reliably kills a worker it is really hard to debug, 
> moreso if it only happens ~12 hours into your job. At certain scales it's 
> quite hard to log every processing event reliably, and the overhead may not 
> be worth it for a 1 in 10,000,000 failure.

I think that the core dump will get you an answer now.
With your example you will have only 1 core dump to look at.

> On 6 Mar 2022, at 08:29, t...@tomforb.es wrote:
> 
> If anyone is interested, I had a play around with this and came up with a 
> pretty simple-ish implementation: https://github.com/orf/cpython/pull/1/files.

I wonder if you should just raise a bug against python for this and provide 
your PR for the implementation.

Barry

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3FWB5HREIHJCCO23O3AZJPMLPI545FEC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Proposal to associate thread-local data/context with a faulthandler traceback

[Barry Scott]

> On 2 Mar 2022, at 14:22, t...@tomforb.es wrote:
> 
> The faulthandler module is invaluable for tracking down segfaults in native 
> code, however it is really lacking the ability to add some kind of useful 
> breadcumb to aide debugging. Imagine you are running a large-scale 
> distributed job over tens of millions of images and a single one causes 
> opencv to segfault reliably. This can be very difficult to track down.
> 
> It would be very useful to be able to add a string (limited by size) via the 
> faulthandler module that is outputted along with the traceback. For example:
> 
> ```
> for image in millions_of_images():
>faulthandler.context(f'{image.id=}')
>segfaulty_function(image)
> 
> # Or maybe:
> for image in millions_of_images():
>with faulthandler.context(f'{image.id=}'):
>segfaulty_function(image)
> ```
> 
> The traceback might be something like:
> 
> ```
> Fatal Python error: Segmentation fault
> 
> Current thread 0x7fb899f39700 (most recent call first):
>   (traceback)
> Segmentation fault
> Context: image_id=foo
> ```
> 
> You could of course add logging to this function to print out the image ID 
> before you run it through `segfaulty_function`, but if you're running this at 
> high volume on a huge number of machines, this becomes a bit of an overhead, 
> and it doesn't handle multiple threads running the function well.

This does sound useful. I use this pattern of setting a context that is only 
logged if there is an error in my own code.

Until you have this have you considered turning on core dumps? Then you will be 
able to see the image id in the dump file.
Alternatively you could write the context to a file. Then log the contents of 
the file in a wrapper script that handlers python exiting on SEGV.

I understand what you are getting at about not wanting to log every attempt.
But you could have another program that was sent a message (over UDS) saying 
"about to process {context}" and
after completion sent a message "processing of {context} completed. You would 
log only the {context}
that are not completed.

Barry


> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/BG36W24KEEX4YJA7CZ3PGW4ISZGVXMH6/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KD4YD7VRDCJPCHCN6PQ3JFAL2MDTLX3H/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: shutil.copyfileobj to return number of bytes copied

[Barry]

> On 2 Mar 2022, at 13:40, Davis, Matthew via Python-ideas 
>  wrote:
> 
> 
> Hi,
> 
> Currently shutil.copyfileobj returns nothing.
> I would like to be able to find out how many bytes were copied.
> 
> Whilst most file-like objects have a .tell() which you could use, some don’t, 
> and .tell() is not guaranteed to measure the number of bytes, it could 
> measure with other units.

Can you use os.fstat() on one of the file objects?
Can you write a helper that gets the size from the object that do not have 
tell()?

Barry

> 
> I don’t think changing the return value from None to not-None would be 
> backwards incompatible.
>  
> We could also do the same change for all other shutil.copy* methods (except 
> copytree).
>  
> Looking at the code, this seems straightforward to implement.
>  
> Existing code:
> 
> ```
> def copyfileobj(fsrc, fdst, length=0):
> """copy data from file-like object fsrc to file-like object fdst"""
> # Localize variable access to minimize overhead.
> if not length:
> length = COPY_BUFSIZE
> fsrc_read = fsrc.read
> fdst_write = fdst.write
> while True:
> buf = fsrc_read(length)
> if not buf:
> break
> fdst_write(buf)
> ```
>  
> New code:
> 
> 
> ```
> def copyfileobj(fsrc, fdst, length=0):
> """copy data from file-like object fsrc to file-like object fdst"""
> # Localize variable access to minimize overhead.
> if not length:
> length = COPY_BUFSIZE
> fsrc_read = fsrc.read
> fdst_write = fdst.write
> bytes_copied = 0
> while True:
> buf = fsrc_read(length)
> if not buf:
> break
> fdst_write(buf)
> bytes_copied += len(buf)
> return bytes_copied
> ```
>  
> Regards,
> Matt
>  
>  
> Matthew Davis
> Data Analytics Senior Lead
> Telstra Energy – Decision Science & AI
> E: matthew.davi...@team.telstra.com | M: 0415762868
>  
>  
>  
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/ZKVI4DE7RLYQNLV3XYUUW24FOO57WB5P/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6C2D45ZBI3ZKBU322J7EC2O6RLJMM6KE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Str to Byte

[Barry Scott]

> On 23 Feb 2022, at 21:05, one last Day  wrote:
> 
> bit = "\xd8\xa3\xd9\x88\xd9\x87 \xd8\xa8\xd8\xaf\xd9\x8a\xd9\x84 
> \xd9\x85\xd9\x86 \xd9\x82\xd9\x88\xd9\x84\xd8\xaa\xd9\x8a 
> \xd9\x88\xd8\xa7\xd9\x87\xd8\xa7"

Is "bit" bidirectional text?

I see what looks like arabic UTF-8 above. Use the b suffix to make a bytes 
literal.


bit = b"\xd8\xa3\xd9\x88\xd9\x87 \xd8\xa8\xd8\xaf\xd9\x8a\xd9\x84 
\xd9\x85\xd9\x86 \xd9\x82\xd9\x88\xd9\x84\xd8\xaa\xd9\x8a 
\xd9\x88\xd8\xa7\xd9\x87\xd8\xa7"

Now you can decode that into a unicode string:

text = bit.decode()

If you want to write that into a file then as unicode:

with open('ss.txt', 'w') as f:
f.write(text)

If you want to write the utf-8 bytes then:

with open('ss.txt', 'wb') as f:
f.write(bit)

or

with open('ss.txt', 'wb') as f:
f.write(text.encode())

Note: encode() and decode() both default to utf-8.

> # here it is a byte but in str 
> encode_bit = bytes(bit , "latin-1") # i can here say to the computer it is 
> byte

Using latin-1 is not helping in this case.

I hope you can see how to make the reading and writing of your text work now.

> 
> #then it will print the same words in but byte
> print(encode_bit)
> decode = encode_bit.decode() # and i can then decode it and get what i want
> print( decode )
> 
> bit = open("ss.txt","r")
> bit = bit.read()
> # but here i read the file and its hold the same bytes above
> encode_bits = bytes(bit , "latin-1") # but here when i want to say it is 
> bytes it encode it so if it was /xa33 it will be //xa33 

> decode = encode_bits.decode()
> # and when i decode it i get /xa33 not the same first one
> print( decode)

Barry


> 
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/MMHWLHWGMT6A6APMICYXJ3JQZIC5D2ZQ/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/N2V3W6OL6IXRIUONRF2H4SW37PLA5VAY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: CLI convenience?

[Barry Scott]

> On 20 Feb 2022, at 08:56, Eliot Lear  wrote:
> 
> Hi everyone,
> 
> I'm sure there's a clear answer, probably written some place obvious, but I 
> wonder if there is a reason why certain functions are not linked to 
> console_scripts.  In particular I would think this would be good for 
> json.tool (though I might call it something else, like pyjson_pp).  Same with 
> venv. I'm sure I'm missing something obvious.

You can use from the CLI like this for modules that a __main__.py has been 
provided.
This includes pip, json, venv, tarfile and zip file (off the top of my head).

python -m pip list
python -m json 
python -m venv
etc.

On windows you would use py not python for preference:

py -m pip list
etc.

This is documented for the modules I named above. Is that what you are after?

Barry

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/TQKG7LAO57YHPPPXJ2H7E2MSCA7UYU66/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Regex timeouts

[Barry]

> On 17 Feb 2022, at 01:04, Tim Peters  wrote:
> 
> [J.B. Langston  ]
>> Thanks for the conclusive answer.
> 
> Not conclusive - just my opinion. Which is informed, but not infallible ;-)
> 
>> I will checkout the regex library soon.
> 
> You may not realize how easy this is? Just in case: go to a shell and type
> 
> pip install regex
> 
> (or, on Windows, "python -m pip install regex" in a DOS box).
> 
> That's it. You're done. Now you can use regex. In some cases, you can
> put "import regex as re" at the top of a module  At worst, replace
> instances of "re" with "regex". Stay away from the new features, and
> it's highly compatible with Python;s re.

I suspect that like me what was meant is that checkout means read the docs
to understand regex features. The install is trivial.

Barry

> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/C35D2Z5GZSWN3T46OR5LECU7VG5YD3LQ/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KLSG7EECZ3UWH4KJ7PAAOEI37XWTKUNP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Adding sortedconatiners to Python or merge the ideas?

[Barry]

> On 3 Feb 2022, at 18:53, Abdur-Rahmaan Janhangeer  
> wrote:
> 
> Fine discussion but i wonder how it ended up over there.
> Hoping over. Thanks for pointers!

Please don’t top post.

The thinking on python dev seems to be that if a package is not needed by core 
python
then PyPI is the best place for it. I know you read python dev isn’t that what 
you
have sensed as well?

Barry


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AKJQLJZ6KX5SBXSB52V6LMV4AWNOO45Q/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Applications user/system directories functions

[Barry]

> On 15 Dec 2021, at 13:45, JGoutin via Python-ideas  
> wrote:
> 
> Hello,
> 
> The idea is to add 3 functions to get "config", "data" and "cache" 
> directories that are commonly used to store application files in user home / 
> system.
> 
> This look very straightforward to get theses directories path, but in 
> practices it depends on many factors like OS, environnement variables, user 
> or system dir.
> 
> For instance, with the "config" directory:
> * Linux, user: os.path.join(os.getenv("XDG_CONFIG_HOME", 
> os.path.expanduser("~/.config")), app_name)
> * Linux, system: os.path.join("/etc", app_name)
> * Windows, user: os.path.join(os.path.expandvars("%APPDATA%"), app_name)
> * Windows, system: os.path.join(os.path.expandvars("%CSIDL_COMMON_APPDATA%"), 
> app_name)
> 
> For linux, the full spec is here: 
> https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html
> 
> I see many applications that just use "~/.app_name" to not have to handle 
> theses cases.
> 
> 
> The functions prototypes may look like and may be added to "shutil" (or 
> "os.path" ?):
> 
> def getcachedir(app_name: str=None, system: bool=False):
> 
> With
> * app_name: The application name
> * system: If the required directory is the systemd directory or user 
> direcotry.
> 
> 
> This may also be implemented as an external library, but I am not sure I 
> would like add add a dependency to my projects "just for this".
> 
> 
> I can implement this if people are interested with this feature.

I wrote this https://pypi.org/project/config-path/ to solve the problem for 
macOs, windows and unix.

There are subtle points to consider on each platform.

Barry



> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/MHEWO4U6SBDU7OU3JH4A62EWCANDM7I2/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LZP4ISNMJETVHCVMOTQR4YKBRY3RODEU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671 review of default arguments evaluation in other languages

[Barry Scott]

> On 6 Dec 2021, at 00:31, Finn Mason  wrote:
> 
> On Sun, Dec 5, 2021, 12:11 PM Brendan Barnwell  <mailto:brenb...@brenbarn.net>> wrote:
> On 2021-12-04 20:01, David Mertz, Ph.D. wrote:
> >
> > There are perfectly good ways to "fake" either one if you only have the
> > other. Probably more work is needed to simulate early binding, but there
> > are ways to achieve the same effect.
> >
> > However, that language would not be Python. That ship sailed in 1991.
> > What's being discussed here isn't changing the behavior of binding in
> > `def f(foo=bar)`.
> >
> > Instead, it's a discussion of adding ADDITIONAL syntax for late-binding
> > behavior. I think the proposed syntax is the worst of all the options
> > discussed. But the real issue is that the cases where it is relevant are
> > vanishingly rate, and the extra cognitive, teaching, and maintenance
> > burden is significant.
> 
> This is a key point that I mentioned in another message (although 
> that 
> message doesn't seem to have reaches the list for some reason). 
> Steven's list is very useful but I don't see any mention there of 
> languages that allow BOTH late-binding and early-binding, and 
> distinguishes them with some kind of syntactic flag in the signature.
> 
> Is its not being done before really a good argument against it? It may be 
> that there simply hasn't been a need in the other languages.
> 
> Also, on a kind of side note, what would be a situation where early binding 
> is advantageous to late binding? I can't think of one off the top of my head.

You can shoot your self in the foot with early-binding with ease.
You have to try harder with late-binding :-)

Barry

> 
> -- 
> Brendan Barnwell
> "Do not follow where the path may lead.  Go, instead, where there is no 
> path, and leave a trail."
> --author unknown
> ___
> Python-ideas mailing list -- python-ideas@python.org 
> <mailto:python-ideas@python.org>
> To unsubscribe send an email to python-ideas-le...@python.org 
> <mailto:python-ideas-le...@python.org>
> https://mail.python.org/mailman3/lists/python-ideas.python.org/ 
> <https://mail.python.org/mailman3/lists/python-ideas.python.org/>
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/WFCIHHCDJBMBX7J4SNHBNMXTHIP7SJ6D/
>  
> <https://mail.python.org/archives/list/python-ideas@python.org/message/WFCIHHCDJBMBX7J4SNHBNMXTHIP7SJ6D/>
> Code of Conduct: http://python.org/psf/codeofconduct/ 
> <http://python.org/psf/codeofconduct/>
> 
> --
> Finn (Mobile)
> ___
> Python-ideas mailing list -- python-ideas@python.org 
> <mailto:python-ideas@python.org>
> To unsubscribe send an email to python-ideas-le...@python.org 
> <mailto:python-ideas-le...@python.org>
> https://mail.python.org/mailman3/lists/python-ideas.python.org/ 
> <https://mail.python.org/mailman3/lists/python-ideas.python.org/>
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/6H5K6SGUDIXPISJVG4SYP2PCGWCWPTRI/
>  
> <https://mail.python.org/archives/list/python-ideas@python.org/message/6H5K6SGUDIXPISJVG4SYP2PCGWCWPTRI/>
> Code of Conduct: http://python.org/psf/codeofconduct/ 
> <http://python.org/psf/codeofconduct/>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/P4QN2Q4W7OA57EWKACAS3VL7TK4IRBVI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!

[Barry Scott]

> On 4 Dec 2021, at 21:21, Chris Angelico  wrote:
> 
> On Sun, Dec 5, 2021 at 5:29 AM Barry Scott  <mailto:ba...@barrys-emacs.org>> wrote:
>> 
>> 
>> 
>>> On 1 Dec 2021, at 06:16, Chris Angelico  wrote:
>>> 
>>> I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/
>>> with some additional information about the reference implementation,
>>> and some clarifications elsewhere.
>> 
>> (I suspect that there was a reply that I should be replying to but, cannot 
>> find one appropriate)
>> 
>> I have a lot of code that exploits the fact that passing an explicit None 
>> will cause the early bound default idiom to set the default for me.
>> 
>> def inner(timestamp=None):
>>if timestamp is None:
>>timestamp = time.time()
>>do_stuff...
>> 
>> def outer(timestamp=None):
>>inner(timestamp=timestamp)
>> 
>> outer can in an idiomatic way have inner default timestamp and not have to 
>> know what that means.
> 
> If you need outer() to be able to have a value that means "use the
> default", then there are three options:
> 
> 1) Don't pass timestamp at all. In simple cases where it will only and
> always specify the default, this is fine.
> 2) Define a sentinel that is indeed part of your API.
> 3) Use *args or **kwargs to choose whether to pass it or not (best if
> there are multiple of them).
> 
> You can continue to use the existing system of "if none, do this", or
> you can flip it around and have the sentinel as a special token within
> your code:
> 
> def inner(timestamp=>time.time()):
>if timestamp is None: timestamp = time.time()

And, obviously, if you end up needing the write the explicit check for None 
there is no
advantage to using late bound default.

> 
> Depends on how important this feature is outside of your own helper
> functions. (I would probably not do this for None specifically - if
> it's purely internal, I'm more likely to use a dedicated local
> sentinel object.)
> 
> But as soon as there are two or three arguments that "might have to be
> passed, might not", it's far more readable to use kwargs to pass just
> the ones you want.
> 
> def outer(**kwargs):
>inner(**kwargs)
> 
> That way, if something changes in inner(), you don't have to worry
> about breaking your caller's API.

Yes that's a good point. Use the *kwargs style to pass down stuff.

> 
>> With late bound I cannot do this without more complex pattern of building an 
>> arg list.
>> 
>> What if passing None still worked? I know the argument that there are more 
>> sentinels then None.
>> 
>> def inner(timestamp=>time.time())
>>do_stuff...
>> 
>> def outer(timestamp=None):
>>inner(timestamp=timestamp)
>> 
>> The code in inner that decides to when to allow the default could check for 
>> timestamp being
>> missing or arg present and None.
>> 
>> Would the lack of support for other sentinels out weight the simple way to 
>> get the default applied?
>> 
> 
> None is most assuredly not going to trigger a late-bound default.

Are you state that this is because in most of the cases where I might think 
that I need this behaviour there are better patterns to use like *kwargs?
Is that worth stating in the PEP in the rejected ideas?

> Python is not JavaScript :)

Thank your choice of deity for that!

Barry

> 
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org 
> <mailto:python-ideas@python.org>
> To unsubscribe send an email to python-ideas-le...@python.org 
> <mailto:python-ideas-le...@python.org>
> https://mail.python.org/mailman3/lists/python-ideas.python.org/ 
> <https://mail.python.org/mailman3/lists/python-ideas.python.org/>
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/QWYXRITR56CKURYKE7CKQ7A4WVNTUVJL/
>  
> <https://mail.python.org/archives/list/python-ideas@python.org/message/QWYXRITR56CKURYKE7CKQ7A4WVNTUVJL/>
> Code of Conduct: http://python.org/psf/codeofconduct/ 
> <http://python.org/psf/codeofconduct/>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/F2ILJ42IWMDH6V3T2AJDMLWA2AIV43MY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!

[Barry Scott]

> On 1 Dec 2021, at 06:16, Chris Angelico  wrote:
> 
> I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/
> with some additional information about the reference implementation,
> and some clarifications elsewhere.

(I suspect that there was a reply that I should be replying to but, cannot find 
one appropriate)

I have a lot of code that exploits the fact that passing an explicit None will 
cause the early bound default idiom to set the default for me.

def inner(timestamp=None):
if timestamp is None:
timestamp = time.time()
do_stuff...

def outer(timestamp=None):
inner(timestamp=timestamp)

outer can in an idiomatic way have inner default timestamp and not have to know 
what that means.

With late bound I cannot do this without more complex pattern of building an 
arg list.

What if passing None still worked? I know the argument that there are more 
sentinels then None.

def inner(timestamp=>time.time())
do_stuff...

def outer(timestamp=None):
inner(timestamp=timestamp)

The code in inner that decides to when to allow the default could check for 
timestamp being
missing or arg present and None.

Would the lack of support for other sentinels out weight the simple way to get 
the default applied?

Barry

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/L7YDL225ETTLWAB64NJY5CJJUABUI4NB/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!

[Barry Scott]

> On 4 Dec 2021, at 09:44, Steven D'Aprano  wrote:
> 
> On Sat, Dec 04, 2021 at 03:14:46PM +1100, Chris Angelico wrote:
> 
>> Lots and lots and lots of potential problems. Consider:
>> 
>> def f():
>>a = 1
>>def f(b, x=>a+b):
>>def g(): return x, a, b
>> 
>> Both a and b are closure variables - one because it comes from an
>> outer scope, one because it's used in an inner scope. So to evaluate
>> a+b, you have to look up an existing closure cell, AND construct a new
>> closure cell.
>> 
>> The only way to do that is for the compiled code of a+b to exist
>> entirely within the context of f's code object.
> 
> I dispute that is the only way. Let's do a thought experiment.

There are many possible implementation of the late bound idea that could create 
an object/default expression.
But is it reasonable to bother with that added complexity/maintenance burden 
for a first implementation.
And maybe no one will care enough to ever implement the ability to modify the 
code of a late bound
variables expression as a separate object later.

I think I understand the argument as being along the lines of
for early bound defaults they can be inspected and modified.
Therefore being able to do the same for late bound defaults must be implemented.

I'm not convinced that that is reasonable to require is implemented.

If python had always had late bound defaults, as it is with most languages in 
the survey
posted earlier in this thread, would that have been implemented as an 
object/expression?
Maybe, but I doubt it.

Summary: I agree it's not impossible, I do not agree that it's needed.

Barry

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/2GSTDHBKU55ZWUPVVGWRL44EYV54KFLV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Official means to release the GIL in Python

[Barry]

> On 3 Dec 2021, at 15:23, TobiasHT  wrote:
> 
> The GIL has been a widely discussed topic in the Python community. It's has 
> it's advantages and disadvantages.
> I was suggesting that an official way be placed in the Python threading 
> module to release the GIL if one needs to perform some tasks that don't need 
> the GIL.
> 
> It could be something as simple as
> 
> thd = threading.Thread(target = encrypt, nogil = True)
> 
> I saw a nogil option in Cython, I just thought it would be great to have it 
> fully support in Python as we await the results from Sam Gross's paper.

You must hold the GIL to execute python code.
Only extensions written in C etc can release the GIL while they do
work that does not access any python API.

Cython generates C code I believe and can release the GIL because it
I’d not python code.

In other words this cannot be made to work.

Barry

> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/5PWD7MXW6EO3ZECUPNXUQHE6HBYZ4JPU/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZTHSAFW3E72TXVX6XOYGV5YMZS4IA7OD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!

[Barry Scott]

> On 1 Dec 2021, at 17:59, Chris Angelico  wrote:
> 
> On Thu, Dec 2, 2021 at 4:40 AM Barry Scott  wrote:
>> 
>> On 1 Dec 2021, at 06:16, Chris Angelico  wrote:
>> 3) If "yes" to question 1, would you use it for any/all of (a) mutable
>> defaults, (b) referencing things that might have changed, (c)
>> referencing other arguments, (d) something else?
>> 
>> 
>> yes (a)
>> What does (b) mean? example please.
>> yes (c)
>> 
> 
> global_default = 500
> def do_thing(timeout=>global_default): ...

> 
> If the global_default timeout changes between function definition and
> call, omitting timeout will use the updated global.
> 
> Similarly, you could say "file=>sys.stdout" and if code elsewhere
> changes sys.stdout, you'll use that.

On a case-by-case basis I might still put defaulting into the body
of the function if that made the intent clearer.

I could see me using @file=sys.stdout.

Barry
> 
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/KVLO3CEXBJUKBUJPJZIJM54U6S5PIFKM/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IZIGDOR2NE2N774SJ4PZ4L7IOWVTUDKF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671 (late-bound arg defaults), next round of discussion!

[Barry Scott]

> On 1 Dec 2021, at 06:16, Chris Angelico  wrote:
> 
> I've just updated PEP 671 https://www.python.org/dev/peps/pep-0671/
> with some additional information about the reference implementation,
> and some clarifications elsewhere.
> 
> *PEP 671: Syntax for late-bound function argument defaults*
> 
> Questions, for you all:
> 
> 1) If this feature existed in Python 3.11 exactly as described, would
> you use it?

no because of name=>

> 
> 2) Independently: Is the syntactic distinction between "=" and "=>" a
> cognitive burden?

yes.

> 
> (It's absolutely valid to say "yes" and "yes", and feel free to say
> which of those pulls is the stronger one.)
> 
> 3) If "yes" to question 1, would you use it for any/all of (a) mutable
> defaults, (b) referencing things that might have changed, (c)
> referencing other arguments, (d) something else?

yes (a)
What does (b) mean? example please.
yes (c)

> 
> 4) If "no" to question 1, is there some other spelling or other small
> change that WOULD mean you would use it? (Some examples in the PEP.)

Use the @name to avoid the confusing with the set of = things.

> 
> 5) Do you know how to compile CPython from source, and would you be
> willing to try this out? Please? :)

no promises, if I get spare time I'll give it a go,
should be easy to hack the Fedora python RPM to build your
version.

> 
> I'd love to hear, also, from anyone's friends/family who know a bit of
> Python but haven't been involved in this discussion. If late-bound
> defaults "just make sense" to people, that would be highly
> informative.
> 
> Any and all comments welcomed. I mean, this is python-ideas after
> all... bikeshedding is what we do best!
> 
> The reference implementation currently has some test failures, which
> I'm looking into. I'm probably going to make this my personal default
> Python interpreter for a while, to see how things go.

Barry

> 
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/UVOQEK7IRFSCBOH734T5GFJOEJXFCR6A/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KPXSG43UU5ARO4HRIGSMYEDVEGWIBDKR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: easier writing to multiple streams

[Barry Scott]

> On 26 Nov 2021, at 16:00, eyalgr...@gmail.com wrote:
> 
> i wonder whether:
> 
> from myutils import myprint as print
> 
> or
> 
> _print = print
> print = myprint
> 
> is really the pythonic way?

The problem with this is that there are more ways to output then the print 
command.
Only replacing print will not catch any code that uses sys.stdout.write() for 
example.

That is why when I need this I replace sys.stdout and usually also sys.stderr 
with a file like object.
BTW did you know that you can get the original sys.stdout is also available in 
sys.__stdout__?

> my use case for multiple files on top of the stdout, is when using e.g. wandb 
> which is a popular ML dashboard and experiment logging platform. i want to 
> write my log file both the a local log.txt and to a second copy in the 
> temporary local wandb folder that later gets synced to the cloud. otherwise i 
> have to take care of copying over the file later including in cases of 
> exceptions. more generally: writing a log to both a local and a remote 
> location.

As has been already suggested, you can use the logging module and add handlers 
for each of the log files that you want to create at the same time,

I have also replaced sys.stdout with an object that takes all lines written to 
sys.stdout and sends them to the logger.

Barry


> 
> thanks
> eyal
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/PEH5SGQWO22OTOQZ6TPOHL3IWYMBPQWY/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/M2LSNHAVCRTEVE5V4O75ROAD3SCIEVEF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: easier writing to multiple streams

[Barry]

> On 25 Nov 2021, at 13:07, Eyal Gruss  wrote:
> 
> 
> hi
> 
> first post here :)
> i have a recurring use pattern where i want to save to file everything i 
> print to the screen. i have used the standard logging module and found it too 
> cumbersome for my simple use case, and i find myself changing all my 
> print()'s to custom myprint()'s. i am interested in suggesting several 
> additions to the standard libraries, as i feel this use case is common 
> enough, important enough, and currently painful enough to justify such 
> changes, and that all four of the following suggestions could be useful in a 
> complementary fashion.
> 
> 1. allow the print() "file" argument to take a list of streams (currently it 
> can take only a single stream). e.g:
> 
> with open('output.txt', 'w') as f:
>print('hello', file=[sys.stdout, f])

You can replace sys.stdout with your own object that prints to a list of 
streams.
I have used this in the past.

The implementation can be as simple as supplying an object with a write method 
that writes to each of the steams I turn.

Barry
> 
> this seems like the quickest and least resistant path for the user. need to 
> decide how multiple streams will work with the flush argument. 
> 
> 2. modify contextlib.redirect_stdout() to take an additional boolean argument 
> which we can call tee/replicate/duplicate/clone etc. when True it will 
> duplicate instead of just redirecting:
> 
> with open('output.txt', 'w') as f:
> with redirect_stdout(f, tee=True):  # will duplicate instead of 
> redirecting
> print('hello')
> 
> or we could add a new context manager contextlib.copy_stdout() 
> 
> 3. allow the contextlib.redirect_stdout() "new_target" argument to take a 
> list of streams, or allow specifying multiple arguments for multiple streams. 
> e.g:
> 
> with open('output.txt', 'w') as f:
> with redirect_stdout(sys.stdout, f):  # or redirect_stdout([sys.stdout, 
> f]) 
> print('hello')
> 
> this has the benefit over the tee argument of allowing more than one 
> additional stream, but you need to explicitly specify stdout. so would be 
> nice to have both modifications.
> 
> 4. add to the standard io library a new class which gives you the write 
> interface of a single stream, but is a wrapper that will write to multiple 
> streams:
> 
> with open('output.txt', 'w') as f:
>multi = io.OutputStreamCollection(sys.stdout, f)
>multi.write('hello\n') 
># or:  print('hello', file=multi)
> 
> this is similar to: 
> https://stackoverflow.com/questions/9130755/wrapper-to-write-to-multiple-streams,
>  and we need to decide about the rest of the stream methods.
> 
> eyal.
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/7I55RDLFAV6A3K762TE3BW42WTTUE4ET/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/526YR6GSE3CNII27GQDVXVUGETNMF5BF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Should bare logical comparisons raise a warning?

[Barry]

> On 25 Nov 2021, at 13:06, ehmatt...@gmail.com wrote:
> 
> Hi everyone,
> 
> I've been really impressed by the recent improvements to error messages in 
> Python 3.10. The specific suggestions in some error messages are going to 
> save countless hours of people's time. The benefits for beginners are fairly 
> obvious. But even for experienced programmers, the difference between seeing 
> a missing colon in an error message and just adding it where it should have 
> been vs spending 30 seconds or a minute figuring out what's missing at the 
> start of a loop adds up.
> 
> One common error that I haven't seen discussed is bare logical comparisons. 
> They're syntactically legal so they don't raise errors, but I have never seen 
> a real-world use case for one. Here's the simplest example I can come up with:
> 
> name = 'eric'
> print(name)
> 
> name == 'ever'

The __eq__ method is run which might have side effects.
Which may mean raising error may be a backwards compat issue,
Maybe it is a problem for linters to report as a problem.
I have not checked if they already do.

> print(name)
> 
> We want to assign a new value to an existing variable, but we accidentally 
> type a second equals sign. The output here makes the mistake fairly obvious, 
> but in longer programs it can take a while to spot this issue.
> 
> I know there are some editors and linters that flag this. Is there any reason 
> not to raise a warning or even an error at the language level? Is there any 
> reason to use a bare logical comparison in real-world code?
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/4QVUWND6FFDWCD3DL3BR3763EQRAAI5D/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VTEHJONXDAV3SX2IOCIOM3Q43J3SRCQ7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

[Barry Scott]

> On 25 Oct 2021, at 08:08, Steven D'Aprano  wrote:
> 
> I would say that it makes most sense to assign early-bound defaults 
> first, then late-bound defaults, specifically so that late-bound 
> defaults can refer to early-bound ones:
> 
>def func(x=0, @y=x+1)
> 
> So step 3 above should become:

In this case you do not need a new rule to make it work as in the left-to-right 
order
x = 0 first.

 def func(@y=x+1, @x=0):

Is it unreasonable to get a UnboundLocal or SyntaxError for this case?

I'm not convinced that extra rules are needed.

Barry


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BAETFKAW6GJUC5G5LRNBEFAALUOJZP66/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP 671: Syntax for late-bound function argument defaults

[Barry Scott]

> On 24 Oct 2021, at 01:13, Chris Angelico  wrote:
> 
> Specification
> =
> 
> Function default arguments can be defined using the new ``=>`` notation::
> 
>def bisect_right(a, x, lo=0, hi=>len(a), *, key=None):
>def connect(timeout=>default_timeout):
>def add_item(item, target=>[]):
> 
> The expression is saved in its source code form for the purpose of inspection,
> and bytecode to evaluate it is prepended to the function's body.
> 
> Notably, the expression is evaluated in the function's run-time scope, NOT the
> scope in which the function was defined (as are early-bound defaults). This
> allows the expression to refer to other arguments.
> 
> Self-referential expressions will result in UnboundLocalError::
> 
>def spam(eggs=>eggs): # Nope
> 
> Multiple late-bound arguments are evaluated from left to right, and can refer
> to previously-calculated values. Order is defined by the function, regardless
> of the order in which keyword arguments may be passed.


Clarification please:

What is the bytecode that will be generated?

Does the bytecode only run the default code if the argument is missing?

And missing is not the same as is None?

Also have you add the @var=default suggestion from Stephen to the syntax 
options.
I'm +1 on the @ syntax as it is easier to pick up on and the other reasons that 
Stephen
provided.

Barry

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/R6K2AS5QRC6H5FMJBATB6SRZUJPPXDZE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Allow regex group name redefinitions

[Barry Scott]

> On 2 Oct 2021, at 10:27, ven...@razdva.cz wrote:
> 
> Hello everybody,
> 
> I've got a suggestion for the std. re module developers: to consider allowing 
> match group name redefinitions, especially in alternatives.
> While you may not see the point at first glance, let me try to reason such a 
> thing using a real-world example from my practice:
> 
> Imagine a company, using certain codes for their products/product components 
> (unfortunately, I'm not at liberty to disclose the true
> nature of them, but bare with me).
> Let's say that they may have the following forms:
> r"(?PAB|C|D)[- ](?P[A-Z])?(?P\d+)(?P[A-Z])?"
> 
> So far so good. But now, imagine that a particular type of code has a bit 
> different syntax:
> r"(?PE)[- ](?P[A-Za-z])?(?P\d+)[- 
> ](?P[A-Za-z])?"
> 
> As you can see, the prefix & postfix may be lowercase in case of code type E 
> and moreover, a space or dash is required before the postfix.
> If I merged the definitions, I'd have to allow that syntax even for the AB, C 
> and D code types---but that would've been incorrect and would
> require post-matching checks.
> 
> Ideally I'd like to have the opportunity to define the regex as an 
> alternative:
> r"(?PAB|C|D)[- 
> ](?P[A-Z])?(?P\d+)(?P[A-Z])?|(?PE)[- 
> ](?P[A-Za-z])?(?P\d+)[- ](?P[A-Za-z])?"
> 
> I can't, of course, getting the re.error: redefinition of group name error 
> upon the regex compilation.
> 
> But is that really a problem, especially in such alternatives?
> If you imagine the regex as a FSA, the code type branches into completely 
> independent sub-trees of the automaton state transitions.
> There's no problem with efficiency; the regex might look a bit complex, but 
> the matching is perfectly efficient---definitely more so than if I match 
> multiple expressions.
> The redefinition of the match group names is IMO technically perfectly 
> possible and note that in such alternatives, re-assignments won't really 
> happen.
> And finally, even if they would happen, what's the problem with that? Might 
> be a logical error in the regex definition of course, but that's the 
> programmer's lookout in general...
> 
> So what do you think?
> If the match group name redefinition was allowed, I could just match a single 
> regex, getting match group dict and read out parsed parts of the codes by 
> name---nice and easy.
> Currently, my 2 choices are:
> 1/ Use uniquely named groups, which requires me to do a post-match group name 
> consolidation of sort or
> 2/ Match multiple reg. expressions, which is unnecessary
> 
> Therefore, I ask you to reconsider issuing the error, which I deem redundant 
> and unnecessarily limiting a justified use-case, IMO.
> Also note that doing that won't break any old code---anything that worked 
> before will continue to work with unchanged semantics; so such a change would 
> be perfectly safe.

Faced with this problem I would write a parser for the product codes that 
understands the syntax and break it into pieces that make sense.
I would not use regex in the parser.

Barry



> 
> Thanks,
> 
> Best Regards
> 
> vasek
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/K2FXXQ2XG75FPDIJIDP4HHXXKCMYRP4I/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HTDGZ6HW2Z32ZNBLIEKIOLJEIA4I3WR5/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Syntax Sugar for __name__ == "__main__" boilerplate?

[Barry Scott]

> On 2 Oct 2021, at 18:17, Paul Bryan  wrote:
> 
> Thanks for finding that.
> 
> While I don't feel strongly one way or the other, I do think the discussion 
> is worthwhile.
> 
> As I understand, the arguments for:
> - let's get rid of boilerplate, that many (esp. beginners) may not understand
> - you could add command line arguments and return values in a more natural way
> 
> As I understand, the arguments against:
> - it's just 2 lines of code; this isn't a big problem being solved
> - boilerplate serves to teach a fundamental concept about the loading and 
> execution of Python modules
> - there may not be a clean way to maintain compatibility with previous 
> versions of Python

It is 2 lines of code that a beginner does not need to write at all.
I see a lot of short scripts that do not bother with the main function at all,
they just put all the code at module level. And why not?
Indeed I do not bother for short scripts either.

Once someone is doing more complex scripting then the benifits of the
2 lines and main function become something worth understanding and using.
Namely it allows importing the code and testing it.

Barry

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/56ZWQTVG4LJKLN3BM5YNW776SRYZLXZ6/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Better exception handling hygiene

[Barry Scott]

> On 30 Sep 2021, at 17:25, Soni L.  wrote:
> 
> Alright, some ppl asked us to rephrase this, so:
> 
> The plan is to take the function syntax:
> 
> def name(args):
> 
> and add an optional "with" to it:
> 
> def name(args) with exceptions:
> 
> these then get added to the function, similar to e.g. default args. when
> an exception is thrown*, the VM then checks these and converts relevant
> exceptions into RuntimeError, e.g.:
> 
> def foo():
> raise Bar
> def baz() with Bar:
> foo()
> baz()
> 
> would make a RuntimeError, because foo raised a Bar and the VM sees that
> Bar is in baz's with.

Does with Bar mean that Bar is expected?

If so who cares if foo raises it? Are you really saying I cannot call functions 
to
implement a complex algorithm that raises exceptions?

The caller of baz is expecting Bar right?

> 
> *except "raise" opcodes SKIP checking these (within the context of the
> function), so the following:
> 
> def baz() with Bar:
> raise Bar
> baz()
> 
> raises a Bar, not a RuntimeError from a Bar.

You want to include the exceptions that a function can raise in its signature 
and have python enforce rules based on that information.

C++ had/has this feature and it failed in practice so it has been deprecated.
I'd be surprised that it will be useful in python given this experience in the 
C++ world.

Barry


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/K52CEUI2EAHGK2NSGUOM2OHW66MNZ377/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: multiprocessing: hybrid CPUs

[Barry]

> On 18 Aug 2021, at 16:03, Chris Angelico  wrote:
> 
> On Thu, Aug 19, 2021 at 12:52 AM Marc-Andre Lemburg  wrote:
>> 
>>> On 18.08.2021 15:58, Chris Angelico wrote:
>>> On Wed, Aug 18, 2021 at 10:37 PM Joao S. O. Bueno  
>>> wrote:
>>>> 
>>>> So,
>>>> It is out of scope of Pythonmultiprocessing, and, as I perceive it, from
>>>> the stdlib as a whole to be able to allocate specific cores for each 
>>>> subprocess -
>>>> that is automatically done by the O.S. (and of course, the O.S. having an 
>>>> interface
>>>> for it, one can write a specific Python library which would allow this 
>>>> granularity,
>>>> and it could even check core capabilities).
>>> 
>>> Python does have a way to set processor affinity, so it's entirely
>>> possible that this would be possible. Might need external tools
>>> though.
>> 
>> There's os.sched_setaffinity(pid, mask) you could use from within
>> a Python task scheduler, if this is managing child processes (you need
>> the right permissions to set the affinity).
> 
> Right; I meant that it might require external tools to find out which
> processors you want to align with.
> 
>> Or you could use the taskset command available on Linux to fire
>> up a process on a specific CPU core. lscpu gives you more insight
>> into the installed set of available cores.
> 
> Yes, those sorts of external tools.
> 
> It MAY be possible to learn about processors by reading /proc/cpuinfo,
> but that'd still be OS-specific (no idea which Unix-like operating
> systems have that, and certainly Windows doesn't).

And next you find out that you have to understand the NUMA details
of your system because the memory attached to the CPUs is not the same speed.

> 
> All in all, far easier to just divide the job into far more pieces
> than you have processors, and then run a pool.

As other already stated using a worker pool solves this problem for you.
All you have to do it break your big job into suitable small pieces.

Barry
> 
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/UQNSUSHUONT4AO6NJEPEUENQG2AINAMO/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/62AXMS62J2H7TBHANIXZTTS2RJPUZZ5Z/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: C API for converting Python integers to/from bytes sequences

[Barry Scott]

> On 7 Aug 2021, at 19:22, Serhiy Storchaka  wrote:
> 
> Python integers have arbitrary precision. For serialization and
> interpolation with other programs and libraries we need to represent
> them as fixed-width integers (little- and big-endian, signed and
> unsigned). In Python, we can use struct, array, memoryview and ctypes
> use for some standard sizes and int methods int.to_bytes and
> int.from_bytes for non-standard sizes. In C, there is the C API for
> converting to/from C types long, unsigned long, long long and unsigned
> long long. For other C types (signed and unsigned char, short, int) we
> need to use the C API for converting to long, and then truncate to the
> destination type with checking for overflow. For integers type aliases
> like pid_t we need to determine their size and signess and use
> corresponding C API or wrapper. For non-standard integers (e.g. 24-bit),
> integers wider than long long, and arbitrary precision integers all is
> much more complicated. There are private C API functions
> _PyLong_AsByteArray and _PyLong_FromByteArray, but they are for internal
> use only.
> 
> I am planning to add public analogs of these private functions, but more
> powerful and convenient.
> 
> PyObject *PyLong_FromBytes(const void *buf, Py_ssize_t size,
>   int byteorder, int signed)
> 
> Py_ssize_t PyLong_AsBytes(PyObject *o, void *buf, Py_ssize_t n,
>  int byteorder, int signed, int *overflow)
> 
> PyLong_FromBytes() returns the int object. It only fails in case of
> memory error or incorrect arguments (e.g. buf is NULL).
> 
> PyLong_AsBytes() writes bytes to the specified buffer, it does not
> allocate memory. If buf is NULL it returns the minimum size of the
> buffer for representing the integer. -1 is returned on error. if
> overflow is NULL, then OverfowError is raised, otherwise *overflow is
> set to +1 for overflowing the upper limit, -1 for overflowing the lower
> limit, and 0 for no overflow.
> 
> Now I have some design questions.
> 
> 1. How to encode the byte order?
> 
> a) 1 -- little endian, 0 -- big endian
> b) 0 -- little endian, 1 -- big endian
> c) -1 -- little endian, +1 -- big endian, 0 -- native endian.

Use an enum and do not use 0 as a valid value to make mistakes easier to detect.
I think you are right to have big endian, little endian and native endian.
I do not think the numeric values of the enum matter (apart from avoiding 0).

> Do we need to reserve some values for mixed endians?

What is mixed endian? I would guess that its use would be application
specific - so I assume you would not need to support it.

> 
> 2. How to specify the reduction modulo 2**(8*size) (like in
> PyLong_AsUnsignedLongMask)?
> 
> Add yet one flag in PyLong_AsBytes()? Use special value for the signed
> argument? 0 -- unsigned, 1 -- signed, 2 (or -1) -- modulo. Or use some
> combination of signed and overflow?
> 
> 3. How to specify saturation (like in PyNumber_AsSsize_t())? I.e. values
> less than the lower limit are replaced with the lower limit, values
> greater than the upper limit are replaced with the upper limit.
> 
> Same options as for (2): separate flag, encode in signed (but we need
> two values here) or combination of other parameters.

Maybe a single enum that has:
signed (modulo)
signed saturate
unsigned (modulo)
unsigned saturate


> 
> 4. What exact names to use?
> 
> PyLong_FromByteArray/PyLong_AsByteArray,
> PyLong_FromBytes/PyLong_AsBytes, PyLong_FromBytes/PyLong_ToBytes?

Barry

> 
> 
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/V2EKXMKSQV25BMRPMDH47IM2OYCLY2TF/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4TSDAQG3BOACRUEH35MD3ME3WQGCZSUA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Pre PEP: Python Literals (was custom strings before)

[Barry Scott]

> On 5 Jul 2021, at 08:07, Thomas Güttler  wrote:
> 
> This means backticks, but without the dollar sign. 

In bash the backtick was so often a problem that $(cmd) was added.

Having removes the grit-on-Tim's-screen backtick in python 3 I would
not like to see it return with its issue of being confused with single-quote.

Barry

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3EU3RLCFRB3JEQ5Q6O7YMANATFOFYGNY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: joining paths without worrying about a leading slash

[Barry Scott]

> On 27 Jun 2021, at 12:07, Zbigniew Jędrzejewski-Szmek  
> wrote:
> 
> [this is a continuation of https://bugs.python.org/issue44452]
> 
> pathlib.Path() has a concatenation operator "/" that allows the
> right-hand-side argument to be an absolute path, which causes the
> left-hand-side argument to be ignored:
> 
>>>> pathlib.Path('/foo') / '/bar'
> PosixPath('/bar')
>>>> pathlib.Path('/var/tmp/instroot') / '/some/path' / '/suffix'
> PosixPath('/suffix')
> 
> This follows the precedent set by os.path.join(), and probably makes
> sense in the scenario of simulating a user typing 'cd' commands in a
> shell.
> 
> But it doesn't work nicely in the case of combining paths from
> two different "namespaces", where we never want to go "up".
> 
> For example: a web server takes an URL, strips the host, and wants
> to look up a file:
> https://example.com/some/path → "/some/path" → /src/www/root + /some/path → 
> /src/www/root/some/path
> 
> or we are constructing a container image and need to refer to a file
> in the container:
>  + /etc/shadow → /var/lib/machines/foo + /etc/shadow → 
> /var/lib/machines/foo/etc/shadow
> 
> To do this kind of operation correctly with pathlib.Path, the user
> needs to do two operations: verify that the rhs argument contains
> no '..' [*], and strip leading slashes:
> 
>>>> lhs = pathlib.Path('/some/namespace/')
>>>> rhs = '/some/path/to/add'
>>>> if '..' in pathlib.Path(rhs).parts: raise ValueError
>>>> path = lhs / rhs.lstrip('/')
> 
> Those last two lines are rather verbose, non-obvious. Also the .lstrip()
> operation attaches on the right side, but operates on the left side, earlier
> than the "/", which is overall not very nice.
> 
> Proposal: 
> 
> add "//"-operator to pathlib.PosixPath() that means "concatenate a rhs path
> that is underneath the lhs". It would disallow paths with '..', and 
> concatenate
> paths as relative to the specified lhs:
> 
>>>> lhs = pathlib.Path('/some/namespace/')
>>>> lhs // "a/b/c"
> PosixPath('/some/namespace/a/b/c')
>>>> lhs // "/a/b/c"
> PosixPath('/some/namespace/a/b/c')
>>>> lhs // "a/../b/c"
> ValueError: cannot use // with a path with '..' on the right
> 
> This would be useful for operations on containers, combining paths from
> namespaces like fs paths and URL components, looking up files
> underneath an unpacked archive, etc.
> 
> [*] Why completely disallow '..' ? Components with '..' cannot be
> correctly resolved without access to the filesystem, because a
> component may be a symlink, and then "a/b/../." may not be "a/.", but
> something completely different. Thus, since the goal is to have a path
> underneath lhs, I think it's best to forbid '..'. In principle '..' at
> the beginning can be resolved reliably, by simply ignoring it,
> '/../../../whatever' is the same as '/whatever/'. But it's a tiny
> corner case, and I think it's better to disallow that too.

There are two ideas here.

1. Allow Path() to join a pair of absolute paths.

2. Prevent '..' from escaping into the first absolute path.

For (1) you can do this today:

>>> root=Path('/var/www')
>>> root / y.relative_to('/')
PosixPath('/var/www/a/b')
>>>

I can think if a number of rules that might apply for (2).
(a) raise an error is there is a '..' or '.' in any path component.
(b) resolve() '..' and ','  as pathlib already does

- I'm not sure that use of the filesystem is needed to validate the use of .. 
is always needed.

>>> y=Path('/a/b/../v.html')
>>> y.relative_to('/')
PosixPath('a/b/../v.html')
>>> root / y.relative_to('/')
PosixPath('/var/www/a/b/../v.html')
>>> root / y.resolve().relative_to('/')
PosixPath('/var/www/a/v.html')

and show that no escape to root happens:

>>> y=Path('/../a//v.html')
>>> root / y.resolve().relative_to('/')
PosixPath('/var/www/a/v.html')
>>>

Barry

> 
> Zbyszek
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/IXYPKVINLD57BOV6VHU4U4ZJCQCQPAHT/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/6HR4IAUAUIQXK5SJAWKVFVOFZ374C4W3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Improved multi-tasking performance through deterministic GIL hand-off

[Barry Scott]

> On 10 May 2021, at 20:23, Barry Scott  wrote:
> 
> 
> 
>> On 10 May 2021, at 15:30, Sophist  wrote:
>> 
>> I don't know how many people will remember some work that David Beazley did 
>> about a decade ago on how the GIL impacts multithreading performance - 
>> essentially he instrumented the Python interpreter to log how multiple 
>> threads competed for  the GIL and gave several presentations over the space 
>> of 2-3 years. A couple of years ago I reached out to him with an idea on how 
>> to significantly improve the way that Python handles multi-threading 
>> hand-off of the GIL, but unfortunately he was not interested in pursuing 
>> this further. I am raising it here in the hope that someone else would be 
>> interested in implementing this.
>> 
>> In essence my idea is to stop Python handing off the GIL through a 
>> competition between threads that are ready to run, and instead for Python to 
>> implement a scheduler for the GIL which decides which thread should get the 
>> GIL next and directly hands it over. 
>> 
>> Background
>> 
>> Here are the links to David Beazley's presentations:
>> 
>> 2009: Inside the Python GIL  - 
>> https://www.youtube.com/watch?v=ph374fJqFPE
>> 2010: Understanding the Python GIL- 
>> https://speakerdeck.com/dabeaz/understanding-the-python-gil 
>> https://www.youtube.com/watch?v=Obt-vMVdM8s
>> 2011: Embracing the Global Interpreter Lock   - 
>> https://speakerdeck.com/dabeaz/embracing-the-global-interpreter-lock 
>> https://www.youtube.com/watch?v=fwzPF2JLoeU
>> 2011: In Search of the Perfect Global Interpreter Lock - 
>> https://speakerdeck.com/dabeaz/in-search-of-the-perfect-global-interpreter-lock
>>  https://www.youtube.com/watch?v=5jbG7UKT1l4
> 
> Given this is very old information I think the first thing needed is to 
> reproduce David's experiments and see if the 3.10 implementation has the same 
> issues.
> 
> Have you done this already?
> 
> If you turn these slides into benchmark code that would make it easier to 
> experiment with.
> 
> Benchmarks will need running on macOS, Windows, Linux at least.

It looks like the GIL code has not changed in a long time.

But for 3.7 FORCE_SWITCHING is always defined that changes the GIL behaviour.

This comment in Python/ceval_gil.h explains what that does:

  - When a thread releases the GIL and gil_drop_request is set, that thread
 ensures that another GIL-awaiting thread gets scheduled.
 It does so by waiting on a condition variable (switch_cond) until
 the value of last_holder is changed to something else than its
 own thread state pointer, indicating that another thread was able to
 take the GIL.

 This is meant to prohibit the latency-adverse behaviour on multi-core
     machines where one thread would speculatively release the GIL, but still
 run and end up being the first to re-acquire it, making the "timeslices"
 much longer than expected.
 (Note: this mechanism is enabled with FORCE_SWITCHING above)

Barry

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RPRFINFGHOMYOVQSEA64PXOVGQGVURLQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Improved multi-tasking performance through deterministic GIL hand-off

[Barry Scott]

> On 10 May 2021, at 15:30, Sophist  wrote:
> 
> I don't know how many people will remember some work that David Beazley did 
> about a decade ago on how the GIL impacts multithreading performance - 
> essentially he instrumented the Python interpreter to log how multiple 
> threads competed for  the GIL and gave several presentations over the space 
> of 2-3 years. A couple of years ago I reached out to him with an idea on how 
> to significantly improve the way that Python handles multi-threading hand-off 
> of the GIL, but unfortunately he was not interested in pursuing this further. 
> I am raising it here in the hope that someone else would be interested in 
> implementing this.
> 
> In essence my idea is to stop Python handing off the GIL through a 
> competition between threads that are ready to run, and instead for Python to 
> implement a scheduler for the GIL which decides which thread should get the 
> GIL next and directly hands it over. 
> 
> Background
> 
> Here are the links to David Beazley's presentations:
> 
> 2009: Inside the Python GIL  - 
> https://www.youtube.com/watch?v=ph374fJqFPE
> 2010: Understanding the Python GIL- 
> https://speakerdeck.com/dabeaz/understanding-the-python-gil 
> https://www.youtube.com/watch?v=Obt-vMVdM8s
> 2011: Embracing the Global Interpreter Lock   - 
> https://speakerdeck.com/dabeaz/embracing-the-global-interpreter-lock 
> https://www.youtube.com/watch?v=fwzPF2JLoeU
> 2011: In Search of the Perfect Global Interpreter Lock - 
> https://speakerdeck.com/dabeaz/in-search-of-the-perfect-global-interpreter-lock
>  https://www.youtube.com/watch?v=5jbG7UKT1l4

Given this is very old information I think the first thing needed is to 
reproduce David's experiments and see if the 3.10 implementation has the same 
issues.

Have you done this already?

If you turn these slides into benchmark code that would make it easier to 
experiment with.

Benchmarks will need running on macOS, Windows, Linux at least.

Barry

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AUASYYYOMQGGWLHOPMP4J5RGRQTDYCAK/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Integer concatenation to byte string

[Barry Scott]

> On 2 Mar 2021, at 23:49, Steven D'Aprano  wrote:
> 
> 
> [Barry]
>> All python byte code is interpreted by calling functions. They take 
>> time and resources.
> 
> That's not entirely correct. Literals such as text strings, ints and 
> floats get compiled directly into the byte-code. Now of course there is 
> some overhead while executing the byte-code, but that doesn't include 
> the heavy cost of a Python function call.

I was thinking of the C functions that are executed in ceval.c to run the 
interpreter
for any byte code.

Barry


> 
> 
> -- 
> Steve
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/7GTPEGXDAQRKWITBAGYWCU3MNY6JJE6U/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/T25IDROJ7NIUUCQTSVTACMWATH7MBAM3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Integer concatenation to byte string

[Barry Scott]

We where no longer on the ideas list...

> On 2 Mar 2021, at 13:04, Memz  wrote:
> 
> There is no specific scenario it solves. The lack of efficiency of the timed 
> code should speak for itself. Non-mutable bytes is a limit of python, since 
> it's reliant on using function calls.
> 
> b"\x00\x00\x00\x00\x00"
> bytearray( b"\x00\x00\x00\x00\x00" )
> struct.pack("i",0,0,0,0,0)
> b"\x00\x00\x00\x00\x00"  + bytes([1,0,0,0,0])

You mean the above is what you timed?
It is not a realistic problem you are measuring.

You also did not share how you measured the the code.

If you are not experienced in benchmarking there are variables
that you must control for to have meaningful results.

> 
> All function calls take time and resources, it would be impossible to 
> streamline a function call to make it faster than building it in. This goes 
> for most languages, including C.

All python byte code is interpreted by calling functions. They take time and 
resources.

Barry

> 
> 
> On Tue, Mar 2, 2021 at 3:29 AM Barry Scott  <mailto:ba...@barrys-emacs.org>> wrote:
> 
> 
> > On 2 Mar 2021, at 02:03, Memz  > <mailto:mmax42...@gmail.com>> wrote:
> > 
> > >When I needed to do the creation of bytes objects from a mix of types the
> > >struct.pack() method has been the obvious way to go.
> > 
> > >What is the use case that leads to needing the above?
> > 
> > >Barry
> > 
> > The use of my suggestion is to reduce the reliance of function calls for 
> > bytes and bytearrays, which currently can only be done through function 
> > calls, and making it more efficient all-together. Here is a timed version 
> > of this: 
> > b-strings: 5-5.4 s
> > bytearray() function call:  67.9 s
> > struct.pack() 80.9 s 
> > b-string + bytes([1,...])  54.3 s
> 
> What code are you benchmarking? What problem does that code solve?
> 
> My experience with creating byte objects is that struct is the fastest way to 
> get the job done.
> But that could be becuase of the problems that I need bytes for.
> For example calling ioctl().
> 
> > 
> > Moving bytearray to a non-function call would overhaul and optimize code 
> > that works with bytes, increase flexibility, and reduce reliance on 
> > imports, including struct.pack(). There is a lot of code that could, and 
> > should be using bytearray but can't, because other, more slow and painful 
> > than should be methods are more efficient because of the function call 
> > nature of bytearray.
> 
> You are assuming that the problem is the function calls.
> Surely its the algorithms that lead to issues in most code that is slow?
> 
> Barry


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UBJUDDMGPGMQPIVF5JV4CVAJYSSY2F35/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Integer concatenation to byte string

[Barry Scott]

> On 1 Mar 2021, at 18:01, mmax42...@gmail.com wrote:
> 
> Currently, the only way to concatenate an integer to a bytes object is by 
> converting the integer to bytes with a function call before concatenating. 
> And there is no way to make a mutable bytes object without a function call. 
> 
> I propose an array-type string like the, or for the bytearray. It would work 
> as a mutable b-string, as 
> 
> foo = a"\x00\x01\x02abcÿ"   # a-string, a mutable bytes object.
> foo[0] = 123  # Item assignment
> foo+= 255 # Works the same as 
> bytesvariable+=b"123"
> foo+= a"\x255\x00"  # Concatenation with itself
> foo+= b"\x255\x00"  # Cross compatibility with bytes objects.
> 
> This would be processed the same as, or would be the bytearray,
>>>> type(a"\x00\x01\x02abcÿ")
> 

When I needed to do the creation of bytes objects from a mix of types the
struct.pack() method has been the obvious way to go.

What is the use case that leads to needing the above?

Barry


> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/H5W3TTDC633NCPFUYIMQ26ZWX7XM4W6S/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/77KYAIS7BAJ2N6XZWHOABUVBR66L7DTS/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Barrier Object in asyncio lib

[Barry Scott]

> On 26 Feb 2021, at 08:31, Jonathan Slenders  wrote:
> 
> Barry,
> 
> What you describe sounds like `asyncio.gather(...)` if I understand correctly.
> 
> The thing with a Barier is that it's usable in situations where we don't know 
> the other tasks. Maybe there is no reference to them from the current scope. 
> Maybe they are even not yet created.
> It certainly can be done with a list of `asyncio.Future` and 
> `asyncio.gather(...)`, but that's a lot of boilerplate.
> 
> IMHO, Yves is right. For both asyncio and threading, we have Lock, Event, 
> Condition, Semaphore and BoundedSemaphore. Only Barier is missing among the 
> asyncio primitives. (RLock doesn't make sense.)

Why would you need locks for async? Is it to sync with things outside of the 
async process?

With the large and complex async app I work on there are no locks at all.

> (I guess we can probably go to bugs.python.org <http://bugs.python.org/> with 
> this proposal.)

Having shown that a Barrier for async is a missing piece it would be good to 
get a thumbs up here.

Barry

> 
> Jonathan
> 
> 
> 
> 
> 
> 
> Le jeu. 25 févr. 2021 à 23:38, Barry Scott  <mailto:ba...@barrys-emacs.org>> a écrit :
> 
> 
>> On 25 Feb 2021, at 17:15, Jonathan Slenders > <mailto:jonat...@slenders.be>> wrote:
>> 
>> It does make sense to have a barrier synchronization primitive for asyncio.
>> The idea is to make a coroutine block until at least X coroutines are 
>> waiting to enter the barrier.
>> This is very useful, if certain actions need to be synchronized.
> 
> I do most of my async coding with twisted where what you calling a barrier is 
> a DeferredList.
> 
> The way its used is that you add in all the deferreds that you want to 
> complete before you continue
> into the list. Once all the deferered have competed the DefferedList 
> completes and its callback is run.
> 
> Barry
> 
> 
>> 
>> Recently, I had to implement a barier myself for our use case. See code 
>> below:
>> 
>> It is simple to implement, but I too would like to have one for asyncio, in 
>> order to be consistent with the concurrency primitives we have for threading.
>> 
>> Jonathan
>> 
>> 
>> class Barier:
>> """
>> Make a coroutine block until there are at least X waiters.
>> 
>> Similar to the threading Barier objects but for asyncio:
>> https://docs.python.org/3/library/threading.html#barrier-objects 
>> <https://docs.python.org/3/library/threading.html#barrier-objects>
>> """
>> 
>> def __init__(self, parties: int) -> None:
>> self.parties = parties
>> self._waiting: int
>> self._event = asyncio.Event()
>> 
>> def add_one(self) -> None:
>> self._waiting += 1
>> if self._waiting == self.parties:
>> self._event.set()
>> 
>> async def wait(self, worker: "Worker") -> None:
>> """
>> Wait until all we have at least `parties` waiters.
>> """
>> self.add_one()
>> await self._event.wait()
>> 
>> 
>> 
>> 
>> Le jeu. 25 févr. 2021 à 16:42, Barry Scott > <mailto:ba...@barrys-emacs.org>> a écrit :
>> 
>> 
>> > On 25 Feb 2021, at 13:14, Yves Duprat > > <mailto:ydup...@gmail.com>> wrote:
>> > 
>> > Hi,the list,
>> > 
>> > I'm wondering why Barrier object does not exist in the synchronization 
>> > primitives of the asyncio lib while it is present in threading and 
>> > multiprocessing libs ?
>> > This may not be the right place to ask this question, but I never found an 
>> > answer on the web.
>> > Thanks for your help.
>> 
>> 
>> I'm assuming that the barrier you are speaking of is the mechanism that is 
>> used to
>> synchronise threads/processes running in parallel to prevent data races.
>> 
>> With async code that is never an issue. Each function runs to completion 
>> uninterrupted.
>> There are no data races. Each time a async function runs it can know that 
>> the state of
>> the objects it uses will not be changed while it is running.
>> 
>> Barry
>> 
>> 
>> 
>> > 
>> > Yves
>> > ___
>> > Python-ideas mailing list -- python-ideas@python.org 
>> > <mailto:python-ideas@python.org>
>> > To unsubscribe send an e

[Python-ideas] Re: Barrier Object in asyncio lib

[Barry Scott]

> On 25 Feb 2021, at 17:15, Jonathan Slenders  wrote:
> 
> It does make sense to have a barrier synchronization primitive for asyncio.
> The idea is to make a coroutine block until at least X coroutines are waiting 
> to enter the barrier.
> This is very useful, if certain actions need to be synchronized.

I do most of my async coding with twisted where what you calling a barrier is a 
DeferredList.

The way its used is that you add in all the deferreds that you want to complete 
before you continue
into the list. Once all the deferered have competed the DefferedList completes 
and its callback is run.

Barry


> 
> Recently, I had to implement a barier myself for our use case. See code below:
> 
> It is simple to implement, but I too would like to have one for asyncio, in 
> order to be consistent with the concurrency primitives we have for threading.
> 
> Jonathan
> 
> 
> class Barier:
> """
> Make a coroutine block until there are at least X waiters.
> 
> Similar to the threading Barier objects but for asyncio:
> https://docs.python.org/3/library/threading.html#barrier-objects 
> <https://docs.python.org/3/library/threading.html#barrier-objects>
> """
> 
> def __init__(self, parties: int) -> None:
> self.parties = parties
> self._waiting: int
> self._event = asyncio.Event()
> 
> def add_one(self) -> None:
> self._waiting += 1
> if self._waiting == self.parties:
> self._event.set()
> 
> async def wait(self, worker: "Worker") -> None:
> """
>     Wait until all we have at least `parties` waiters.
> """
> self.add_one()
> await self._event.wait()
> 
> 
> 
> 
> Le jeu. 25 févr. 2021 à 16:42, Barry Scott  <mailto:ba...@barrys-emacs.org>> a écrit :
> 
> 
> > On 25 Feb 2021, at 13:14, Yves Duprat  > <mailto:ydup...@gmail.com>> wrote:
> > 
> > Hi,the list,
> > 
> > I'm wondering why Barrier object does not exist in the synchronization 
> > primitives of the asyncio lib while it is present in threading and 
> > multiprocessing libs ?
> > This may not be the right place to ask this question, but I never found an 
> > answer on the web.
> > Thanks for your help.
> 
> 
> I'm assuming that the barrier you are speaking of is the mechanism that is 
> used to
> synchronise threads/processes running in parallel to prevent data races.
> 
> With async code that is never an issue. Each function runs to completion 
> uninterrupted.
> There are no data races. Each time a async function runs it can know that the 
> state of
> the objects it uses will not be changed while it is running.
> 
> Barry
> 
> 
> 
> > 
> > Yves
> > ___
> > Python-ideas mailing list -- python-ideas@python.org 
> > <mailto:python-ideas@python.org>
> > To unsubscribe send an email to python-ideas-le...@python.org 
> > <mailto:python-ideas-le...@python.org>
> > https://mail.python.org/mailman3/lists/python-ideas.python.org/ 
> > <https://mail.python.org/mailman3/lists/python-ideas.python.org/>
> > Message archived at 
> > https://mail.python.org/archives/list/python-ideas@python.org/message/IAFAH7PWMUDUTLXYLNSXES7VMDQ26A3W/
> >  
> > <https://mail.python.org/archives/list/python-ideas@python.org/message/IAFAH7PWMUDUTLXYLNSXES7VMDQ26A3W/>
> > Code of Conduct: http://python.org/psf/codeofconduct/ 
> > <http://python.org/psf/codeofconduct/>
> > 
> ___
> Python-ideas mailing list -- python-ideas@python.org 
> <mailto:python-ideas@python.org>
> To unsubscribe send an email to python-ideas-le...@python.org 
> <mailto:python-ideas-le...@python.org>
> https://mail.python.org/mailman3/lists/python-ideas.python.org/ 
> <https://mail.python.org/mailman3/lists/python-ideas.python.org/>
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/B6WDPXNZH5KYK2BLHJXUFZF2DLFBLCBR/
>  
> <https://mail.python.org/archives/list/python-ideas@python.org/message/B6WDPXNZH5KYK2BLHJXUFZF2DLFBLCBR/>
> Code of Conduct: http://python.org/psf/codeofconduct/ 
> <http://python.org/psf/codeofconduct/>

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/IHSEWHYYJI3BQB4GTT7UCFAS5DOCRNVM/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Barrier Object in asyncio lib

[Barry Scott]

> On 25 Feb 2021, at 13:14, Yves Duprat  wrote:
> 
> Hi,the list,
> 
> I'm wondering why Barrier object does not exist in the synchronization 
> primitives of the asyncio lib while it is present in threading and 
> multiprocessing libs ?
> This may not be the right place to ask this question, but I never found an 
> answer on the web.
> Thanks for your help.


I'm assuming that the barrier you are speaking of is the mechanism that is used 
to
synchronise threads/processes running in parallel to prevent data races.

With async code that is never an issue. Each function runs to completion 
uninterrupted.
There are no data races. Each time a async function runs it can know that the 
state of
the objects it uses will not be changed while it is running.

Barry



> 
> Yves
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/IAFAH7PWMUDUTLXYLNSXES7VMDQ26A3W/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/B6WDPXNZH5KYK2BLHJXUFZF2DLFBLCBR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Deprecate/change the behaviour of ~bool

[Barry Scott]

> On 23 Feb 2021, at 22:10, Steven D'Aprano  wrote:
> 
> There are exactly 2**4 = 16 boolean operators of two variables. Python 
> only supports two: `and` and `or`. Plus a single unary operator `not` 
> (out of four possible unary operators). What makes xnor so special that 
> you want it to be an operator?

Python implements more then 2 of them:

True
False
not
and
or

https://en.wikipedia.org/wiki/Boolean_algebras_canonically_defined#Truth_tables

Barry

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/XXN3NDW2KZDPQWRAF4L3MCAC7FP6DXYU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Arrow functions polyfill

[Barry]

> On 23 Feb 2021, at 21:42, Barry  wrote:
> 
> 
> 
> 
>>> On 23 Feb 2021, at 18:08, Stéfane Fermigier  wrote:
>>> 
>> 
>> Both '%' and .format() support both positional and named arguments.
>> 
>> There are probably a few use cases for .format() (vs. f-strings) but overall 
>> I don't believe there is much reasons left to prefer %.
> 
> I18n you can translate a f string.

Sigh... you *cannot* translate an f string.

Barry

> 
> Barry
> 
>> 
>> Note that the existence, and popularity, of tools like flynt and pyupgrade 
>> (that convert % and .format() directives to f-strings automatically) 
>> supports this affirmation.
>> 
>> I found the 'un-fstring' project on pypi that does the reverse, but it's use 
>> case, as advertised in the README, is clear: "Sometimes, unfortunately, you 
>> need to write code that is compatible with Python 3.5"...
>> 
>>   S.
>> 
>> 
>>> On Tue, Feb 23, 2021 at 6:50 PM Chris Angelico  wrote:
>>> On Wed, Feb 24, 2021 at 4:13 AM Richard Damon  
>>> wrote:
>>> > As far as between % or .format(), I think the documentation is fairly
>>> > clear that the % method is 'old' and if not 'formally' deprecated, is no
>>> > longer considered the 'obvious' way to do it (even if some people will
>>> > still do it that way for the simplest cases).
>>> 
>>> Not really - both forms have their places. You use .format() when you
>>> need to be able to reorder arguments, you use percent formatting when
>>> you want a compact and simple notation. It's like how we have both
>>> string methods and regular expressions - neither one deprecates the
>>> other.
>>> 
>>> ChrisA
>>> ___
>>> Python-ideas mailing list -- python-ideas@python.org
>>> To unsubscribe send an email to python-ideas-le...@python.org
>>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>>> Message archived at 
>>> https://mail.python.org/archives/list/python-ideas@python.org/message/UEZIQ56UYPDWQE3UU5BP72ADRKM2ZSNB/
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>> 
>> 
>> -- 
>> Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - 
>> http://linkedin.com/in/sfermigier
>> Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/
>> Chairman, National Council for Free & Open Source Software (CNLL) - 
>> http://cnll.fr/
>> Founder & Organiser, PyParis & PyData Paris - http://pyparis.org/ & 
>> http://pydata.fr/
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at 
>> https://mail.python.org/archives/list/python-ideas@python.org/message/TTOMZH2L27DU3LCC7PSKNFQL2NA7KEKH/
>> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZIFEA4I6AMKPOQBICFSGYB3JS2JD3ISN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Arrow functions polyfill

[Barry]

> On 23 Feb 2021, at 18:08, Stéfane Fermigier  wrote:
> 
> 
> Both '%' and .format() support both positional and named arguments.
> 
> There are probably a few use cases for .format() (vs. f-strings) but overall 
> I don't believe there is much reasons left to prefer %.

I18n you can translate a f string.

Barry

> 
> Note that the existence, and popularity, of tools like flynt and pyupgrade 
> (that convert % and .format() directives to f-strings automatically) supports 
> this affirmation.
> 
> I found the 'un-fstring' project on pypi that does the reverse, but it's use 
> case, as advertised in the README, is clear: "Sometimes, unfortunately, you 
> need to write code that is compatible with Python 3.5"...
> 
>   S.
> 
> 
>> On Tue, Feb 23, 2021 at 6:50 PM Chris Angelico  wrote:
>> On Wed, Feb 24, 2021 at 4:13 AM Richard Damon  
>> wrote:
>> > As far as between % or .format(), I think the documentation is fairly
>> > clear that the % method is 'old' and if not 'formally' deprecated, is no
>> > longer considered the 'obvious' way to do it (even if some people will
>> > still do it that way for the simplest cases).
>> 
>> Not really - both forms have their places. You use .format() when you
>> need to be able to reorder arguments, you use percent formatting when
>> you want a compact and simple notation. It's like how we have both
>> string methods and regular expressions - neither one deprecates the
>> other.
>> 
>> ChrisA
>> ___
>> Python-ideas mailing list -- python-ideas@python.org
>> To unsubscribe send an email to python-ideas-le...@python.org
>> https://mail.python.org/mailman3/lists/python-ideas.python.org/
>> Message archived at 
>> https://mail.python.org/archives/list/python-ideas@python.org/message/UEZIQ56UYPDWQE3UU5BP72ADRKM2ZSNB/
>> Code of Conduct: http://python.org/psf/codeofconduct/
> 
> 
> -- 
> Stefane Fermigier - http://fermigier.com/ - http://twitter.com/sfermigier - 
> http://linkedin.com/in/sfermigier
> Founder & CEO, Abilian - Enterprise Social Software - http://www.abilian.com/
> Chairman, National Council for Free & Open Source Software (CNLL) - 
> http://cnll.fr/
> Founder & Organiser, PyParis & PyData Paris - http://pyparis.org/ & 
> http://pydata.fr/
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/TTOMZH2L27DU3LCC7PSKNFQL2NA7KEKH/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/G32LOYMNBMFQX5RNL4HYFT5IEGZGWQ3B/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Inadequate error reporting during function call setup stage

[Barry Scott]

> On 22 Feb 2021, at 10:15, Paul Sokolovsky  wrote:
> 
> It looks like:
> 
> Traceback (most recent call last):
>  File "pseudoc_tool.py", line 91, in 
>  File ".../xforms.py", line 25, in print
> TypeError: unexpected keyword argument 'noann'
> 
> - that makes clear that it's "print" function of "xforms.py" module,
> line 25, which got an unexpected keyword argument.

You are proposing to fake a stack frame that I have to know is not a stack 
frame but is in fact the location of the function in the exception?
I'm -1 on that as its confusing.

Having checked that its python code and not a C extension function you could 
use the info
in fn.__code__ to get the filename and line of where the function is defined 
and put that info into the exception.

Example of the info:
| >>> os.path.join.__code__


I use repr(fn.__code__) a lot when debugging complex code.

Barry

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/YJ3RNTHGZWJ2DG2T5HFGXKTKFYN3CGAN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

[Barry Scott]

> On 5 Feb 2021, at 11:49, Inada Naoki  wrote:
> 
> On Fri, Feb 5, 2021 at 8:15 PM Barry Scott  wrote:
>> 
>>> 
>>> The main limitation is that users can not write config file in install
>>> location when Python is installed for system, not for user.
>> 
>> This is the problem that I was thinking about when I proposed using
>> a py.ini like solution where the file is looked for in the users config 
>> folder.
>> I think that is the %LOCALAPPDATA% folder for py.exe.
>> 
>> As Chris points out in his summary of the issue.
>> 
>> How would this work for different version of python being installed and 
>> needing different config?
> 
> Each installation have each config file.

I'm talking about the user's override of the system default.
The system default is the easy part.

> 
>> How would this work for python installed from different vendors?
>> 
> 
> Vendor installer should provide an option for it.
> 
>> Maybe the answer is that there is only one user defined override possible 
>> and all versions use it.
>> 
>> Also am I right to assume that the impact of these changes would only impact 
>> on Windows?
>> 
> 
> I think we don't have any reason to restrict this for Windows.
> But since this idea is proposed only for Windows users, only Windows
> installer will have "Enable UTF-8 mode" option.

I'm not sure that Linux and macOS suffer from this problem. Am I wrong to think 
that?

Barry


> 
> -- 
> Inada Naoki  
> 
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CBQ6CYQLQM5KJ6CIBG7VQOIIN4FMN5V3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

[Barry Scott]

> On 5 Feb 2021, at 11:06, Inada Naoki  wrote:
> 
> On Fri, Feb 5, 2021 at 7:59 PM Barry Scott  wrote:
>> 
>> I'm under the impression that new users will not create a venv.
>> Indeed I run a lot of python scripts outside of venv world.
>> I only use venv as part of my development pipe lines.
>> 
>> I not sure that a venv cfg file would not help.
>> But a python.ini could.
>> 
> 
> python.exe lookup pyvenv.cfg even outside of venv.
> So we can write utf8mode=1 in pyvenv.cfg even outside of venv.

Oh I did not know that. I'm happy that a new file is not need for the
system wide setting.

> 
> The main limitation is that users can not write config file in install
> location when Python is installed for system, not for user.

This is the problem that I was thinking about when I proposed using
a py.ini like solution where the file is looked for in the users config folder.
I think that is the %LOCALAPPDATA% folder for py.exe.

As Chris points out in his summary of the issue.

How would this work for different version of python being installed and needing 
different config?
How would this work for python installed from different vendors?

Maybe the answer is that there is only one user defined override possible and 
all versions use it.

Also am I right to assume that the impact of these changes would only impact on 
Windows?

Barry

> 
> Regards,
> -- 
> Inada Naoki  
> 
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/WXYZFAGGBZGDPBKXBRIY6O4VUFA36APL/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

[Barry Scott]

> On 5 Feb 2021, at 03:11, Inada Naoki  wrote:
> 
> On Fri, Feb 5, 2021 at 6:17 AM Barry Scott  wrote:
>> 
>> Rather than reply point by point I will summarise my input.
>> 
>> I think that utf-8 mode is a great idea.
>> 
>> I think that an .INI file in the style that py.exe uses is better then env 
>> var.
>> 
>> Env var on WIndows could be used but there can be surprises with the way
>> windows merges user and system env vars. Maybe that only with PATH that
>> is very odd.
>> 
>> I'm hoping that the solution implemented allows new users to get a great
>> experience and also that advanced users can get control of the mode.
>> 
>> Personally I'd prefer to have files that I edit to configure python then 
>> registry
>> keys. I can put files into git, not exmple. I have to work hard to manage 
>> registry
>> keys via git.
>> 
> 
> I 100% agree with you. And pyvenv.cfg satisfies all your needs.
> 
> When compared pyvenv.cfg with py.ini-like new config file:
> 
> Cons:
> 
> * Need system privilege to change the setting of system installed Python.
>  * But user can install another Python, or create venv anyway.
> 
> Pros:
> 
> * The file is already supported.
>  * No need to lookup another file at startup.
> * No need to edit any file outside the install location.
>  * Easy to clean uninstall
>  * Portable app friendly
> * One file per environment
>  * Breaking the config file affects only one environment.
> 
> So I still prefer pyvenv.cfg.

I'm under the impression that new users will not create a venv.
Indeed I run a lot of python scripts outside of venv world.
I only use venv as part of my development pipe lines.

I not sure that a venv cfg file would not help.
But a python.ini could.

Barry


> 
> -- 
> Inada Naoki  
> 
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7QALMNRYQDKZPGH2N5URXNCRO4F7CBEW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

[Barry Scott]

> On 3 Feb 2021, at 02:49, Christopher Barker  wrote:
> 


Rather than reply point by point I will summarise my input.

I think that utf-8 mode is a great idea.

I think that an .INI file in the style that py.exe uses is better then env var.

Env var on WIndows could be used but there can be surprises with the way
windows merges user and system env vars. Maybe that only with PATH that
is very odd.

I'm hoping that the solution implemented allows new users to get a great
experience and also that advanced users can get control of the mode.

Personally I'd prefer to have files that I edit to configure python then 
registry
keys. I can put files into git, not exmple. I have to work hard to manage 
registry
keys via git.

Barry
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/N4HV3CM3QAUNZA4XRP2PQTNNR2BV7H2V/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

[Barry Scott]

> On 3 Feb 2021, at 02:49, Christopher Barker  wrote:
> 
> Aside: HTML 5 even has a encoding rule that acknowledges that web pages marked
> utf-8 are really windows USA code page and show how to fall back!
> 
> But that doesn't depend ina. system setting does it? So I don't get your 
> point:

The bug in most (?) .net web apps apparently is that the .net libraries convert 
to the default
system locale and do not assume utf-8. The programmer has to explicity use 
utf-8.

So yes it does depend on a system setting.

I came across this in the HTML 5 specs because of working on web page content 
that did not decode
not because I'm a .NET developer.

I raise this as this seems to be the same problem that python faces with system 
locale conflicting
with the wider world using utf-8.

Barry

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/QZYVQQX2PKO2OY3L2QRACJ5DKQY3KM53/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

[Barry Scott]

> On 2 Feb 2021, at 19:45, Christopher Barker  wrote:
> 
> On Tue, Feb 2, 2021 at 11:12 AM Barry Scott  <mailto:ba...@barrys-emacs.org>> wrote:
>>>> Where would Python look for a "configuration file like `pyvenv.cfg`" ?
>>>> 
>>>> I am not a Windows expert so I am not sure. But I think it should be
>>>> the same directory where `python.exe` is in.
> 
> A small note here -- ideally there would be nothing Windows specific here. 
> Yes, UTF-8 mode is Windows only, but:
> 
> 1) should it be? I'm still unsure on this, but while the vast majority of 
> other platforms use UTF-8 -- maybe it would be more robust to have UTF-8 mode 
> available everywhere -- essentially, ignore the "system encoding", no matter 
> the system.
> 
> 2) perhaps UTF-8 mode isn't the only use-case for this -- it would be good to 
> have a way to have Python startup parameters that are installation / 
> environment specific -- that could help with other issues with "global" 
> configuration: PYTHONPATH, PYTHONHOME, PYTHONSTARTUP, other PYTHON* 
> environment variables.

Forgive me if I'm miss understanding your position.

Windows is successful because of its backward compatibility.
That includes big problems with text encoding in the modern world.

Aside: HTML 5 even has a encoding rule that acknowledges that web pages marked
utf-8 are really windows USA code page and show how to fall back!

Are you calling env vars global config?

But it is a Windows problem only.

I write code accepting that unix (*BSD, Linux), macOS and Windows have unique 
quirks.
To believe that if you ignore that reality is not a path that leads to 
happiness.

>>> 
>>> I'd suggest that you could have a %LOCALAPPDATA%\python.ini
>> But what happen if user installed Python from python.org 
>> <http://python.org/> and Python from conda?
>> User may have two or more Pythons having the same version.
> 
> and different environments, be they conda environments, pipenv, what have you.
>  
> Apple showed the way by using reversed FQD's.
> 
> Python.org <http://python.org/> would use org.python.python.ini
> Conda.io <http://conda.io/> would use io.conda.python.ini
> barry-emacs.org <http://barry-emacs.org/>'s python would use 
> org.barrys-emacs.python.python.ini
> 
> Further we would need to support multiple versions of python from the same 
> org installed side-by-side.
> Structure the .ini so that it has default settings and version specific 
> settings.
> 
> This would be pretty painful to manage -- it's a "bad idea" to have a single 
> configuration file that is being managed and updated by any number of 
> different tools. And those tools are managed by different groups of people.

Why would *tools* be messing with my personal config file?
No *tool* messes with py.exe config that I know of.

> 
> And this would be VERY hard for end users to manage -- as people installed 
> and uninstalled python versions and environments, they would get a very, very 
> messy global ini file.

End users are best served with good, practical defaults. Choosing those 
defaults is very hard
which is the point of this thread.

I do not advocate a global ini file. I do not think I asked for that.

I'm suggesting a mechanism that is identical to the way the py.exe is 
configured.

> 
> Also: "conda python" is not necessarily any different than python.org 
> <http://python.org/> python -- it's generally built exactly the same way -- 
> the only difference is how it's installed.

A distro will often patch in distro specific changes.
If conda needs to be configure independently from python.org 
<http://python.org/> kits this is an obvious requirement.

> 
> So it would be much better for the config file to be located inside the 
> Python installation: essentially a 1:1 relationship between the python 
> executable and the config file. So you know for sure that if you fire up the 
> python you are intending to, you will get the configuration that comes with 
> it.

You mean where it may well not be editable without admin privs?
That is a bad thing surely?

>  
> [3.8-64]
> utf8-mode = false
> 
> The key issue here is that the configuration is not "version number specific" 
> -- it's (or should be) application specific. And Python has had that issue 
> for ages: as a run-time system (for lack of a better word), each application 
> needs a different set of packages in various versions. And that's been 
> addressed with with "environments" of various sorts. So it would be good to 
> leverage that, and have a config file that goes right along with the 
> environ

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

[Barry Scott]

> On 2 Feb 2021, at 00:22, Inada Naoki  wrote:
> 
> On Tue, Feb 2, 2021 at 6:31 AM Barry Scott  <mailto:ba...@barrys-emacs.org>> wrote:
>> 
>>> On 30 Jan 2021, at 12:05, Inada Naoki  wrote:
>>> 
>>> Where would Python look for a "configuration file like `pyvenv.cfg`" ?
>>> 
>>> I am not a Windows expert so I am not sure. But I think it should be
>>> the same directory where `python.exe` is in.
>> 
>> You can put the system default there but each user needs to have a file that 
>> they can control
>> to set the per user config.
>> 
>> py.exe uses %LOCALAPPDATA%\py.ini
>> 
>> I'd suggest that you could have a %LOCALAPPDATA%\python.ini.
>> 
> 
> But what happen if user installed Python from python.org <http://python.org/> 
> and Python from conda?
> User may have two or more Pythons having the same version.

Apple showed the way by using reversed FQD's.

Python.org <http://python.org/> would use org.python.python.ini
Conda.io <http://conda.io/> would use io.conda.python.ini
barry-emacs.org <http://barry-emacs.org/>'s python would use 
org.barrys-emacs.python.python.ini

Further we would need to support multiple versions of python from the same org 
installed side-by-side.

Structure the .ini so that it has default settings and version specific 
settings.

---
[default]
utf8_mode = true

[3.8-64]
utf8-mode = false
---


> 
> In my idea, if user can not change the config of system installed
> Python, user still can create venv and change the setting for the
> venv.

That fine for the venv users but does not help the people that do not need/want 
to use venv.

Barry


> 
> -- 
> Inada Naoki  mailto:songofaca...@gmail.com>>

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/UWFIHR5AOIO56RTJMLBBKUXW2FNJ67KR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Make UTF-8 mode more accessible for Windows users.

[Barry Scott]

> On 30 Jan 2021, at 12:05, Inada Naoki  wrote:
> 
>> Where would Python look for a "configuration file like `pyvenv.cfg`" ?
> 
> I am not a Windows expert so I am not sure. But I think it should be
> the same directory where `python.exe` is in.


You can put the system default there but each user needs to have a file that 
they can control
to set the per user config.

py.exe uses %LOCALAPPDATA%\py.ini

I'd suggest that you could have a %LOCALAPPDATA%\python.ini.

Barry

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4UUIHD7Z7REJ63Y2NXMFKHKGP3GRL6KN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Adding `open_text()` builtin function. (relating to PEP 597)

[Barry Scott]

> On 23 Jan 2021, at 11:00, Steven D'Aprano  wrote:
> 
> On Sat, Jan 23, 2021 at 12:40:55AM -0500, Random832 wrote:
>> On Fri, Jan 22, 2021, at 20:34, Inada Naoki wrote:
>>> * Default encoding is "utf-8".
>> 
>> it might be worthwhile to be a little more sophisticated than this.
>> 
>> Notepad itself uses character set detection [it might not be 
>> reasonable to do this on the whole file as notepad does, but maybe the 
>> first 512 bytes, or the result of read1(512)?] when opening a file of 
>> unknown encoding, and msvcrt's "ccs=UTF-8" option to fopen will at 
>> least detect at the presence of UTF-8 and UTF-16 BOMs [and treat the 
>> file as UTF-16 in the latter case].
> 
> 
> I like Random's idea. If we add a new "open text file" builtin function, 
> we should seriously consider having it attempt to auto-detect the 
> encoding. It need not be as sophisticated as `chardet`.

I think that you are going to create a bug magnet if you attempt to auto
detect the encoding.

First problem I see is that the file may be a pipe and then you will block
until you have enough data to do the auto detect.

Second problem is that the first N bytes are all in ASCII and only later
do you see Windows code page signature (odd lack of utf-8 signature).

> That auto-detection behaviour could be enough to differentiate it from 
> the regular open(), thus solving the "but in ten years time it will be 
> redundant and will need to be deprecated" objection.
> 
> Having said that, I can't say I'm very keen on the name "open_text", but 
> I can't think of any other bikeshed colour I prefer.

Given the the functions purpose is to open unicode text use a name that
reflects that it is the encoding that is set not the mode (binary vs. text).

open_unicode maybe?

If you are teaching open_text then do you also need to have open_binary?

Barry

> 
> 
> -- 
> Steve
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/VAWFPIAA4WIVLIF4LFJ4OATJK6JDJS2N/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/4LHLZ5QIBOCLIZUVYQ2UXAU6MEX6VMJH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: PEP Draft: Build Dependency Specification for Manylinux Wheels

[Barry Scott]

> On 4 Jan 2021, at 14:27, Chris Antonellis  
> wrote:
> 
> `system_dependencies`
> -
> System dependencies to install with `yum` prior to building. 
> Entries are expected to be in `yum` `name-version` format.

If dnf is avaiable I think it is better to use dnf over yum.

So your intention is to install deps so that a package can to source compiler 
and linked?

That will only work for root.

You might want to output the yum/dnf command and ask the user to run that 
command
then retry the install.

Also not that you will need to know for each distro where to find the deps.
I do not think you can rely of package names across distros.

> 
> `environment_variables`
> ---
> Environment variables to set prior to building.
> 
> `python_dependencies`
> -
> Python libraries to install with `pip` prior to building.
> Will be installed for each version of python available in 
> `/opt/python/`.

At least with Centos 8 and fedora its /usr that python and its libs are 
installed into not /opt.
Are you thinking of SCL here?

Barry


___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/56QLHPL6DR2AO6UB24PS77D62LE75KRR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Off-topic: What costs NaN pounds for a null amount?

[Barry Scott]

> On 14 Jan 2021, at 17:14, Jonathan Fine  wrote:
> 
> Hi
> 
> There's interest here in arithmetic operations on NaN . I've just seen a 
> product listed as costing NaN pounds to buy a null amount. That was written 
> as £NaN/null.
> 
> The bargain item is Glade Shake & Vacuum Citrus, and you can see it at
> https://www.tesco.com/groceries/en-GB/products/253732570 
> <https://www.tesco.com/groceries/en-GB/products/253732570>
> 
> Searching on this site for 'NaN' brings up many entries for 
> https://en.wikipedia.org/wiki/Naan <https://en.wikipedia.org/wiki/Naan> bread.

Nice to know that there are well tested web sites out there.

Do you recall seeing sites in 2000 that showed the year date as:

06-01- 19100

Barry


> 
> with best regards
> 
> Jonathan
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/FIP7EUZ3E3KNCCK7XVWAS3GJY6KMX6C2/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/LQ3LB7653DK7VLQYIUWBP6V3R5YZES4H/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: fsync-on-close io object

[Barry Scott]

> On 24 Dec 2020, at 17:15, Michael A. Smith  wrote:
> 
> With all the buffering that modern disks and filesystems do, a
> specific question has come up a few times with respect to whether or
> not data was actually written after flush. I think it would be pretty
> useful for the standard library to have a variant in the io module
> that would explicitly fsync on close.
> 
> You might be tempted to argue that this can be done very easily in
> Python already, so why include it in the standard io module?
> 
> 1. It seems to me that it would be better to do this in C, so for the
> folks who need to make a consistency > performance kind of choice,
> they don't have to sacrifice any additional performance.
> 2. Having it in the io library will call attention to this issue,
> which I think is something a lot of folks don't consider. Assuming
> that `close` or `flush` are sufficient for consistency has always been
> wrong (at its limits), but it was less likely to be a stumbling block
> in the past, when buffering was less aggressive and less layered and
> the peak size and continuous-ness of data streams was a more niche
> concern.
> 3. There are many ways to do this, and I think several of them could
> be subtly incorrect. In other words, maybe it can't be done very
> easily and correctly after all. Providing "obviously right" ways to do
> things is the baileywick of the standard library, isn't it?

I have used rename to make a new file appear atomically after it is closed
and I have used fsync to ensure records are on disk before a file is closed.

I've not needed fsync on close yet.

What is the use case that needs this?

Without a compelling use case I'm -1 on this.

Barry



> 
> Thanks for your consideration and feedback,
> Michael Smith
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/UOUS4BNSI6WFBPWMXHJF5IEAV2PQ47VN/
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CNIIRBIEXL46FEUTVHSDU63ASBMHE72X/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: built in to clear terminal

[Barry Scott]

> On 22 Dec 2020, at 12:39, Eryk Sun  wrote:
> 
> On 12/22/20, Barry Scott  wrote:
>> 
>> import sys
>> 
>> def clear_terminal():
>>if sys.platform == 'win32':
>>import ctypes
>>kernel32 = ctypes.windll.kernel32
>># turn on the console ANSI colour handling
>>kernel32.SetConsoleMode(kernel32.GetStdHandle(-11), 7)
>> 
>>sys.stdout.write('\x1b[2J' '\x1b[H')
> 
> Here are some concerns I have:
> 
> * Does not support Windows 8
> * Does not support legacy console in Windows 10 (on the "options" tab)
> * Does not check for SetConsoleMode failure
> * Does not support a different active screen buffer
> * Assumes StandardOutput is a screen buffer for the current console
> * Assumes the current mode of the screen buffer is 3 or 7. New modes
> have been added, and even more may be added
> * Sets a global console setting that persists after Python exits
> 
snip...

Eryk,

Does this work for you:

import sys, os

def clear():
if sys.playform == 'win32':
os.system('cls')

else:
sys.stdout.write('\x1b[2J' '\x1b[H')


No need to address the list above because CLS does it.
No concern about CLS being a trojan becuse as a builtin to CMD and PowerShell
and it takes priority over cls.bat etc.

For all other platforms I'm assuming the VT100 level of ANSI escape sequences
is supported.

Barry
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/5SCM3VKDQORXYAEWBURP7IULSUYB6RUR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: built in to clear terminal

[Barry Scott]

> On 22 Dec 2020, at 09:49, Barry Scott  wrote:
> 
> The simplest answer is
> 
>   print('`\x1b[2J\x1b[H')
> 
> Are there any terminals that this does not work on that are in active use?
> 
> Is using curses that uses termcap needed these days?
> 
> Of course Windows is the outlier, but the new Windows Terminal
> supports ANSI escapes sequences and utf-8.
> 
> I tested the above with Windows Terminal 1.4 on Windows 10
> and it just works.
> 
> Otherwise os.system('cls') works for windows terminal and the old
> windows console stuff.

It turns out that on Windows 10 it works for old console API if you do this:

-
import sys

def clear_terminal():
if sys.platform == 'win32':
import ctypes
kernel32 = ctypes.windll.kernel32
# turn on the console ANSI colour handling
kernel32.SetConsoleMode(kernel32.GetStdHandle(-11), 7)

sys.stdout.write('\x1b[2J' '\x1b[H')
-

The above should work in all but the old none ANSI terminals.

Barry
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/7KA7OGZKLFG6DQDV7VO7DGKOYRG72Y5D/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: built in to clear terminal

[Barry Scott]

The simplest answer is

print('`\x1b[2J\x1b[H')

Are there any terminals that this does not work on that are in active use?

Is using curses that uses termcap needed these days?

Of course Windows is the outlier, but the new Windows Terminal
supports ANSI escapes sequences and utf-8.

I tested the above with Windows Terminal 1.4 on Windows 10
and it just works.

Otherwise os.system('cls') works for windows terminal and the old
windows console stuff.

Barry
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/NPB5HWQ3YHPXUVBEGBDMBMLJRJ3FWOEC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Dict unpacking assignment

[Barry Scott]

> On 31 Oct 2020, at 11:27, Steven D'Aprano  wrote:
> 
> On Sat, Oct 31, 2020 at 08:24:04AM +, Barry Scott wrote:
> 
>>> On 31 Oct 2020, at 06:35, Steven D'Aprano  wrote:
>>> 
>>> I think we can promise that:
>>> 
>>> * if the mapping on the right has missing keys, no assignments occur;
>>> 
>>> * if the mapping on the right has extra keys, and there is no double 
>>> star target to capture the extras, then no assignments occur;
>> 
>> Why must I always handle all keys in the dictionary?
> 
> You don't. Use a double-star target to capture the excess, then ignore 
> it. This is similar to sequence unpacking:
> 
>spam, eggs, *who_cares = sequence
> 
>{'spam': spam, 'eggs', eggs, **who_cares} = mapping
> 
> will collect any excess items into `who_cares`. In the first case, it 
> will be a list; in the second, it will be a dict.

I do not see why you would force the who_cares dict to be created when the
only thing that my use case will do with it is delete it.

I'd like to think that python had the freedom to optimise this construct and
forcing the creating of who_cares seems like it would limit optimisation 
choices.

I do not think being the same as sequences is required for dict.

> Barry:
> 
>> {'myitem': self.myitem, **other_kwds} = kwds
> 
> Indeed.

Barry
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VJUL5DQ23YE25A7MLXKBWZ7FMN5FUE6Z/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Dict unpacking assignment

[Barry Scott]

> On 31 Oct 2020, at 06:35, Steven D'Aprano  wrote:
> 
> I think we can promise that:
> 
> * if the mapping on the right has missing keys, no assignments occur;
> 
> * if the mapping on the right has extra keys, and there is no double 
>  star target to capture the extras, then no assignments occur;

Why must I always handle all keys in the dictionary?

I can think of cases where I want to pull more than 1 key out of a dictionary 
but do not care about the rest.

I have a dict containing config items. I just need a couple of the keys not all 
of them.
I have a JSON response and the code is only interested in some of the values 
returned.

For the pattern of passing **kwds down the __init__ chain I would clearly want 
to use the form:

{'myitem': self.myitem, **other_kwds} = kwds


> 
> * otherwise, assignments occur from left to right.

Barry

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZEAGXTXNV3CM65MM6TCC2J53LM7EUKK3/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: New feature

[Barry Scott]

> On 16 Oct 2020, at 14:44, Rob Cliffe via Python-ideas 
>  wrote:
> 
> 
> 
> On 16/10/2020 13:55, Chris Angelico wrote:
>> 
>>> I do precisely that in many of my programs for e.g. single-line progress
>>> displays.
>>> But for multi-line output I don't know of any way to move the cursor
>>> back up.
>>> I work in Windows 10.
>> Try \x1b[A to move up a line, should work.

I find that you have to do this to turn on ANSI processing in CMD.EXE on Window 
10
and I assume earlier Windwows as wel:

import ctypes
kernel32 = ctypes.windll.kernel32
# turn on the console ANSI colour handling
kernel32.SetConsoleMode( kernel32.GetStdHandle( -11 ), 7 )

Now you can use escape sequences:

print('\x1b[31m Hello in red \x1b[m')

Given the need to call WIN32 APIs to get this working making this part
of stdlib would help a lot.

Barry


>> 
>> ChrisA
> Got it working!  The answer is
>  \x1b[1A
> Thanks for the pointer!
> Rob Cliffe
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/B3HVTOIQBA6YGCK4TJRLQW4FTBF2IXAQ/
> Code of Conduct: http://python.org/psf/codeofconduct/
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ZJVCAOCUUFCVWVPJQEUROUX5U4TUF2K4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Naming Accepted PEPs as PAPs

[Barry Scott]

> On 21 Sep 2020, at 18:42, Abdur-Rahmaan Janhangeer  
> wrote:
> 
> 
> 
> Kind Regards,
> 
> Abdur-Rahmaan Janhangeer
> about <https://compileralchemy.github.io/> | blog 
> <https://abdur-rahmaanj.github.io/> 
> github <https://github.com/Abdur-RahmaanJ>
> Mauritius
> 
> 
> On Mon, Sep 21, 2020 at 9:23 PM Barry Scott  <mailto:ba...@barrys-emacs.org>> wrote:
> 
> This is like RFC that can be draft, accepted or rejected.
> RFC's have not needed to change there names.
> I'd rather not have PEP's change there names either.
> 
> As you say there is a Status in the PEP that is clear.
> 
> I see that the RFC docs have more info like "obsoleted by"
> and "updated by" info.
> 
> I work with RFC's all the time an appreciate the forward and
> backward references. So if I'm working on code that refers to
> an RFC I can check to see if it is still current for example.
> 
> The 3 last mails have not added much to the discussion.
> Pinning down on RFC is like saying we'll give cat the same
> food as catfish as they both do seem similar and we have not
> found the need to change catfish food since we have been giving
> them to catfish. I seem to think that people do read each and every
> mail in an ongoing thread to know what has already covered so as
> to elaborate and enrich the discussion but i have the impression i am wrong.
> Saying 'Agile is the new gold' twice or twenty times does not effectively 
> make Agile the new gold.

No they are saying that PEP and RFC are the same type of thing with the same
type of users. RFC is older then PEP and could have changed but did not.
The RFC process works. I also think that the PEP process works.

Multiple people independently offered up RFC as similar process.

I do not see what problem PAP will solve that will not create more problems 
then it
*may* solve. And I'm not convinced that there is a problem to solve.

Barry

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HPBW6NXAWJH4QGLC4URM5OEOVIY4MRPQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Naming Accepted PEPs as PAPs

[Barry Scott]

> On 21 Sep 2020, at 08:35, Abdur-Rahmaan Janhangeer  
> wrote:
> 
> Maybe they have not thought of doing it and also, request for comments is
> like the first mail on python-ideas

You might find this interesting on RFC process: 
https://en.wikipedia.org/wiki/Request_for_Comments

Or maybe they have thought of doing this and rejected it.

Barry

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MXB2XLN4G43O7YVX6SWEIAFSIXUL5Y37/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Naming Accepted PEPs as PAPs

[Barry Scott]

> On 21 Sep 2020, at 07:26, Abdur-Rahmaan Janhangeer  
> wrote:
> 
> Greeting lists,
> 
> I am thinking of proposing to name accepted PEPs as PAPs
> namely: Python Accepted Proposals.
> 
> Hence if we say PAP8 we know it's an accepted proposal.
> 
> Backstory:
> 
> I quoted V. Stinner in this 
> <https://dev.to/abdurrahmaanj/the-zen-of-python-as-related-by-masters-1p9i> 
> for PEP 608 <https://www.python.org/dev/peps/pep-0608/>, he told me
> by the way it was not accepted. This got me thinking that to know 
> accepted peps on reading or hearing of it without seeing the status, 
> you need to be a PEP historian. But, if on the other hand you see 
> PEP 0608, you instantly know it has not been accepted and when 
> you hear of PAP 561 you know it is an accepted PEP
> 
> I know that PEPs have different status as enumerated here 
> <https://www.python.org/dev/peps/> but at least
> such a naming would make a clear distinction.
> 
> Curious to hear your thoughts ^^

This is like RFC that can be draft, accepted or rejected.
RFC's have not needed to change there names.
I'd rather not have PEP's change there names either.

As you say there is a Status in the PEP that is clear.

I see that the RFC docs have more info like "obsoleted by"
and "updated by" info.

I work with RFC's all the time an appreciate the forward and
backward references. So if I'm working on code that refers to
an RFC I can check to see if it is still current for example.

Barry




> 
> Kind Regards,
> 
> Abdur-Rahmaan Janhangeer
> about <https://compileralchemy.github.io/> | blog 
> <https://abdur-rahmaanj.github.io/> 
> github <https://github.com/Abdur-RahmaanJ>
> Mauritius
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/MQWM42U3BWLFONQW7JLQHEVDFBMMK3BU/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/GYK6IW7XYJWPYFRGPYKJ7WXMLPVQSVPF/
Code of Conduct: http://python.org/psf/codeofconduct/