[Python-Dev] Re: Merging PRs without CLA signed

2020-02-26 Thread Kyle Stanley 
> I remember seeing it too.  It may have originally been in the tracker
> instructions, but should definitely be in the devguide now.

>From looking through the devguide for every instance of "CLA" and
"trivial", there seems to be just one section that mentions anything
regarding the triviality of the patch in relation to the CLA:

"If the submitter lacks a signed CLA and the patch is non-trivial, direct
them to the electronic Contributor Licensing Agreement
 to ensure the PSF has
the appropriate authorizations in place to relicense and redistribute their
code."

(https://devguide.python.org/committing/#handling-others-code)

Since the licensing section of the pullrequest page is linked to from the
index of the devguide, as being the main reference for first-time
contributors signing the CLA, I think we could benefit from mentioning it
there as well:

"To accept your change we must have your formal approval for distributing
your work under the PSF license
.
Therefore, you need to sign a contributor agreement
 which allows the Python Software
Foundation  to license your code for use with
Python (you retain the copyright)."

(https://devguide.python.org/pullrequest/#cla)

This could involve a very minimal change to the first sentence, which would
help both sections to better convey the same policy [1]:

"To accept any non-trivial change we must have..."
Would that be reasonable? It wouldn't change the existing policy (since
it's mentioned on the committing page), but I think it would make it more
visibly clear that the CLA isn't a hard requirement for trivial PRs. This
would likely just reinforce the current state of affairs, where it's
ultimately at the discretion of the core dev(s) reviewing the patch [2];
while hopefully clearing up misconceptions.

I think it would also be useful to briefly describe guidelines on the
committing page that help to define what may constitute as trivial vs
non-trivial, along the lines of how Terry described the distinction. I
personally found the comparison of minor fact correction vs creative
expression to be particularly helpful. But, a section like that would
likely require some oversight from the PSF legal staff.

---

[1] - From my interpretation at least, "To accept your change we must have
..." and "If the submitter lacks a signed CLA and the patch is non-trivial
..." seem to be mildly in conflict of one another regarding the CLA policy.

[2] - Discretion to determine triviality based on best judgement, and
whether or not they personally consider merging any PRs without the CLA
signed.

On Mon, Feb 24, 2020 at 1:54 PM Terry Reedy  wrote:

> On 2/23/2020 11:44 PM, Guido van Rossum wrote:
> > On Sun, Feb 23, 2020 at 8:11 PM Kyle Stanley  > > wrote:
> >
> > In a recently opened typo fixing PR [1], an issue came up regarding
> > the lack of a signed CLA, where the author specifically mentioned
> > they did not want to sign it for privacy concerns.
> >
> >
> > In that case I'm not sure the author ought to get credit for the PR.
>
> If the account has a real name, then there cannot be a privacy concern.
> If if does not, then credit can only be claimed privately.
>
> > They can file a bug pointing out the typo and someone else can submit a
> > fix.
>
> One of that justifications given for moving to github was that is would
> allow trivial changes to be submitted without an issue.  Allowing merges
> for trivial changes without a CLA was intentional, after discussion.
>
> To summarize my response a few minutes ago to Antoine and Chris
> Angelico, I consider trivial to mean non-copyrightable because short and
> fact based.
>
> > (This is what Glyph had to do for all his contributions while he
> > was employed at Apple.)
>
> And what anyone in a similar situation should still do for anything
> non-trivial.
> > Yeah, typically we don't insist on a CLA for trivial fixes -- it's at
> > the discretion of the core dev reviewing/merging the PR. I actually
> > thought this was a policy that was written down somewhere, but I don't
> > know where (maybe somewhere in the devguide?).
>
> I remember seeing it too.  It may have originally been in the tracker
> instructions, but should definitely be in the devguide now.
>
>
>
>
> --
> Terry Jan Reedy
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/UARCSQSBLTU2YHOM4QDTGNYXXDWVM26W/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- python-dev@

[Python-Dev] Re: Accepting PEP 584: Add Union Operators To dict

2020-02-26 Thread Claudio Jolowicz 
First off, apologies for entering the discussion after it has formally
ended.
I'm not sure this has any relevance now that the PEP is accepted and the
reference implementation merged. If not, sorry and feel free to ignore.

"Leftmost Value (First-Seen) Wins" was rejected because

> it is not clear that this behavior has many use-cases

This is probably based on a thorough analysis of usage patterns, but the
result
still surprises me. I could not find the related discussion in the mail
archives. Could you elaborate, or point to more detailed information?

In my experience, the expression `value |= other` is a common idiom across
programming languages to provide a default for `value` if it is "unset".
For a
container-type, I would expect this operation, informally spoken, to be
applied
element-wise. In other words, the resulting dict would have the union of
keys,
with values supplied from `other` only if keys are missing in `value`.

This would allow users to provide defaults in a dictionary of settings, like
this:

settings |= defaults

As this usage generalizes a (perceived) common idiom, I would expect people
to
get bitten by this.

Claudio

On Mon, 17 Feb 2020 at 22:05, Guido van Rossum  wrote:

> Now that the last bits of discussion about PEP 584 have been settled (we
> will *not* be calling the `copy` or `update` methods) I am accepting this
> PEP. Congratulations Steven and Brandt!
>
> Thanks to everyone else who gave their opinion.
>
> Formally, I will just send my recommendation to accept the PEP to the
> Steering Council -- they will then let us know whether they agree, and once
> that's done we can update the PEP with the "Accepted" status and land the
> implementation (https://github.com/python/cpython/pull/12088). (Hm, the
> PEP should probably also link to that PR rather than to Brandt's private
> branch.)
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/6KT2KIOTYXMDCD2CCAOLOI7LUGTN6MBS/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/VRR6XTPQXGGCDNMBUMR7RVRAOOOILJEW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Elad Lahav 
More information:
The hang happens when building extensions, using the setup.py script. The 
script determines that the build is parallel (build_ext.py/build_extensions) 
and creates a thread pool. Each thread then executes a compilation job by 
fork()ing a compiler process.

I don't see how it works on any system that keeps the semaphore state in user 
mode.

--Elad
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/DGEMKEX37HEIN7MBNSBJH5P2VQVKUEI7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Accepting PEP 584: Add Union Operators To dict

2020-02-26 Thread Guido van Rossum 
On Wed, Feb 26, 2020 at 7:43 AM Claudio Jolowicz 
wrote:

> In my experience, the expression `value |= other` is a common idiom across
> programming languages to provide a default for `value` if it is "unset".
> For a
> container-type, I would expect this operation, informally spoken, to be
> applied
> element-wise. In other words, the resulting dict would have the union of
> keys,
> with values supplied from `other` only if keys are missing in `value`.
>
> This would allow users to provide defaults in a dictionary of settings,
> like
> this:
>
> settings |= defaults
>
> As this usage generalizes a (perceived) common idiom, I would expect
> people to
> get bitten by this.
>

Interesting. Can you point to specific examples of this? In what other
languages have you seen this? (Not that it would make us change PEP 584,
but if this appears common we could probably warn about it prominently in
docs and tutorials.)

The pattern I'm familiar with for applying defaults always starts with the
defaults and then adds the local overrides, so the PEP 584 behavior fits
perfectly. (Probably because in Python before PEP 584 the most common
operation is dict.update(), which has the same semantics.)

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/JHXN342STNGSPX3NRAEXRS6JOXH5BCNU/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Guido van Rossum 
What semaphore is the subprocess blocking on? I.e. where in the Python code
(presumably of ThreadPoolExecutor) is that semaphore defined?

Given your hypothesis of the cause of the problem, can you perhaps write a
much simper program (which is simpler to debug and simpler to reason about)
that exhibits the same hang?

Does the same problem happen in earlier Python versions? On other operating
systems? (I've never had a problem with `make -j` on macOS, for example; I
suspect that there's something unique to threads and semaphores on QNX that
contributes to the hang.)


On Wed, Feb 26, 2020 at 7:53 AM Elad Lahav  wrote:

> More information:
> The hang happens when building extensions, using the setup.py script. The
> script determines that the build is parallel (
> build_ext.py/build_extensions) and creates a thread pool. Each thread
> then executes a compilation job by fork()ing a compiler process.
>
> I don't see how it works on any system that keeps the semaphore state in
> user mode.
>
> --Elad
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/DGEMKEX37HEIN7MBNSBJH5P2VQVKUEI7/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/5Q2WEDVHCWJWKPRJZ32NFY23PFAMKW6Y/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Accepting PEP 584: Add Union Operators To dict

2020-02-26 Thread David Mertz 
I think the "if unset" behavior is well handled by collections.ChainMap.
But I do think that fact should be prominent in any documentation of the
new dict Union operator.

On Wed, Feb 26, 2020, 11:06 AM Guido van Rossum  wrote:

> On Wed, Feb 26, 2020 at 7:43 AM Claudio Jolowicz 
> wrote:
>
>> In my experience, the expression `value |= other` is a common idiom across
>> programming languages to provide a default for `value` if it is "unset".
>> For a
>> container-type, I would expect this operation, informally spoken, to be
>> applied
>> element-wise. In other words, the resulting dict would have the union of
>> keys,
>> with values supplied from `other` only if keys are missing in `value`.
>>
>> This would allow users to provide defaults in a dictionary of settings,
>> like
>> this:
>>
>> settings |= defaults
>>
>> As this usage generalizes a (perceived) common idiom, I would expect
>> people to
>> get bitten by this.
>>
>
> Interesting. Can you point to specific examples of this? In what other
> languages have you seen this? (Not that it would make us change PEP 584,
> but if this appears common we could probably warn about it prominently in
> docs and tutorials.)
>
> The pattern I'm familiar with for applying defaults always starts with the
> defaults and then adds the local overrides, so the PEP 584 behavior fits
> perfectly. (Probably because in Python before PEP 584 the most common
> operation is dict.update(), which has the same semantics.)
>
> --
> --Guido van Rossum (python.org/~guido)
> *Pronouns: he/him **(why is my pronoun here?)*
> 
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at
> https://mail.python.org/archives/list/[email protected]/message/JHXN342STNGSPX3NRAEXRS6JOXH5BCNU/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/KYVGC2RYPSOXGOHQO5WDQUUE4WJ4TBE7/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Antoine Pitrou 


Hi,

On Tue, 25 Feb 2020 11:48:44 -
[email protected] wrote:
> Hello,
> 
> I have successfully built Python 3.8.1 on QNX, but ran into a problem when 
> using 'make -j4'. The build process invariably hangs with multiple 
> single-threaded child processes stuck indefinitely waiting on semaphores. 
> These semaphores will clearly never be posted to, as the processes are all 
> single threaded and the semaphores are not shared with any other process.
> A backtrace shows that the the offending calls come from run_at_forkers(), 
> which is not surprising. I consider a multi-threaded fork() to be an 
> ill-defined operation and my arch-nemesis...

Can you post the detailed backtrace(s) somewhere?
Otherwise nobody will be able to dive deeper.

> 3. Any reason not to use posix_spawn() instead of fork()/exec()? While some 
> systems implement posix_spawn() with fork()/exec() others (at least QNX) 
> implements it without first creating a duplicate of the parent, making it 
> both more efficient and safer in a multi-threaded parent.

Well, subprocess can do so in some cases:
https://bugs.python.org/issue35537

I'm not sure what can prevent it here, but you're welcome to
investigate.

> 4. thread_pthread.h seems to go to great lengths to implement locks without 
> using native mutexes.

Python locks are not like mutexes.  They can be locked from one thread
and unlocked from another, so they're more like binary semaphores.

Regards

Antoine.

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/UZUUWLWZSODZHBKQL257EW26JLOQCQFR/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Accepting PEP 584: Add Union Operators To dict

2020-02-26 Thread Guido van Rossum 
Here's our current proposal for docs. Is there anything you'd like to add?
https://github.com/python/cpython/pull/18659/files

On Wed, Feb 26, 2020 at 8:12 AM David Mertz  wrote:

> I think the "if unset" behavior is well handled by collections.ChainMap.
> But I do think that fact should be prominent in any documentation of the
> new dict Union operator.
>
> On Wed, Feb 26, 2020, 11:06 AM Guido van Rossum  wrote:
>
>> On Wed, Feb 26, 2020 at 7:43 AM Claudio Jolowicz 
>> wrote:
>>
>>> In my experience, the expression `value |= other` is a common idiom
>>> across
>>> programming languages to provide a default for `value` if it is "unset".
>>> For a
>>> container-type, I would expect this operation, informally spoken, to be
>>> applied
>>> element-wise. In other words, the resulting dict would have the union of
>>> keys,
>>> with values supplied from `other` only if keys are missing in `value`.
>>>
>>> This would allow users to provide defaults in a dictionary of settings,
>>> like
>>> this:
>>>
>>> settings |= defaults
>>>
>>> As this usage generalizes a (perceived) common idiom, I would expect
>>> people to
>>> get bitten by this.
>>>
>>
>> Interesting. Can you point to specific examples of this? In what other
>> languages have you seen this? (Not that it would make us change PEP 584,
>> but if this appears common we could probably warn about it prominently in
>> docs and tutorials.)
>>
>> The pattern I'm familiar with for applying defaults always starts with
>> the defaults and then adds the local overrides, so the PEP 584 behavior
>> fits perfectly. (Probably because in Python before PEP 584 the most common
>> operation is dict.update(), which has the same semantics.)
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>> *Pronouns: he/him **(why is my pronoun here?)*
>> 
>> ___
>> Python-Dev mailing list -- [email protected]
>> To unsubscribe send an email to [email protected]
>> https://mail.python.org/mailman3/lists/python-dev.python.org/
>> Message archived at
>> https://mail.python.org/archives/list/[email protected]/message/JHXN342STNGSPX3NRAEXRS6JOXH5BCNU/
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>

-- 
--Guido van Rossum (python.org/~guido)
*Pronouns: he/him **(why is my pronoun here?)*

___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/S2OW7WOAXXT2YI5P6EOP6CMBRLQME3K2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Elad Lahav 
Sorry, should have posted the backtrace from the beginning. It goes deeper than 
this, but the important part is in the child after fork():

#0  SyncSemWait () at 
/builds/workspace/710-SDP/build_x86_64/lib/c/kercalls/x86_64/SyncSemWait.S:37
#1  0x004bfa174ac6 in PyThread_acquire_lock_timed 
(lock=lock@entry=0x24f0cd9430, microseconds=microseconds@entry=-100, 
intr_flag=intr_flag@entry=1) at Python/thread_pthread.h:459
#2  0x004bfa1c77bd in acquire_timed (lock=0x24f0cd9430, 
timeout=-10) at ./Modules/_threadmodule.c:63
#3  0x004bfa1c78e7 in rlock_acquire (self=0x24f13027e0, args=, kwds=) at ./Modules/_threadmodule.c:308
#4  0x004bfa204631 in method_vectorcall_VARARGS_KEYWORDS 
(func=0x24f0d61ef0, args=0x24f131c660, nargsf=, 
kwnames=)
at Objects/descrobject.c:332
#5  0x004bfa06eff6 in _PyObject_Vectorcall (kwnames=, 
nargsf=, args=, callable=)
at ./Include/cpython/abstract.h:123
#6  call_function (kwnames=0x0, oparg=1, pp_stack=, 
tstate=0x24f0d8ee00) at Python/ceval.c:4987
#7  _PyEval_EvalFrameDefault (f=, throwflag=) at 
Python/ceval.c:3486
#8  0x004bfa06c95b in function_code_fastcall (co=, 
args=, nargs=0, globals=) at Objects/call.c:283
#9  0x004bfa06ed29 in _PyObject_Vectorcall (kwnames=, 
nargsf=, args=, callable=)
at ./Include/cpython/abstract.h:127
#10 call_function (kwnames=0x0, oparg=, pp_stack=, tstate=0x24f0d8ee00) at Python/ceval.c:4987
#11 _PyEval_EvalFrameDefault (f=, throwflag=) at 
Python/ceval.c:3500
#12 0x004bfa06c95b in function_code_fastcall (co=, 
args=, nargs=1, globals=) at Objects/call.c:283
#13 0x004bfa06ed29 in _PyObject_Vectorcall (kwnames=, 
nargsf=, args=, callable=)
at ./Include/cpython/abstract.h:127
#14 call_function (kwnames=0x0, oparg=, pp_stack=, tstate=0x24f0d8ee00) at Python/ceval.c:4987
#15 _PyEval_EvalFrameDefault (f=, throwflag=) at 
Python/ceval.c:3500
#16 0x004bfa06c95b in function_code_fastcall (co=, 
args=, nargs=1, globals=) at Objects/call.c:283
#17 0x004bfa06eff6 in _PyObject_Vectorcall (kwnames=, 
nargsf=, args=, callable=)
at ./Include/cpython/abstract.h:123
#18 call_function (kwnames=0x0, oparg=1, pp_stack=, 
tstate=0x24f0d8ee00) at Python/ceval.c:4987
#19 _PyEval_EvalFrameDefault (f=, throwflag=) at 
Python/ceval.c:3486
#20 0x004bfa06c95b in function_code_fastcall (co=, 
args=, nargs=0, globals=) at Objects/call.c:283
#21 0x004bfa08626a in _PyObject_FastCallDict (callable=0x24f152fd30, 
args=, nargsf=, kwargs=)
at Objects/call.c:96
#22 0x004bfa189584 in run_at_forkers (lst=, 
reverse=) at ./Modules/posixmodule.c:435
#23 0x004bfa194bfb in run_at_forkers (reverse=0, lst=) at 
./Modules/posixmodule.c:420
#24 PyOS_AfterFork_Child () at ./Modules/posixmodule.c:474
#25 0x004bfa194d08 in os_fork_impl (module=) at 
./Modules/posixmodule.c:6082
#26 os_fork (module=, _unused_ignored=) at 
./Modules/clinic/posixmodule.c.h:2709

--Elad
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/GCCEBGMUDK2UAMCDSW4VRWCUAHGAPLCN/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Elad Lahav 
It's actually not clear to me what lock it is from the core file I took, as 
rlock_acquire() is called through a function pointer from 
method_vectorcall_VARARGS_KEYWORDS() (I posted the backtrace separately).

My suspicion is that it doesn't fail on macOS because it may keep all of the 
semaphore's state in the kernel, which means that it is not necessarily 
inherited on fork(). QNX keeps the count in user mode, in a similar fashion to 
the way some state is kept in user mode for fast mutexes.

I'll see if I can come up with a simple scenario. In the mean time I am trying 
to switch to posix_spawn() to see if it fixes the problem (and will also be 
much faster on QNX, as you don't need to create a duplicate address space just 
to tear it down with exec()).

--Elad
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/XHHRNVTWADJTMHODQUVRH5QF3TLTIT4N/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Elad Lahav 
A change to posix_spawnp() fixes the problem for me:

diff --git a/Lib/distutils/spawn.py b/Lib/distutils/spawn.py
index ceb94945dc..cb69de4242 100644
--- a/Lib/distutils/spawn.py
+++ b/Lib/distutils/spawn.py
@@ -90,7 +90,7 @@ def _spawn_posix(cmd, search_path=1, verbose=0, 
dry_run=0):
 return
 executable = cmd[0]
 exec_fn = search_path and os.execvp or os.execv
-env = None
+env = os.environ
 if sys.platform == 'darwin':
 global _cfg_target, _cfg_target_split
 if _cfg_target is None:
@@ -112,7 +112,7 @@ def _spawn_posix(cmd, search_path=1, verbose=0, 
dry_run=0):
 env = dict(os.environ,
MACOSX_DEPLOYMENT_TARGET=cur_target)
 exec_fn = search_path and os.execvpe or os.execve
-pid = os.fork()
+pid = os.posix_spawnp(executable, cmd, env)
 if pid == 0: # in the child
 try:
 if env is None:

--Elad
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/3SJ53WLPMMHT7AJUBGBS3ANEMAEAVWAW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Elad Lahav 
A simple example that reproduces the hang (please keep in mind that I have very 
little experience writing Python code...):

import os
from concurrent.futures import ThreadPoolExecutor

def new_process(arg):
pid = os.fork()
if pid == 0:
exec_fn("/bin/true", "/bin/true")
else:
pid, status = os.waitpid(pid, 0)

with ThreadPoolExecutor(max_workers=4) as executor:
futures = [executor.submit(new_process, None)
   for i in range(0, 4)]
for fut in futures:
fut.result()

--Elad
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/ZOJNVZJIRJJF6RTEWFJ4HG2KZXYY6CLV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Antoine Pitrou 


Admittedly, the whole distutils spawn code should be rewritten to use
subprocess.

Regards

Antoine.


On Wed, 26 Feb 2020 17:52:53 -
"Elad Lahav"  wrote:
> A change to posix_spawnp() fixes the problem for me:
> 
> diff --git a/Lib/distutils/spawn.py b/Lib/distutils/spawn.py
> index ceb94945dc..cb69de4242 100644
> --- a/Lib/distutils/spawn.py
> +++ b/Lib/distutils/spawn.py
> @@ -90,7 +90,7 @@ def _spawn_posix(cmd, search_path=1, verbose=0, 
> dry_run=0):
>  return
>  executable = cmd[0]
>  exec_fn = search_path and os.execvp or os.execv
> -env = None
> +env = os.environ
>  if sys.platform == 'darwin':
>  global _cfg_target, _cfg_target_split
>  if _cfg_target is None:
> @@ -112,7 +112,7 @@ def _spawn_posix(cmd, search_path=1, verbose=0, 
> dry_run=0):
>  env = dict(os.environ,
> MACOSX_DEPLOYMENT_TARGET=cur_target)
>  exec_fn = search_path and os.execvpe or os.execve
> -pid = os.fork()
> +pid = os.posix_spawnp(executable, cmd, env)
>  if pid == 0: # in the child
>  try:
>  if env is None:
> 
> --Elad


___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/KU6TJ3LKKAX6C45NQZRVGWZCEC4FQ5J4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Elad Lahav 
I believe that the problem is in logging/__init__.py, which registers an 
atfork() handler for re-initializing its lock. However, as part of this process 
it attempts to acquire the module lock, which has not been reinitialized and so 
still reflects the parent's state of the lock.

--Elad
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/XIWDXXWKFDDSEZ7B3IQZZYEI2GO4G774/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Antoine Pitrou 


Hi Elad,

Can you open an issue on https://bugs.python.org/ and post your
findings there?  I don't think it makes sense to continue discussing
this on python-dev.

(note that opening a bug doesn't mean it will be fixed quickly, but at
least it's recorded somewhere instead of being lost in the mailing-list
archives)

Best regards

Antoine.


On Wed, 26 Feb 2020 18:47:39 -
"Elad Lahav"  wrote:
> I believe that the problem is in logging/__init__.py, which registers an 
> atfork() handler for re-initializing its lock. However, as part of this 
> process it attempts to acquire the module lock, which has not been 
> reinitialized and so still reflects the parent's state of the lock.
> 
> --Elad


___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/6RH32RIIACZZ3PLLASELUNNQCWYU23W4/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Hang with parallel make

2020-02-26 Thread Elad Lahav 
Done.

Thanks,
--Elad
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/5YBM6LDLV7HSYG44ZA5CPFJBVXDQRRYD/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] 3.7.7 schedule accelerated, cutoff now 2020-03-02

2020-02-26 Thread Ned Deily 
https://discuss.python.org/t/3-7-7-schedule-accelerated-cutoff-now-2020-03-02/3511

--
  Ned Deily
  [email protected] -- []
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/GIVKRSVTCYOUBLWX6LPX3X2KZH4WAV4Y/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Accepting PEP 584: Add Union Operators To dict

2020-02-26 Thread Nick Coghlan 
On Thu., 27 Feb. 2020, 2:03 am Guido van Rossum,  wrote:

> On Wed, Feb 26, 2020 at 7:43 AM Claudio Jolowicz 
> wrote:
>
>> In my experience, the expression `value |= other` is a common idiom across
>> programming languages to provide a default for `value` if it is "unset".
>>
>
> Interesting. Can you point to specific examples of this? In what other
> languages have you seen this? (Not that it would make us change PEP 584,
> but if this appears common we could probably warn about it prominently in
> docs and tutorials.)
>

I was thinking that bash scripting might be an example, but I double
checked, and that's spelled 'VAR="${$VAR:-default value}" '

make has 'VAR ?= "default value"'

C# uses "??=" for null coalescence on assignment.

So I've also never come across "|=" being used for this purpose.

Cheers,
Nick.

>
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/HMKYUZP5T6HTURG46GU3L72KANB65MLQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-Dev] Re: Accepting PEP 584: Add Union Operators To dict

2020-02-26 Thread Antoine Rozo 
Maybe it's more the `a ||= b` that acts like a `a = a or b` in Python?
But then I don't think there is a confusion on Python side because
languages with a || operator usually already has a simple | with a
different meaning.

Le jeu. 27 févr. 2020 à 00:28, Nick Coghlan  a écrit :
>
>
>
> On Thu., 27 Feb. 2020, 2:03 am Guido van Rossum,  wrote:
>>
>> On Wed, Feb 26, 2020 at 7:43 AM Claudio Jolowicz  wrote:
>>>
>>> In my experience, the expression `value |= other` is a common idiom across
>>> programming languages to provide a default for `value` if it is "unset".
>>
>>
>> Interesting. Can you point to specific examples of this? In what other 
>> languages have you seen this? (Not that it would make us change PEP 584, but 
>> if this appears common we could probably warn about it prominently in docs 
>> and tutorials.)
>
>
> I was thinking that bash scripting might be an example, but I double checked, 
> and that's spelled 'VAR="${$VAR:-default value}" '
>
> make has 'VAR ?= "default value"'
>
> C# uses "??=" for null coalescence on assignment.
>
> So I've also never come across "|=" being used for this purpose.
>
> Cheers,
> Nick.
>
> ___
> Python-Dev mailing list -- [email protected]
> To unsubscribe send an email to [email protected]
> https://mail.python.org/mailman3/lists/python-dev.python.org/
> Message archived at 
> https://mail.python.org/archives/list/[email protected]/message/HMKYUZP5T6HTURG46GU3L72KANB65MLQ/
> Code of Conduct: http://python.org/psf/codeofconduct/



-- 
Antoine Rozo
___
Python-Dev mailing list -- [email protected]
To unsubscribe send an email to [email protected]
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at 
https://mail.python.org/archives/list/[email protected]/message/73D2MVBXG6MAZ7KI76PKTEZPMMLQH6PQ/
Code of Conduct: http://python.org/psf/codeofconduct/