Re: [Python-ideas] PEP: Dict addition and subtraction

[Christopher Barker]

On Sat, Mar 16, 2019 at 12:39 AM Antoine Pitrou  wrote:

> On Sat, 16 Mar 2019 01:41:59 +1100
> Steven D'Aprano  wrote:
>


> > Matrix multiplication is a perfect example: adding the @ operator could
> > have been done in Python 0.1 if anyone had thought of it, but it took 15
> > years of numerical folk "whinging" about the lack until it happened:
>
> Not so perfect, as the growing use of Python for scientific computing
> has made it much more useful to promote a dedicated matrix
> multiplication operator than, say, 15 or 20 years ago.
>

Theres more to it than that, really, but not really relevant here...


> This is precisely why I worded my question this way: what has changed
> in the last 20 years that make a "+" dict operator more compelling
> today than it was?  Do we merge dicts much more frequently than we
> did?


The analogy doesn't hold because @ was a new operator -- a MUCH bigger
change than dimply defining the use of + (or | ) for dicts.

I wouldn't mind the new operator if its meaning was clear-cut.  But
> here we have potential for confusion, both for writers and readers of
> code.
>

but it's NOT a new operator, it is making use of an existing one, and sure
you could guess at a couple meanings, but the merge one is probably one of
the most obvious to guess, and one quick test and you know -- I really
can't see it being a ongoing source of confusion.

-CHB

-- 
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Add subprocess.Popen suspend() and resume()

[eryk sun]

On 3/18/19, Giampaolo Rodola'  wrote:
>
> I've been having these 2 implemented in psutil for a long time. On
> POSIX these are convenience functions using os.kill() + SIGSTOP /
> SIGCONT (the same as CTRL+Z  / "fg"). On Windows they use
> undocumented NtSuspendProcess and NtResumeProcess Windows
> APIs available since XP.

Currently, Windows Python only calls documented C runtime-library and
Windows API functions. It doesn't directly call NT runtime-library and
system functions. Maybe it could in the case of documented functions,
but calling undocumented functions in the standard library should be
avoided. Unfortunately, without NtSuspendProcess and NtResumeProcess,
I don't see a way to reliably implement this feature for Windows. I'm
CC'ing Steve Dower. He might say it's okay in this case, or know of
another approach.

DebugActiveProcess, the other simple approach mentioned in the linked
SO answer [1], is unreliable and has the wrong semantics.  A process
only has a single debug port, so DebugActiveProcess will fail the PID
as an invalid parameter if another debugger is already attached to the
process. (The underlying NT call, DbgUiDebugActiveProcess, fails with
STATUS_PORT_ALREADY_SET.) Additionally, the semantics that I expect
here, at least for Windows, is that each call to suspend() will
require a corresponding call to resume(), since it's incrementing the
suspend count on the threads; however, a debugger can't reattach to
the same process. Also, if the Python process exits while it's
attached as a debugger, the system will terminate the debugee as well,
unless we call DebugSetProcessKillOnExit(0), but that interferes with
the Python process acting as a debugger normally, as does this entire
wonky idea. Also, the debugging system creates a thread in the debugee
that calls NT DbgUiRemoteBreakin, which executes a breakpoint. This
thread is waiting, but it's not suspended, so the process will never
actually appear as suspended in Task Manager or Process Explorer.

That leaves enumerating threads in a snapshot and calling OpenThread
and SuspendThread on each thread that's associated with the process.
In comparison, let's take an abridged look at the guts of
NtSuspendProcess.

nt!NtSuspendProcess:
...
mov r8,qword ptr [nt!PsProcessType]
...
callnt!ObpReferenceObjectByHandleWithTag
...
callnt!PsSuspendProcess
...
mov ebx,eax
callnt!ObfDereferenceObjectWithTag
mov eax,ebx
...
ret

nt!PsSuspendProcess:
...
callnt!ExAcquireRundownProtection
cmp al,1
jne nt!PsSuspendProcess+0x74
...
callnt!PsGetNextProcessThread
xor ebx,ebx
jmp nt!PsSuspendProcess+0x62

nt!PsSuspendProcess+0x4d:
...
callnt!PsSuspendThread
...
callnt!PsGetNextProcessThread

nt!PsSuspendProcess+0x62:
...
testrax,rax
jne nt!PsSuspendProcess+0x4d
...
callnt!ExReleaseRundownProtection
jmp nt!PsSuspendProcess+0x79

nt!PsSuspendProcess+0x74:
mov ebx,0C10Ah (STATUS_PROCESS_IS_TERMINATING)

nt!PsSuspendProcess+0x79:
...
mov eax,ebx
...
ret

This code repeatedly calls PsGetNextProcessThread to walk the
non-terminated threads of the process in creation order (based on a
linked list in the process object) and suspends each thread via
PsSuspendThread. In contrast, a Tool-Help thread snapshot is
unreliable since it won't include threads created after the snapshot
is created. The alternative is to use a different undocumented system
call, NtGetNextThread [2], which is implemented via
PsGetNextProcessThread. But that's slightly worse than calling
NtSuspendProcess.

[1]: https://stackoverflow.com/a/11010508
[2]: 
https://github.com/processhacker/processhacker/blob/v2.39/phnt/include/ntpsapi.h#L848
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/

Re: [Python-ideas] Problems (and solutions?) in writing decorators

[Sylvain MARIE via Python-ideas]

All of your answers are true,

> (Chris)
> "my_decorator(x=foo)" is not going to look like "@my_decorator \n def foo".

That’s one of the many ways that `decopatch` uses to perform the 
disambiguation, indeed. But that’s already the user helping the lib, not the 
other way round

> (Christopher)
> @something applied on def fun is exactly the same as something(fun)

True. However applying decorator manually is already for advanced users, so 
users of a decopatche-d decorator will not mind calling something()(fun). In 
fact it is consistent with when they use it with arguments : 
something(arg)(fun).

Another criterion is : how easy would it be to implement an 
inspect.is_decorator_call(frame) method returning True if and only if frame is 
the @ statement ? If that’s fairly easy, well, I’m pretty sure that this is 
good stuff. From a naïve user, not accustomed with the long history of this 
language, is very strange that an operator such as @ (the decorator one, not 
the other one) is completely not detectable by code, while there are so many 
hooks available in python for all other operators (+, -, etc.).

Eventually that’s obviously your call, I’m just there to give feedback from 
what I see of the python libs development community.

--
Sylvain

De : Python-ideas  De la 
part de Christopher Barker
Envoyé : mercredi 20 mars 2019 06:50
À : Greg Ewing 
Cc : python-ideas 
Objet : Re: [Python-ideas] Problems (and solutions?) in writing decorators


[External email: Use caution with links and attachments]




Also:

@something
def fun():
...

Is exactly the same as:

def fun()
...

fun = something(fun)

So you can’t make a distinction based whether a given usage  is as a decoration.

-CHB

On Tue, Mar 19, 2019 at 12:26 PM Greg Ewing 
mailto:greg.ew...@canterbury.ac.nz>> wrote:
Sylvain MARIE via Python-ideas wrote:
> `my_decorator(foo)` when foo is a callable will always look like
> `@my_decorator` applied to function foo, because that's how the language is
> designed.

I don't think it's worth doing anything to change that. Everywhere
else in the language, there's a very clear distinction between
'foo' and 'foo()', and you confuse them at your peril. I don't see
why decorators should be any different.

--
Greg
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: 
http://python.org/psf/codeofconduct/
--
Christopher Barker, PhD

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython

__
This email has been scanned by the Symantec Email Security.cloud service.
__
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/