[Python-ideas] Re: Allow user extensions to operators [related to: Moving PEP 584 forward (dict + and += operators)]
On Wed, Dec 4, 2019 at 3:12 PM Guido van Rossum wrote: > We could treat it as a kind of future statement. If there’s a top level > statement that defines the magic identitier we generate the special > bytecode. > True, that would help solve the performance issue. But I'm still -1 on the idea regardless of the performance. :) > > On Wed, Dec 4, 2019 at 12:26 Brett Cannon wrote: > >> >> >> On Tue, Dec 3, 2019 at 5:58 PM Random832 wrote: >> >>> On Tue, Dec 3, 2019, at 13:43, Brett Cannon wrote: >>> > -1 from me. I can see someone not realizing an operator was changed, >>> > assuming it's standard semantics, and then having things break subtly. >>> > And debugging this wouldn't be fun either. To me this is >>> monkeypatching >>> > without an explicit need for it, i.e. if you really want different >>> > semantics in your module then define a function and use that instead >>> of >>> > influence-at-a-distance overriding of syntax. >>> >>> Does it make a difference that it'd only apply to code that is >>> physically in the same module where the function is defined? I'd originally >>> planned to suggest full lexical scope for the lookup, in fact, so you could >>> in theory do it within a single function. >>> >> >> Not enough to change my opinion. Changing how fundamental operators work >> just in a module is still influencing too far from the code site. For >> instance, if I navigate in my editor directly into the middle of a file or >> open a file and immediately start searching for the function I care about I >> won't notice that "+" no longer means what I thought it meant for integers >> because someone thought it would be smart to redefine that. >> >> And as Serhiy pointed out, performance is going to get slammed by this, >> no opcode or not as you just introduced a new lookup on every syntactic >> operation. >> ___ >> Python-ideas mailing list -- python-ideas@python.org >> To unsubscribe send an email to python-ideas-le...@python.org >> https://mail.python.org/mailman3/lists/python-ideas.python.org/ >> Message archived at >> https://mail.python.org/archives/list/python-ideas@python.org/message/DXZO7UVMBXMV7KNBPZU25YA3PQLUI4NF/ >> Code of Conduct: http://python.org/psf/codeofconduct/ >> > -- > --Guido (mobile) > ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/34DZCM7APSYSN2WGVP6U4RDGYYTROQ32/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Allow user extensions to operators [related to: Moving PEP 584 forward (dict + and += operators)]
We could treat it as a kind of future statement. If there’s a top level statement that defines the magic identitier we generate the special bytecode. On Wed, Dec 4, 2019 at 12:26 Brett Cannon wrote: > > > On Tue, Dec 3, 2019 at 5:58 PM Random832 wrote: > >> On Tue, Dec 3, 2019, at 13:43, Brett Cannon wrote: >> > -1 from me. I can see someone not realizing an operator was changed, >> > assuming it's standard semantics, and then having things break subtly. >> > And debugging this wouldn't be fun either. To me this is monkeypatching >> > without an explicit need for it, i.e. if you really want different >> > semantics in your module then define a function and use that instead of >> > influence-at-a-distance overriding of syntax. >> >> Does it make a difference that it'd only apply to code that is physically >> in the same module where the function is defined? I'd originally planned to >> suggest full lexical scope for the lookup, in fact, so you could in theory >> do it within a single function. >> > > Not enough to change my opinion. Changing how fundamental operators work > just in a module is still influencing too far from the code site. For > instance, if I navigate in my editor directly into the middle of a file or > open a file and immediately start searching for the function I care about I > won't notice that "+" no longer means what I thought it meant for integers > because someone thought it would be smart to redefine that. > > And as Serhiy pointed out, performance is going to get slammed by this, no > opcode or not as you just introduced a new lookup on every syntactic > operation. > ___ > Python-ideas mailing list -- python-ideas@python.org > To unsubscribe send an email to python-ideas-le...@python.org > https://mail.python.org/mailman3/lists/python-ideas.python.org/ > Message archived at > https://mail.python.org/archives/list/python-ideas@python.org/message/DXZO7UVMBXMV7KNBPZU25YA3PQLUI4NF/ > Code of Conduct: http://python.org/psf/codeofconduct/ > -- --Guido (mobile) ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/O7HUPYUCMNFRO36FOMWTS7HNY6WXNK2K/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Allow user extensions to operators [related to: Moving PEP 584 forward (dict + and += operators)]
On Tue, Dec 3, 2019 at 5:58 PM Random832 wrote: > On Tue, Dec 3, 2019, at 13:43, Brett Cannon wrote: > > -1 from me. I can see someone not realizing an operator was changed, > > assuming it's standard semantics, and then having things break subtly. > > And debugging this wouldn't be fun either. To me this is monkeypatching > > without an explicit need for it, i.e. if you really want different > > semantics in your module then define a function and use that instead of > > influence-at-a-distance overriding of syntax. > > Does it make a difference that it'd only apply to code that is physically > in the same module where the function is defined? I'd originally planned to > suggest full lexical scope for the lookup, in fact, so you could in theory > do it within a single function. > Not enough to change my opinion. Changing how fundamental operators work just in a module is still influencing too far from the code site. For instance, if I navigate in my editor directly into the middle of a file or open a file and immediately start searching for the function I care about I won't notice that "+" no longer means what I thought it meant for integers because someone thought it would be smart to redefine that. And as Serhiy pointed out, performance is going to get slammed by this, no opcode or not as you just introduced a new lookup on every syntactic operation. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/DXZO7UVMBXMV7KNBPZU25YA3PQLUI4NF/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Allow user extensions to operators [related to: Moving PEP 584 forward (dict + and += operators)]
On Dec 3, 2019, at 08:31, Random832 wrote: > > What if there was a general mechanism to allow operators to be implemented > by user code that does not belong to the class? This seems like something you could build as an import hook pretty easily. Obviously it won’t be as efficient as something using new bytecodes would be, but I don’t think it would be too slow to play with and show off how it can be useful. And instead of people having to ask you how it deals with every edge case (as they think them up one by one), they could just test it. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/NAJP5LEXAJ4W3PZ7O7CAVY5KLGDXOFC5/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Allow user extensions to operators [related to: Moving PEP 584 forward (dict + and += operators)]
04.12.19 18:39, Random832 пише: On Wed, Dec 4, 2019, at 08:26, Serhiy Storchaka wrote: 03.12.19 20:43, Brett Cannon пише: -1 from me. I can see someone not realizing an operator was changed, assuming it's standard semantics, and then having things break subtly. And debugging this wouldn't be fun either. To me this is monkeypatching without an explicit need for it, i.e. if you really want different semantics in your module then define a function and use that instead of influence-at-a-distance overriding of syntax. This will also add a significant performance penalty to any code (even if you do not use any hooks). -1 from me too. My proposal was that any module that never uses any hooks compiles to the exact same bytecode, and executes exactly the same way, as it does now. Only modules where the name of the hook is defined get a different operation that looks for the hook. I specifically wrote it that way to *avoid* any performance penalty for modules that do not use any hooks, or for operators other than the one a hook is defined for. It is not possible. First, the code using operators can be above the code for the hook. Second, the code is generated at the compile time, but the hook is defined at the run time (when all module variables are defined). Third, the hook can be defined not only by the "def" statement in the module body. So the hook should be checked dynamically and it is much more expensive than using a field of the C structure. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/DYWPPICDCRIAH7RVZNNY6WS75UVLIFIX/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Allow user extensions to operators [related to: Moving PEP 584 forward (dict + and += operators)]
On Wed, Dec 4, 2019, at 11:51, Chris Angelico wrote:
> Which of these are you expecting to be detected, and thus cause the
> change in bytecode?
More or less the same sort of operations where the use of the name super is
detected within methods and causes the enclosing class to have a __class__ cell.
> __operatorhook_or__ = lambda obj1, obj2: ...
yes
> def init():
> global __operatorhook_or__
> def __operatorhook_or__(obj1, obj2): ...
> init()
yes
> globals()["__operatorhook_or__"] = lambda obj1, obj2: ...
no
> exec("def __operatorhook_or__(obj1, obj2): ...")
no
> from othermodule import __operatorhook_or__
yes
> from othermodule import *
no
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/BKISATWOR5T3ZJQTAD6STAA5E4ZPJKGK/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Allow user extensions to operators [related to: Moving PEP 584 forward (dict + and += operators)]
On Wed, Dec 4, 2019, at 11:49, Ethan Furman wrote: > On 12/04/2019 08:39 AM, Random832 wrote: > > On Wed, Dec 4, 2019, at 08:26, Serhiy Storchaka wrote: > >> 03.12.19 20:43, Brett Cannon пише: > >>> -1 from me. I can see someone not realizing an operator was changed, > >>> assuming it's standard semantics, and then having things break subtly. > >>> And debugging this wouldn't be fun either. To me this is monkeypatching > >>> without an explicit need for it, i.e. if you really want different > >>> semantics in your module then define a function and use that instead of > >>> influence-at-a-distance overriding of syntax. > >> > >> This will also add a significant performance penalty to any code (even > >> if you do not use any hooks). -1 from me too. > > > > My proposal was that any module that never uses any hooks compiles to the > > exact same bytecode, and executes exactly the same way, as it does now. > > Are the low-level details, such as following the mro, in the bytecode? > If not, the bytecode would be the same either way. No, but my proposal was for a new bytecode operation that is "check for hook and execute operator", with the explicit goal of avoiding both the performance penalty (especially on frequent operations like getattr and call) and the influence-at-a-distance concern. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/YV3JIPNNJ5EORZDQP74GBPRBJ53Y7ZJ6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Allow user extensions to operators [related to: Moving PEP 584 forward (dict + and += operators)]
On Thu, Dec 5, 2019 at 3:41 AM Random832 wrote:
>
> On Wed, Dec 4, 2019, at 08:26, Serhiy Storchaka wrote:
> > 03.12.19 20:43, Brett Cannon пише:
> > > -1 from me. I can see someone not realizing an operator was changed,
> > > assuming it's standard semantics, and then having things break subtly.
> > > And debugging this wouldn't be fun either. To me this is monkeypatching
> > > without an explicit need for it, i.e. if you really want different
> > > semantics in your module then define a function and use that instead of
> > > influence-at-a-distance overriding of syntax.
> >
> > This will also add a significant performance penalty to any code (even
> > if you do not use any hooks). -1 from me too.
>
> My proposal was that any module that never uses any hooks compiles to the
> exact same bytecode, and executes exactly the same way, as it does now. Only
> modules where the name of the hook is defined get a different operation that
> looks for the hook. I specifically wrote it that way to *avoid* any
> performance penalty for modules that do not use any hooks, or for operators
> other than the one a hook is defined for.
>
> (There may be some confusion because of the 'if the hook is undefined' clause
> in the proposal: I intended that to apply if the hook is deleted or
> conditionally assigned - if it's never assigned at all, the code won't look
> for it when the operator is used.)
>
Which of these are you expecting to be detected, and thus cause the
change in bytecode?
__operatorhook_or__ = lambda obj1, obj2: ...
def init():
global __operatorhook_or__
def __operatorhook_or__(obj1, obj2): ...
init()
globals()["__operatorhook_or__"] = lambda obj1, obj2: ...
exec("def __operatorhook_or__(obj1, obj2): ...")
from othermodule import __operatorhook_or__
from othermodule import *
ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/EH4LNLGX53KD2KCMUCTX6TL3C3GZ257L/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Allow user extensions to operators [related to: Moving PEP 584 forward (dict + and += operators)]
On 12/04/2019 08:39 AM, Random832 wrote: On Wed, Dec 4, 2019, at 08:26, Serhiy Storchaka wrote: 03.12.19 20:43, Brett Cannon пише: -1 from me. I can see someone not realizing an operator was changed, assuming it's standard semantics, and then having things break subtly. And debugging this wouldn't be fun either. To me this is monkeypatching without an explicit need for it, i.e. if you really want different semantics in your module then define a function and use that instead of influence-at-a-distance overriding of syntax. This will also add a significant performance penalty to any code (even if you do not use any hooks). -1 from me too. My proposal was that any module that never uses any hooks compiles to the exact same bytecode, and executes exactly the same way, as it does now. Are the low-level details, such as following the mro, in the bytecode? If not, the bytecode would be the same either way. -- ~Ethan~ ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/5EW56XW24HFCCC2ZYZHJX6AWJR5S3EBQ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Allow user extensions to operators [related to: Moving PEP 584 forward (dict + and += operators)]
On Wed, Dec 4, 2019, at 08:26, Serhiy Storchaka wrote: > 03.12.19 20:43, Brett Cannon пише: > > -1 from me. I can see someone not realizing an operator was changed, > > assuming it's standard semantics, and then having things break subtly. > > And debugging this wouldn't be fun either. To me this is monkeypatching > > without an explicit need for it, i.e. if you really want different > > semantics in your module then define a function and use that instead of > > influence-at-a-distance overriding of syntax. > > This will also add a significant performance penalty to any code (even > if you do not use any hooks). -1 from me too. My proposal was that any module that never uses any hooks compiles to the exact same bytecode, and executes exactly the same way, as it does now. Only modules where the name of the hook is defined get a different operation that looks for the hook. I specifically wrote it that way to *avoid* any performance penalty for modules that do not use any hooks, or for operators other than the one a hook is defined for. (There may be some confusion because of the 'if the hook is undefined' clause in the proposal: I intended that to apply if the hook is deleted or conditionally assigned - if it's never assigned at all, the code won't look for it when the operator is used.) ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/QYSBJBP3VGYPVYRGM4FG23QGZKRDNZGI/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Allow user extensions to operators [related to: Moving PEP 584 forward (dict + and += operators)]
03.12.19 20:43, Brett Cannon пише: -1 from me. I can see someone not realizing an operator was changed, assuming it's standard semantics, and then having things break subtly. And debugging this wouldn't be fun either. To me this is monkeypatching without an explicit need for it, i.e. if you really want different semantics in your module then define a function and use that instead of influence-at-a-distance overriding of syntax. This will also add a significant performance penalty to any code (even if you do not use any hooks). -1 from me too. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/HBMYHHVPPIWW6FVT7AJCT2OOBMYAAFZQ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Allow user extensions to operators [related to: Moving PEP 584 forward (dict + and += operators)]
On Tue, Dec 3, 2019, at 13:43, Brett Cannon wrote: > -1 from me. I can see someone not realizing an operator was changed, > assuming it's standard semantics, and then having things break subtly. > And debugging this wouldn't be fun either. To me this is monkeypatching > without an explicit need for it, i.e. if you really want different > semantics in your module then define a function and use that instead of > influence-at-a-distance overriding of syntax. Does it make a difference that it'd only apply to code that is physically in the same module where the function is defined? I'd originally planned to suggest full lexical scope for the lookup, in fact, so you could in theory do it within a single function. ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/CDXXFLM633TQLKKLXFNSU3YZLCGUODW6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Allow user extensions to operators [related to: Moving PEP 584 forward (dict + and += operators)]
On Tue, Dec 3, 2019 at 8:29 AM Random832 wrote:
> What if there was a general mechanism to allow operators to be implemented
> by user code that does not belong to the class?
>
> If the name [e.g.] __operatorhook_or__ is defined anywhere in a module,
> within that module all calls to that operator are replaced with a two-step
> process:
>
> call __operatorhook_or__(dict1, dict2)
> if it returns a value, use that value
> if it returns NotImplemented use the ordinary operator lookup process,
> i.e. dict1.__or__ and dict2.__ror___
> [if __operatorhook_or__ is not defined, treat the same as if it had
> returned NotImplemented]
>
> Then a user could simply do:
>
> def __operatorhook_or__(obj1, obj2):
> if isinstance(obj1, dict) and isinstance(obj2, dict):
> return {**obj1, **obj2}
> return NotImplemented
>
> def __operatorhook_ior__(obj1, obj2):
> if isinstance(obj1, dict):
> obj1.update(obj2)
> return obj1
> return NotImplemented
>
-1 from me. I can see someone not realizing an operator was changed,
assuming it's standard semantics, and then having things break subtly. And
debugging this wouldn't be fun either. To me this is monkeypatching without
an explicit need for it, i.e. if you really want different semantics in
your module then define a function and use that instead of
influence-at-a-distance overriding of syntax.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/YZWWDFL4LY7NEJDF5Y3GHV3URIJ6QIVQ/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Allow user extensions to operators [related to: Moving PEP 584 forward (dict + and += operators)]
On Tue, Dec 3, 2019, at 11:36, Ricky Teachey wrote: > > > What if there was a general mechanism to allow operators to be implemented > > by user code that does not belong to the class? > > > > If the name [e.g.] __operatorhook_or__ is defined anywhere in a module, > > within that module all calls to that operator are replaced with a two-step > > process: > > > > * within that module * > > So this would not happen in other modules when that module was > imported...? But would this mean that if I wanted the operation be > effective for all modules, I would either have to define it in every > module I have written, or import the function by name? E.g.: I figured this was the lesser evil, vs two different modules that want to define an operator differently interfering with each other, or a badly written one interfering with a stdlib module... it also allows modules that don't use the facility to not have to pay for it, since overriding an operator like __getattr__ or __call__ in this way would have an enormous performance cost. This also allows you to get the original operator if you need it by importing from the 'operator' module. > # mod1 > def __operatorhook_or__(obj1, obj2): ... > > # mod2 > from mod1 import __operatorhook_or__ ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/IVTSKHTPI5JPICQEY2XGGMPOI557IQSB/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] Re: Allow user extensions to operators [related to: Moving PEP 584 forward (dict + and += operators)]
> What if there was a general mechanism to allow operators to be implemented > by user code that does not belong to the class? > > If the name [e.g.] __operatorhook_or__ is defined anywhere in a module, > within that module all calls to that operator are replaced with a two-step > process: > > * within that module * So this would not happen in other modules when that module was imported...? But would this mean that if I wanted the operation be effective for all modules, I would either have to define it in every module I have written, or import the function by name? E.g.: # mod1 def __operatorhook_or__(obj1, obj2): ... # mod2 from mod1 import __operatorhook_or__ ___ Python-ideas mailing list -- python-ideas@python.org To unsubscribe send an email to python-ideas-le...@python.org https://mail.python.org/mailman3/lists/python-ideas.python.org/ Message archived at https://mail.python.org/archives/list/python-ideas@python.org/message/GLIFYEA3EDQWBOTHJC5T53DLKPVBWLQU/ Code of Conduct: http://python.org/psf/codeofconduct/
