[Python-ideas] Re: Fill in missing contextvars/asyncio task support

[Mark Gordon]

Yury Selivanov wrote:
> On Mon, Jun 21, 2021 at 7:20 PM Mark Gordon msg...@gmail.com wrote:
> > Yeah, it would indeed inherit the copy. We could, theoretically, make
> > asyncio.Task accept context objects and not copy them, but what would
> > that
> > give us?
> > My main thinking was to just be similar to the closest synchronous analog
> > I'm aware of, contextvars.Context.run. I would think an explanation of why
> > the Context object API exists as it does, letting you manipulate and run in
> > contexts directly, would equally motivate the async analogs. Maybe the
> > exception would be if this API exists purely just to support async tasks
> > (then maybe it should be private?).
> > At any rate, the issue attached to the pull requests gives one example of
> > seeking to do asyncio tests with providing fixture data through an existing
> > context object. I could also imagine a use case of wanting to track the
> > number of database requests made within a logical request that may span
> > multiple tasks. Having the subtasks inherit the same context could help
> > with this.
> > To track things like database requests just put a mutable object in the
> context somewhere at the top level, referenced by a well-known contextvar
> in your code. That single object will be part of all contexts derived from
> the top one throughout the application lifecycle.

Using mutable objects in contexts seems strange as it works against their copy 
semantics. I agree it will work in this use case.

> > *Ultimately, contextvars enable implicit flow
> > of information from outer code to nested code and not vice versa. *
> > Just to clarify, are you stating an established position of the python
> > community or is this your personal view of how context vars should be used?
> > I'm stating this as the original contextvars PEP author and implementer. I

Oh very cool, thanks for taking the time to look at this.

> don't see how the reverse flow is possible to implement within all
> restrictions of the design space. To propagate information from nested
> calls to outer calls just use mutable objects, as I outlined above.
> > Additionally (obviously) it will also be running in a separate task.
> > There's no way around that, unfortunately. Even if we add some kind of
> > helper to run coroutines in a context, there still we be a task object
> > that
> > iterates the coroutine.
> > I was just pointing out that the stated work-around requires creating an
> > additional task to run the called coroutine rather than running directly in
> > the calling task.
> > Yes, and I'm saying that running a coroutine "directly on the stack" within
> some context is not technically possible, without wrapping it into a Task.

Is there something wrong with the solution of changing the context associated 
with a task before execution, yielding, run the coroutine, then swap it back 
and yield again when the coroutine exits? This is the hacky solution that 
appears to work for me on python 3.8 if I force use of _PyTask (and the linked 
PR extends this idea to work for the CTask implementation).

import asyncio.tasks
asyncio.tasks.Task = asyncio.tasks._PyTask
import asyncio
import contextvars

var = contextvars.ContextVar('var', default=123)

async def run_in_context(context, coro):
task = asyncio.current_task()
prev_context = task._context
task._context = context
await asyncio.sleep(0)

try:
return await coro
finally:
task._context = prev_context  
await asyncio.sleep(0)

async def foo(x):
old_val = var.get()
var.set(x)
return old_val

async def main():
var.set(5)

context = contextvars.Context()
print(await run_in_context(context, foo(555))) # 123
print(var.get()) # 5
print(await run_in_context(context, foo(999))) # 555
print(var.get()) # 5
print(context.run(var.get)) # 999

asyncio.run(main())


> > I guess we can add a keyword argument to asyncio.create_task() for that.
> > It
> > is an open question if the task factory would just use the passed context
> > object or would copy it first. I'm leaning towards the latter.
> > My vote would be for not a copy as mentioned above.
> > Having a asyncio.run_in_context(context, coro()) API is more important as
> > this feature is currently completely missing. So happy to table this if we
> > can't decide on if/what semantics this task kwarg API change should have.
> > Is asyncio.run_in_context() a version of asyncio.run() or a shortcut for
> Context.run(asyncio.create_task, coro)?

Yeah I mean it to be more or less the same except without creating a new task 
(which maybe is impossible?).

> Yury
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 

[Python-ideas] Re: Fill in missing contextvars/asyncio task support

[Yury Selivanov]

On Mon, Jun 21, 2021 at 7:20 PM Mark Gordon  wrote:

> > Yeah, it would indeed inherit the copy. We could, theoretically, make
> > asyncio.Task accept context objects and not copy them, but what would
> that
> > give us?
>
> My main thinking was to just be similar to the closest synchronous analog
> I'm aware of, contextvars.Context.run. I would think an explanation of why
> the Context object API exists as it does, letting you manipulate and run in
> contexts directly, would equally motivate the async analogs. Maybe the
> exception would be if this API exists purely just to support async tasks
> (then maybe it should be private?).
>
> At any rate, the issue attached to the pull requests gives one example of
> seeking to do asyncio tests with providing fixture data through an existing
> context object. I could also imagine a use case of wanting to track the
> number of database requests made within a logical request that may span
> multiple tasks. Having the subtasks inherit the same context could help
> with this.
>

To track things like database requests just put a mutable object in the
context somewhere at the top level, referenced by a well-known contextvar
in your code. That single object will be part of all contexts derived from
the top one throughout the application lifecycle.


> > *Ultimately, contextvars enable implicit flow
> > of information from outer code to nested code and not vice versa. *
>
> Just to clarify, are you stating an established position of the python
> community or is this your personal view of how context vars should be used?
>

I'm stating this as the original contextvars PEP author and implementer. I
don't see how the reverse flow is possible to implement within all
restrictions of the design space. To propagate information from nested
calls to outer calls just use mutable objects, as I outlined above.


>
> > > Additionally (obviously) it will also be running in a separate task.
> > There's no way around that, unfortunately. Even if we add some kind of
> > helper to run coroutines in a context, there still we be a task object
> that
> > iterates the coroutine.
>
> I was just pointing out that the stated work-around requires creating an
> additional task to run the called coroutine rather than running directly in
> the calling task.
>

Yes, and I'm saying that running a coroutine "directly on the stack" within
some context is not technically possible, without wrapping it into a Task.


>
> > I guess we can add a keyword argument to asyncio.create_task() for that.
> It
> > is an open question if the task factory would just use the passed context
> > object or would copy it first. I'm leaning towards the latter.
>
> My vote would be for not a copy as mentioned above.
>
> Having a asyncio.run_in_context(context, coro()) API is more important as
> this feature is currently completely missing. So happy to table this if we
> can't decide on if/what semantics this task kwarg API change should have.
>

Is asyncio.run_in_context() a version of asyncio.run() or a shortcut for
Context.run(asyncio.create_task, coro)?

Yury
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CXQAKE65TBOJZGFGEAY5EKIEQ2WRSG6S/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Fill in missing contextvars/asyncio task support

[Mark Gordon]

> Yeah, it would indeed inherit the copy. We could, theoretically, make
> asyncio.Task accept context objects and not copy them, but what would that
> give us?

My main thinking was to just be similar to the closest synchronous analog I'm 
aware of, contextvars.Context.run. I would think an explanation of why the 
Context object API exists as it does, letting you manipulate and run in 
contexts directly, would equally motivate the async analogs. Maybe the 
exception would be if this API exists purely just to support async tasks (then 
maybe it should be private?).

At any rate, the issue attached to the pull requests gives one example of 
seeking to do asyncio tests with providing fixture data through an existing 
context object. I could also imagine a use case of wanting to track the number 
of database requests made within a logical request that may span multiple 
tasks. Having the subtasks inherit the same context could help with this.

> *Ultimately, contextvars enable implicit flow
> of information from outer code to nested code and not vice versa. *

Just to clarify, are you stating an established position of the python 
community or is this your personal view of how context vars should be used?

> > Additionally (obviously) it will also be running in a separate task.
> There's no way around that, unfortunately. Even if we add some kind of
> helper to run coroutines in a context, there still we be a task object that
> iterates the coroutine.

I was just pointing out that the stated work-around requires creating an 
additional task to run the called coroutine rather than running directly in the 
calling task.

> I guess we can add a keyword argument to asyncio.create_task() for that. It
> is an open question if the task factory would just use the passed context
> object or would copy it first. I'm leaning towards the latter.

My vote would be for not a copy as mentioned above.

Having a asyncio.run_in_context(context, coro()) API is more important as this 
feature is currently completely missing. So happy to table this if we can't 
decide on if/what semantics this task kwarg API change should have.

-Mark
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/66P67IVADSUB425HLU55ZYLS2WLEQMQE/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-21 9:39 p.m., Steven D'Aprano wrote:


>
> Fourth step is that you go ahead and use lists as normal. Whether you 
> use getattr or dot syntax, any extension methods defined in spam.py will 
> show up, as if they were actual list methods.
>
> hasattr([], 'head')  # returns True
> list.tail  # returns the spam.tail function object (unbound method)
>
> They're not monkey-patched: other modules don't see that.
>
>

Python is a dynamic language. Maybe you're using hasattr/getattr to
forward something from A to B. If "other modules don't see that" then
this must work as if there were no extension methods in place. So you
actually wouldn't want the local load_attr override to apply to those.
If you did... well, just call the override directly. If the override was
called __opcode_load_attr_impl__ you'd just call
__opcode_load_attr_impl__ directly instead of going through getattr.
There needs to be an escape hatch for this.

Or you *could* have getattr be special (called by load_attr) and
overridable, and builtins.getattr be the escape hatch, but nobody would
like that.
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/VCNS6AEZ7NDMUOD2AEKOWWGH7XLTIRGP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-21 8:57 p.m., Thomas Grainger wrote:
> It seems odd that it would be per module and not per scope?

It's unusual to import things at the scope level. Usually things get
imported at the module level, so, using module language doesn't seem
that bad.

But yes, it's per scope, but in practice it's per module because nobody
would actually use this per scope even tho they could. :p

>
> On Tue, 22 Jun 2021, 00:55 Soni L.,  > wrote:
>
>
>
> On 2021-06-21 8:42 p.m., Steven D'Aprano wrote:
> > On Mon, Jun 21, 2021 at 02:54:52PM -0300, Soni L. wrote:
> >
> > > Quite the opposite. You ask the local module (the one that the
> code was
> > > compiled in), and the module decides whether/when to ask the
> object itself.
> > >
> > > In other words, every
> > >
> > > foo.bar
> > >
> > > would be sugar for
> > >
> > > __getattr__(foo, "bar")
> > >
> > > (where __getattr__ defaults to builtins.getattr) instead of
> being sugar for
> > >
> > > (foo, "bar")
> >
> > All you've done here is push the problem further along -- how does
> > `__getattr__` (`__getattribute__`?) decide what to do?
> >
> > * Why is this extension-aware version per module, instead of a
> builtin?
> >
> > * Does that mean the caller has to write it in every module they
> want to
> >   make use of extensions?
> >
> > * Why do we need a second attribute lookup mechanism instead of
> having
> >   the existing mechanism do the work?
> >
> > * And most problematic, if we have an extension method on a
> type, the
> >   builtin getattr ought to pick it up.
> >
> >
> > By the way, per-module `__getattr__` already has a meaning, so
> this name
> > won't fly.
> >
> > https://www.python.org/dev/peps/pep-0562/
> 
> >
> >
>
> No, you got it wrong. Extension methods don't go *on* the type being
> extended. Indeed, that's how they differ from monkeypatching.
>
> The whole point of extension methods *is* to be per-module. You could
> shove it in the existing attribute lookup mechanism (aka the
> builtins.getattr) but that involves runtime reflection, whereas
> making a
> new, per-module attribute lookup mechanism specifically designed to
> support a per-module feature would be a lot better.
>
> Extension methods *do not go on the type*.
>
> And sure, let's call it __opcode_load_attr_impl__ instead. Sounds
> good?
> ___
> Python-ideas mailing list -- python-ideas@python.org
> 
> To unsubscribe send an email to python-ideas-le...@python.org
> 
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> 
> Message archived at
> 
> https://mail.python.org/archives/list/python-ideas@python.org/message/MTOP22VK2ZC3GWCQHU5RDFVIT5AAR4DW/
> 
> 
> Code of Conduct: http://python.org/psf/codeofconduct/
> 
>

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MBHRH4FTGN7NCCHEHU3HVE4UPDDWHHHY/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Steven D'Aprano]

On Tue, Jun 22, 2021 at 03:56:00AM +1000, Chris Angelico wrote:

> I'm actually not concerned so much with the performance as the
> confusion. What exactly does the registration apply to?

Good question.

Extension methods have four steps:

- you write a method;

- declare which class it extends;

- the caller declares that they want to use extensions;

- and they get looked up at runtime (because we can't do static 
  lookups).


The first two can go together. I might write a module "spam.py". 
Borrowing a mix of Kotlin and/or C# syntax, maybe I write:

def list.head(self, arg):
...

def list.tail(self, arg):
...

or maybe we have a decorator:

@extends(list)
def head(self, arg):
...


The third step happens at the caller site. Using the C# keyword, you 
might write this in your module "stuff.py":

uses spam

or maybe there's a way to do it with the import keyword:

# could be confused for `from spam import extensions`?
import extensions from spam

from functools import extension_methods
import spam
extension_methods.load_from(spam)


whatever it takes. Depends on how much of this needs to be baked into 
the interpreter.

Fourth step is that you go ahead and use lists as normal. Whether you 
use getattr or dot syntax, any extension methods defined in spam.py will 
show up, as if they were actual list methods.

hasattr([], 'head')  # returns True
list.tail  # returns the spam.tail function object (unbound method)

They're not monkey-patched: other modules don't see that.


> And suppose
> you have a series of extension methods that you want to make use of in
> several modules in your project, how can you refactor a bunch of
> method registration calls so you can apply them equally in multiple
> modules? We don't need an implementation yet - but we need clear
> semantics.

I put the extension modules in one library. That may not literally 
require me to put their definitions in a single .py file, I should be 
able to use a package and import extension methods from modules the same 
as any other object. But for ease of use for the caller, I probably want 
to make all my related extension methods usable from a single place.

Then you, the caller, import/use them from each of your modules where 
you want to use them:

# stuff.py
uses spam

# things.py
uses spam

And in modules where you don't want to use them, you just don't use 
them.


[...]
> True, all true, but considering that this is *not* actually part of
> the class, some of that doesn't really apply. For instance, is it
> really encapsulation? What does that word even mean when you're
> injecting methods in from the outside?

Sure it's encapsulation. We can already do this with non-builtin 
classes:

class SpammySpam:
def spam(self, arg):
...

from another_module import eggy_method

def aardvarks(self, foo, bar):
...

SpammySpam.aardvarks = aardvarks


The fact that two of those methods have source code that wasn't indented 
under the class statement is neither here nor there. Even the fact that 
eggy_method was defined in another module is irrelevant. What matters is 
that once I've put the class together, all three methods are fully 
encapsulated into the SpammySpam class, and other classes can define 
different methods with the same name.

Encapsulation is less about where you write the source code, and more 
about the fact that I can have

SpammySpam().spam

and 

Advertising().spam

without the two spam methods stomping on each other.


[...]
> And that's a very very big "if". Monkey-patching can be used for
> unittest mocking, but that won't work here. Monkey-patching can be
> used to fix bugs in someone else's code, but that only works here if
> *your* code is in a single module, or you reapply the monkey-patch in
> every module. I'm really not seeing a lot of value in the proposal.

LINQ is a pretty major part of the C# ecosystem. I think that 
proves the value of extension methods :-)

I know we're not really comparing apples with apples, Python's 
trade-offs are not the same as C#'s trade-offs. But Ruby is a dynamic 
language like Python, and they use monkey-patching all the time, proving 
the value of being able to extend classes without subclassing them.

Extension methods let us extend classes without the downsides of 
monkey-patching. Extension methods are completely opt-in while 
monkey-patching is mandatory for everyone. If we could only have one, 
extension methods would clearly be the safer choice.

We don't make heavy use of monkey-patching, not because it isn't a 
useful technique, but because:

- unlike Ruby, we can't extend builtins without subclassing;

- we're very aware that monkey-patching is a massively powerful 
  technique with huge foot-gun potential;

- and most of all, the Python community is a hell of a lot more 
  conservative than Ruby.

Even basic techniques intentionally 

[Python-ideas] Re: Fill in missing contextvars/asyncio task support

[Yury Selivanov]

> However this is not exactlly the same as the task will inherit a copy of
my_context rather than running directly on my_context.

Yeah, it would indeed inherit the copy. We could, theoretically, make
asyncio.Task accept context objects and not copy them, but what would that
give us? If a coroutine awaits on some code that calls `copy_context()`
internally you will not be able to observe the modifications that code
makes to its forked context. *Ultimately, contextvars enable implicit flow
of information from outer code to nested code and not vice versa. *

> Additionally (obviously) it will also be running in a separate task.

There's no way around that, unfortunately. Even if we add some kind of
helper to run coroutines in a context, there still we be a task object that
iterates the coroutine.

> Similarly it would be nice if create_task and the Task constructor could
take an optional context kwarg to use as the task context rather than the
default of copying the calling context.

I guess we can add a keyword argument to asyncio.create_task() for that. It
is an open question if the task factory would just use the passed context
object or would copy it first. I'm leaning towards the latter.

Yury



On Sun, Jun 20, 2021 at 3:13 PM Mark Gordon  wrote:

> With normal synchronous code you can use `contextvars.Context.run()` to
> change what context code is executing within. However, there is no
> analagous concept for asyncio code. I'm proposing something similar, for
> example:
>
> coro = foo()
> my_context = convextvars.Context()
> await asyncio.run_in_context(coro)
>
> Currently the workaround is to run the coroutine on a separate task.
>
> coro = foo()
> my_context = convextvars.Context()
> await my_context.run(asycnio.create_task, coro)
>
> However this is not exactlly the same as the task will inherit a copy of
> my_context rather than running directly on my_context. Additionally
> (obviously) it will also be running in a separate task.
>
> Similarly it would be nice if create_task and the Task constructor could
> take an optional context kwarg to use as the task context rather than the
> default of copying the calling context.
>
> Pull request with sample implementation (although I think missing the
> change to create_task): https://github.com/python/cpython/pull/26664
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/3EQO67J7IBAO6YSK2LBIPU4E7HU6UQJH/
> Code of Conduct: http://python.org/psf/codeofconduct/
>


-- 
 Yury
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/J7NOEWBUSB5TLHD2ISB5HC3GB5E6AAVQ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Thomas Grainger]

It seems odd that it would be per module and not per scope?

On Tue, 22 Jun 2021, 00:55 Soni L.,  wrote:

>
>
> On 2021-06-21 8:42 p.m., Steven D'Aprano wrote:
> > On Mon, Jun 21, 2021 at 02:54:52PM -0300, Soni L. wrote:
> >
> > > Quite the opposite. You ask the local module (the one that the code was
> > > compiled in), and the module decides whether/when to ask the object
> itself.
> > >
> > > In other words, every
> > >
> > > foo.bar
> > >
> > > would be sugar for
> > >
> > > __getattr__(foo, "bar")
> > >
> > > (where __getattr__ defaults to builtins.getattr) instead of being
> sugar for
> > >
> > > (foo, "bar")
> >
> > All you've done here is push the problem further along -- how does
> > `__getattr__` (`__getattribute__`?) decide what to do?
> >
> > * Why is this extension-aware version per module, instead of a builtin?
> >
> > * Does that mean the caller has to write it in every module they want to
> >   make use of extensions?
> >
> > * Why do we need a second attribute lookup mechanism instead of having
> >   the existing mechanism do the work?
> >
> > * And most problematic, if we have an extension method on a type, the
> >   builtin getattr ought to pick it up.
> >
> >
> > By the way, per-module `__getattr__` already has a meaning, so this name
> > won't fly.
> >
> > https://www.python.org/dev/peps/pep-0562/
> >
> >
>
> No, you got it wrong. Extension methods don't go *on* the type being
> extended. Indeed, that's how they differ from monkeypatching.
>
> The whole point of extension methods *is* to be per-module. You could
> shove it in the existing attribute lookup mechanism (aka the
> builtins.getattr) but that involves runtime reflection, whereas making a
> new, per-module attribute lookup mechanism specifically designed to
> support a per-module feature would be a lot better.
>
> Extension methods *do not go on the type*.
>
> And sure, let's call it __opcode_load_attr_impl__ instead. Sounds good?
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/MTOP22VK2ZC3GWCQHU5RDFVIT5AAR4DW/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/OQOJ3LZLCYKCYJW2ZTHQ2OQULKDFPWVC/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-21 8:42 p.m., Steven D'Aprano wrote:
> On Mon, Jun 21, 2021 at 02:54:52PM -0300, Soni L. wrote:
>
> > Quite the opposite. You ask the local module (the one that the code was
> > compiled in), and the module decides whether/when to ask the object itself.
> > 
> > In other words, every
> > 
> > foo.bar
> > 
> > would be sugar for
> > 
> > __getattr__(foo, "bar")
> > 
> > (where __getattr__ defaults to builtins.getattr) instead of being sugar for
> > 
> > (foo, "bar")
>
> All you've done here is push the problem further along -- how does 
> `__getattr__` (`__getattribute__`?) decide what to do?
>
> * Why is this extension-aware version per module, instead of a builtin?
>
> * Does that mean the caller has to write it in every module they want to
>   make use of extensions?
>
> * Why do we need a second attribute lookup mechanism instead of having
>   the existing mechanism do the work?
>
> * And most problematic, if we have an extension method on a type, the 
>   builtin getattr ought to pick it up.
>
>
> By the way, per-module `__getattr__` already has a meaning, so this name 
> won't fly.
>
> https://www.python.org/dev/peps/pep-0562/
>
>

No, you got it wrong. Extension methods don't go *on* the type being
extended. Indeed, that's how they differ from monkeypatching.

The whole point of extension methods *is* to be per-module. You could
shove it in the existing attribute lookup mechanism (aka the
builtins.getattr) but that involves runtime reflection, whereas making a
new, per-module attribute lookup mechanism specifically designed to
support a per-module feature would be a lot better.

Extension methods *do not go on the type*.

And sure, let's call it __opcode_load_attr_impl__ instead. Sounds good?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/MTOP22VK2ZC3GWCQHU5RDFVIT5AAR4DW/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Steven D'Aprano]

On Mon, Jun 21, 2021 at 02:54:52PM -0300, Soni L. wrote:

> Quite the opposite. You ask the local module (the one that the code was
> compiled in), and the module decides whether/when to ask the object itself.
> 
> In other words, every
> 
> foo.bar
> 
> would be sugar for
> 
> __getattr__(foo, "bar")
> 
> (where __getattr__ defaults to builtins.getattr) instead of being sugar for
> 
> (foo, "bar")

All you've done here is push the problem further along -- how does 
`__getattr__` (`__getattribute__`?) decide what to do?

* Why is this extension-aware version per module, instead of a builtin?

* Does that mean the caller has to write it in every module they want to
  make use of extensions?

* Why do we need a second attribute lookup mechanism instead of having
  the existing mechanism do the work?

* And most problematic, if we have an extension method on a type, the 
  builtin getattr ought to pick it up.


By the way, per-module `__getattr__` already has a meaning, so this name 
won't fly.

https://www.python.org/dev/peps/pep-0562/


-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/BK5VM4IVWW7UFQ3D3RK5LBMKQOYBNIDV/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[André Roberge]

On Mon, Jun 21, 2021 at 3:28 PM Soni L.  wrote:

>
>
>
> For exec/eval you just pass in the locals:
>
> exec(foo, globals(), locals())
>
> because this __getattribute__ is just a local like any other.
>
> As for each module, you'd import them. But not quite with "import":
>
> import extension_methods # magic module, probably provides an
> @extend(class_) e.g. @extend(list)
> import shallow_flatten
> import deep_flatten
> __getattribute__ = extension_methods.getattribute(
>   shallow_flatten.flatten, # uses __name__
>   deepflatten=deep_flatten.flatten, # name override
>   __getattribute__=__getattribute__, # optional, defaults to
> builtins.getattr
> )
>
> This would have to be done for each .py that wants to use the extension
> methods.
>

I bet you that you could already do this today with a custom import hook.
If you want to "easily" experiment with this, I would suggest having a look
at https://aroberge.github.io/ideas/docs/html/index.html
which likely has all the basic scaffolding that you would need.

André Roberge



> >
> > ChrisA
> > ___
> > Python-ideas mailing list -- python-ideas@python.org
> > To unsubscribe send an email to python-ideas-le...@python.org
> > https://mail.python.org/mailman3/lists/python-ideas.python.org/
> > Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/3DL46SIS2A7D3W4FNSTH37O6VDJJB2ZP/
> > Code of Conduct: http://python.org/psf/codeofconduct/
>
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/ROZN62HQYL6X5LCJ7J5DNOC7W3DBGYHF/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KLEBIWRLYLLXHTJT264OPMSVY3CBUG37/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-21 3:01 p.m., Chris Angelico wrote:
> Thanks for clarifying. This doesn't change the problem though - it
> just changes where the issue shows up. (BTW, what you're describing is
> closer to __getattribute__ than it is to __getattr__, so if you're
> proposing this as the semantics, I strongly recommend going with that
> name.)

Oh, sorry, thought __getattribute__ was the fallback and __getattr__ the
one always called, what with getattr -> __getattr__. But yeah,
__getattribute__ then.

>
> So, here's the question - a clarification of what I asked vaguely up
> above. Suppose you have a bunch of these extension methods, and a
> large project. How are you going to register the right extension
> methods in the right modules within your project? You're binding the
> functionality to the module in which the code was compiled, which will
> make exec/eval basically unable to use them, and that means you'll
> need some way to set them in each module, or to import the setting
> from somewhere else. How do you propose doing this?

For exec/eval you just pass in the locals:

exec(foo, globals(), locals())

because this __getattribute__ is just a local like any other.

As for each module, you'd import them. But not quite with "import":

import extension_methods # magic module, probably provides an
@extend(class_) e.g. @extend(list)
import shallow_flatten
import deep_flatten
__getattribute__ = extension_methods.getattribute(
  shallow_flatten.flatten, # uses __name__
  deepflatten=deep_flatten.flatten, # name override
  __getattribute__=__getattribute__, # optional, defaults to
builtins.getattr
)

This would have to be done for each .py that wants to use the extension
methods.

>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/3DL46SIS2A7D3W4FNSTH37O6VDJJB2ZP/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/ROZN62HQYL6X5LCJ7J5DNOC7W3DBGYHF/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Chris Angelico]

On Tue, Jun 22, 2021 at 3:55 AM Soni L.  wrote:
>
>
>
> On 2021-06-21 12:49 p.m., Chris Angelico wrote:
> > On Tue, Jun 22, 2021 at 1:44 AM Soni L.  wrote:
> > >
> > >
> > >
> > > On 2021-06-21 12:26 p.m., Stephen J. Turnbull wrote:
> > > > Soni L. writes:
> > > >
> > > >  > The trick to extension methods is that they're only available when 
> > > > you
> > > >  > explicitly use them.
> > > >
> > > > What does "explicitly use them" mean?  How does this help avoid the
> > > > kinds of problems we know that monkey-patching causes?
> > >
> > > Monkey-patching:
> > >
> > > ```py mod1.py
> > > import foo
> > >
> > > foo.Bar.monkeymethod = ...
> > > ```
> > >
> > > ```py mod2.py
> > > import foo
> > >
> > > foo.Bar.monkeymethod = ...
> > > ```
> > >
> > > "Extension methods":
> > >
> > > ```py mod1.py
> > > import foo
> > >
> > > def __getattr__(o, attr):
> > >   if isinstance(o, foo.Bar) and attr == "monkeymethod":
> > > return ...
> > >   return getattr(o, attr)
> > > ```
> > >
> > > ```py mod2.py
> > > import foo
> > >
> > > def __getattr__(o, attr):
> > >   if isinstance(o, foo.Bar) and attr == "monkeymethod":
> > > return ...
> > >   return getattr(o, attr)
> > > ```
> > >
> > > Note how the former changes foo.Bar, whereas the latter only changes the
> > > module's own __getattr__. You can't have conflicts with the latter.
> > > (Also note that this "module's own __getattr__" doesn't provide
> > > extension methods by itself, but can be used as a mechanism to implement
> > > extension methods.)
> >
> > So what you're saying is that, in effect, every attribute lookup has
> > to first ask the object itself, and then ask the module? Which module?
> > The one that the code was compiled in? The one that is currently
> > running? Both?
> >
> > And how is this better than just using a plain ordinary function? Not
> > everything has to be a method.
>
> Quite the opposite. You ask the local module (the one that the code was
> compiled in), and the module decides whether/when to ask the object itself.
>
> In other words, every
>
> foo.bar
>
> would be sugar for
>
> __getattr__(foo, "bar")
>
> (where __getattr__ defaults to builtins.getattr) instead of being sugar for
>
> (foo, "bar")
>
> (where <> is used to indicate that it doesn't quite desugar that way -
> otherwise you'd need to recursively desugar it to
> builtins.getattr(builtins, "getattr") which uh, doesn't work.)
>

Thanks for clarifying. This doesn't change the problem though - it
just changes where the issue shows up. (BTW, what you're describing is
closer to __getattribute__ than it is to __getattr__, so if you're
proposing this as the semantics, I strongly recommend going with that
name.)

So, here's the question - a clarification of what I asked vaguely up
above. Suppose you have a bunch of these extension methods, and a
large project. How are you going to register the right extension
methods in the right modules within your project? You're binding the
functionality to the module in which the code was compiled, which will
make exec/eval basically unable to use them, and that means you'll
need some way to set them in each module, or to import the setting
from somewhere else. How do you propose doing this?

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/3DL46SIS2A7D3W4FNSTH37O6VDJJB2ZP/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-21 12:49 p.m., Chris Angelico wrote:
> On Tue, Jun 22, 2021 at 1:44 AM Soni L.  wrote:
> >
> >
> >
> > On 2021-06-21 12:26 p.m., Stephen J. Turnbull wrote:
> > > Soni L. writes:
> > >
> > >  > The trick to extension methods is that they're only available when you
> > >  > explicitly use them.
> > >
> > > What does "explicitly use them" mean?  How does this help avoid the
> > > kinds of problems we know that monkey-patching causes?
> >
> > Monkey-patching:
> >
> > ```py mod1.py
> > import foo
> >
> > foo.Bar.monkeymethod = ...
> > ```
> >
> > ```py mod2.py
> > import foo
> >
> > foo.Bar.monkeymethod = ...
> > ```
> >
> > "Extension methods":
> >
> > ```py mod1.py
> > import foo
> >
> > def __getattr__(o, attr):
> >   if isinstance(o, foo.Bar) and attr == "monkeymethod":
> > return ...
> >   return getattr(o, attr)
> > ```
> >
> > ```py mod2.py
> > import foo
> >
> > def __getattr__(o, attr):
> >   if isinstance(o, foo.Bar) and attr == "monkeymethod":
> > return ...
> >   return getattr(o, attr)
> > ```
> >
> > Note how the former changes foo.Bar, whereas the latter only changes the
> > module's own __getattr__. You can't have conflicts with the latter.
> > (Also note that this "module's own __getattr__" doesn't provide
> > extension methods by itself, but can be used as a mechanism to implement
> > extension methods.)
>
> So what you're saying is that, in effect, every attribute lookup has
> to first ask the object itself, and then ask the module? Which module?
> The one that the code was compiled in? The one that is currently
> running? Both?
>
> And how is this better than just using a plain ordinary function? Not
> everything has to be a method.

Quite the opposite. You ask the local module (the one that the code was
compiled in), and the module decides whether/when to ask the object itself.

In other words, every

foo.bar

would be sugar for

__getattr__(foo, "bar")

(where __getattr__ defaults to builtins.getattr) instead of being sugar for

(foo, "bar")

(where <> is used to indicate that it doesn't quite desugar that way -
otherwise you'd need to recursively desugar it to
builtins.getattr(builtins, "getattr") which uh, doesn't work.)

>
> ChrisA
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at 
> https://mail.python.org/archives/list/python-ideas@python.org/message/EMA2EBZLIR7DLNJAHWHURATJ4WVBOEUJ/
> Code of Conduct: http://python.org/psf/codeofconduct/

___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/AGYFBB6XJ4V7ASID54PNGFZK74ZD4HWA/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Chris Angelico]

On Tue, Jun 22, 2021 at 3:43 AM Steven D'Aprano  wrote:
>
> On Tue, Jun 22, 2021 at 01:49:56AM +1000, Chris Angelico wrote:
>
> > So what you're saying is that, in effect, every attribute lookup has
> > to first ask the object itself, and then ask the module? Which module?
> > The one that the code was compiled in? The one that is currently
> > running? Both?
>
> Mu.
>
> https://en.wikipedia.org/wiki/Mu_(negative)#%22Unasking%22_the_question
>
> We don't have an implementation yet, so it is too early to worry about
> precisely where you look up the extension methods, except that it is
> opt-in (so by default, there's no additional cost involved) and there
> must be *some* sort of registry *somewhere* that handles the mapping of
> extension methods to classes.
>
> We certainly don't want this to slow down *every method call*, but if it
> only slowed down method lookups a little bit when you actually used the
> feature, that might be acceptable. We already make use of lots of
> features which are slow as continental drift compared to C, because they
> add power to the language and are *fast enough*.
>
> E.g. name lookups are resolved at runtime, not compile-time; dynamic
> attribute lookups using gettattribute and getattr dunders; virtual
> subclasses; generic functions (functools.singledispatch); descriptors.

I'm actually not concerned so much with the performance as the
confusion. What exactly does the registration apply to? And suppose
you have a series of extension methods that you want to make use of in
several modules in your project, how can you refactor a bunch of
method registration calls so you can apply them equally in multiple
modules? We don't need an implementation yet - but we need clear
semantics.

> > And how is this better than just using a plain ordinary function? Not
> > everything has to be a method.
>
> You get method syntax, obj.method, which is nice but not essential.
>
> When a method is called from an instance, you know that the first
> parameter `self` has got to be the correct type, no type-checking is
> required. That's good. And the same would apply to extension methods.
>
> You get bound methods as first class values, which is useful.
>
> You get inheritance, which is powerful.
>
> And you get encapsulation, which is important.

True, all true, but considering that this is *not* actually part of
the class, some of that doesn't really apply. For instance, is it
really encapsulation? What does that word even mean when you're
injecting methods in from the outside?

> I think this is a Blub moment. We don't think it's useful because we
> have functions, and we're not Java, so "not everything needs to be a
> method". Sure, but methods are useful, and they do bring benefits that
> top-level functions don't have. (And vice versa of course.)
>
> We have staticmethod that allows us to write a "function" (-ish) but get
> the benefits of inheritance, encapsulation, and method syntax. This
> would be similar.
>
> We acknowledge that there are benefits to monkey-patching. But we can't
> monkey-patch builtins and we are (rightly) suspicious of those who use
> monkey-patching in production. And this is good. But this would give us
> the benefits of monkey-patching without the disadvantages.
>
> *If* we can agree on semantics and come up with a reasonable efficient
> implementation that doesn't slow down every method call.
>

And that's a very very big "if". Monkey-patching can be used for
unittest mocking, but that won't work here. Monkey-patching can be
used to fix bugs in someone else's code, but that only works here if
*your* code is in a single module, or you reapply the monkey-patch in
every module. I'm really not seeing a lot of value in the proposal.

Let's completely ignore the performance cost for the moment and just
try to figure out semantics, with it being actually useful and not
unwieldy.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/HGZ73H5YT3WSD6B26SWOMVYC6W6POGM2/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Steven D'Aprano]

On Tue, Jun 22, 2021 at 01:49:56AM +1000, Chris Angelico wrote:

> So what you're saying is that, in effect, every attribute lookup has
> to first ask the object itself, and then ask the module? Which module?
> The one that the code was compiled in? The one that is currently
> running? Both?

Mu.

https://en.wikipedia.org/wiki/Mu_(negative)#%22Unasking%22_the_question

We don't have an implementation yet, so it is too early to worry about 
precisely where you look up the extension methods, except that it is 
opt-in (so by default, there's no additional cost involved) and there 
must be *some* sort of registry *somewhere* that handles the mapping of 
extension methods to classes.

We certainly don't want this to slow down *every method call*, but if it 
only slowed down method lookups a little bit when you actually used the 
feature, that might be acceptable. We already make use of lots of 
features which are slow as continental drift compared to C, because they 
add power to the language and are *fast enough*.

E.g. name lookups are resolved at runtime, not compile-time; dynamic 
attribute lookups using gettattribute and getattr dunders; virtual 
subclasses; generic functions (functools.singledispatch); descriptors.


> And how is this better than just using a plain ordinary function? Not
> everything has to be a method.

You get method syntax, obj.method, which is nice but not essential.

When a method is called from an instance, you know that the first 
parameter `self` has got to be the correct type, no type-checking is 
required. That's good. And the same would apply to extension methods.

You get bound methods as first class values, which is useful.

You get inheritance, which is powerful.

And you get encapsulation, which is important.

I think this is a Blub moment. We don't think it's useful because we 
have functions, and we're not Java, so "not everything needs to be a 
method". Sure, but methods are useful, and they do bring benefits that 
top-level functions don't have. (And vice versa of course.)

We have staticmethod that allows us to write a "function" (-ish) but get 
the benefits of inheritance, encapsulation, and method syntax. This 
would be similar.

We acknowledge that there are benefits to monkey-patching. But we can't 
monkey-patch builtins and we are (rightly) suspicious of those who use 
monkey-patching in production. And this is good. But this would give us 
the benefits of monkey-patching without the disadvantages.

*If* we can agree on semantics and come up with a reasonable efficient 
implementation that doesn't slow down every method call.

-- 
Steve
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/QCEC764ILFOUDWO5HXVKTTJSA6XIT6KI/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Flatten function (was: Deprecate sum of lists)

[Christopher Barker]

>
> If there is a problem to solve here it's that people want to
> concatenate things and reach out to the sum function because the
> concatenate function doesn't exist.


But that’s not the problem here — sum IS concatenation for any type for
which __add__ means concatonate.

The problem is that continuous concatenation is inefficient for many types.

np.concatonate() exists because __add__ means something different for
arrays. And it is only used for arrays, so it can be done efficiently.

But how would a builtin concatonate() function know how to concatonate
arbitrary objects?

I suppose there could be a __concat__ dunder. Which is what __add__ already
is for Sequences.

The fact is that only the type itself knows how to efficiently concatonate
a lot of smaller objects. Which is why str.join() is a string method — it
is about strings, and only works with strings.

Which makes me think maybe the solution is to make join() (or call it
concatonate()) a Sequence method.

Or, since it’s problematic to add new methods to ABCs ( the name may
already be used by existing custom Sequence types), go back to the
__concat__ dunder, which would take an  iterable of objects to concatonate.

Which inspires a new idea: extend the Sequence __add__ to take an optional
additional parameter, which could be an iterable of objects to add up.

I’m not sure which is more expensive— trying to pass an extra parameter to
an object to see if it will except it, or checking for the existence for a
dunder. But I do like that there wouldn’t be a need for a new dunder.

What remains is a key question of what people need: a generic and efficient
way to concatonate Sequences, or a way to flatten sequences.

They are overlap, but are not the same, particularly if there is a need to
flatten more than one level deep.

-CHB


The sum function refuses to sum
> strings but the preferred alternative is
> text = ''.join(strings)
> which is just weird looking. Why would you call a method on the empty
> string? It would be much better spelt as
> text = concatenate(strings)
> and that spelling could work equally well for lists, tuples, etc.
>
> --
> Oscar
> ___
> Python-ideas mailing list -- python-ideas@python.org
> To unsubscribe send an email to python-ideas-le...@python.org
> https://mail.python.org/mailman3/lists/python-ideas.python.org/
> Message archived at
> https://mail.python.org/archives/list/python-ideas@python.org/message/YFGGY4TDNHHEGJ2T5UED67W44SXRKNHA/
> Code of Conduct: http://python.org/psf/codeofconduct/
>
-- 
Christopher Barker, PhD (Chris)

Python Language Consulting
  - Teaching
  - Scientific Software Development
  - Desktop GUI and Web Development
  - wxPython, numpy, scipy, Cython
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/KRMARZDFCU4K3ORS3TYP4HPY7DJYEF4H/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Chris Angelico]

On Tue, Jun 22, 2021 at 1:44 AM Soni L.  wrote:
>
>
>
> On 2021-06-21 12:26 p.m., Stephen J. Turnbull wrote:
> > Soni L. writes:
> >
> >  > The trick to extension methods is that they're only available when you
> >  > explicitly use them.
> >
> > What does "explicitly use them" mean?  How does this help avoid the
> > kinds of problems we know that monkey-patching causes?
>
> Monkey-patching:
>
> ```py mod1.py
> import foo
>
> foo.Bar.monkeymethod = ...
> ```
>
> ```py mod2.py
> import foo
>
> foo.Bar.monkeymethod = ...
> ```
>
> "Extension methods":
>
> ```py mod1.py
> import foo
>
> def __getattr__(o, attr):
>   if isinstance(o, foo.Bar) and attr == "monkeymethod":
> return ...
>   return getattr(o, attr)
> ```
>
> ```py mod2.py
> import foo
>
> def __getattr__(o, attr):
>   if isinstance(o, foo.Bar) and attr == "monkeymethod":
> return ...
>   return getattr(o, attr)
> ```
>
> Note how the former changes foo.Bar, whereas the latter only changes the
> module's own __getattr__. You can't have conflicts with the latter.
> (Also note that this "module's own __getattr__" doesn't provide
> extension methods by itself, but can be used as a mechanism to implement
> extension methods.)

So what you're saying is that, in effect, every attribute lookup has
to first ask the object itself, and then ask the module? Which module?
The one that the code was compiled in? The one that is currently
running? Both?

And how is this better than just using a plain ordinary function? Not
everything has to be a method.

ChrisA
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/EMA2EBZLIR7DLNJAHWHURATJ4WVBOEUJ/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Soni L.]

On 2021-06-21 12:26 p.m., Stephen J. Turnbull wrote:
> Soni L. writes:
>
>  > The trick to extension methods is that they're only available when you
>  > explicitly use them.
>
> What does "explicitly use them" mean?  How does this help avoid the
> kinds of problems we know that monkey-patching causes?

Monkey-patching:

```py mod1.py
import foo

foo.Bar.monkeymethod = ...
```

```py mod2.py
import foo

foo.Bar.monkeymethod = ...
```

"Extension methods":

```py mod1.py
import foo

def __getattr__(o, attr):
  if isinstance(o, foo.Bar) and attr == "monkeymethod":
    return ...
  return getattr(o, attr)
```

```py mod2.py
import foo

def __getattr__(o, attr):
  if isinstance(o, foo.Bar) and attr == "monkeymethod":
    return ...
  return getattr(o, attr)
```

Note how the former changes foo.Bar, whereas the latter only changes the
module's own __getattr__. You can't have conflicts with the latter.
(Also note that this "module's own __getattr__" doesn't provide
extension methods by itself, but can be used as a mechanism to implement
extension methods.)
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/RD5LKZUBBDAUH7UUMCRY6HUYZHPSKFQH/
Code of Conduct: http://python.org/psf/codeofconduct/

[Python-ideas] Re: Extension methods in Python

[Stephen J. Turnbull]

Soni L. writes:

 > The trick to extension methods is that they're only available when you
 > explicitly use them.

What does "explicitly use them" mean?  How does this help avoid the
kinds of problems we know that monkey-patching causes?
___
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at 
https://mail.python.org/archives/list/python-ideas@python.org/message/CIWP46E7WGBXZ3TLUYSKDEW6LKPS4CSV/
Code of Conduct: http://python.org/psf/codeofconduct/