On Wed, Jun 23, 2021 at 11:25 PM Steven D'Aprano <st...@pearwood.info> wrote:
>
> On Wed, Jun 23, 2021 at 03:47:05PM +1000, Chris Angelico wrote:
>
> > Okay. Lemme give it to you *even more clearly* since the previous
> > example didn't satisfy.
> >
> > # file1.py
> >
> > @extend(list)
> > def in_order(self):
> > return sorted(self)
> >
> > def frob(stuff):
> > return stuff.in_order()
> >
> > # file2.py
> >
> > from file1 import frob
> > thing = [1, 5, 2]
> > frob(thing) # == [1, 2, 5]
> > def otherfrob(stuff):
> > return stuff.in_order()
> > otherfrob(thing) # AttributeError
> >
> >
> >
> > Am I correct so far? The function imported from file1 has the
> > extension method, the code in file2 does not. That's the entire point
> > here, right?
>
> Correct so far.
>
>
> > Okay. Now, what if getattr is brought into the mix?
>
> To a first approximation (ignoring shadowing) every dot lookup can be
> replaced with getattr and vice versa:
>
> obj.name <--> getattr(obj, 'name')
>
> A simple source code transformation could handle that, and the behaviour
> of the code should be the same. Extension methods shouldn't change that.
Alright. In that case, getattr() has stopped being a function, and is
now a magical construct of the compiler. What happens if I do this?
if random.randrange(2):
def getattr(obj, attr): return lambda: "Hello, world"
def foo(thing):
return getattr(thing, "in_order")()
Does it respect extension methods or not? If getattr is a perfectly
ordinary function, as it now is, then it should be perfectly
acceptable to shadow it. It should also be perfectly acceptable to use
any other way of accessing attributes - for instance, the
PyObject_GetAttr() function in C.
Why should getattr() become magical?
> > import random
> >
> > @extend(list)
> > def unordered(self):
> > return random.shuffle(self[:])
>
> I think that's going to always return None :-)
Oops, my bad :) Not that it changes anything, given that we care more
about whether they trigger AttributeError than what they actually do.
(Chomp the rest of the discussion, since that was all based on the
assumption that getattr was a normal function.)
> > What exactly are the semantics of getattr?
>
> Oh gods, I don't know the exact semantics of attribute look ups now!
> Something like this, I think:
>
> obj.attr (same as getattr(obj, 'attr'):
That is exactly what's weird about it. Instead of looking up the name
getattr and then calling a perfectly ordinary function, now it has to
be a magical construct of the compiler, handled right there. It is, in
fact, impossible to craft equivalent semantics in a third-party
function.
Currently, getattr() can be defined in C on top of the C API function
PyObject_GetAttr, which looks solely at the object and not the
execution context. By your proposal, getattr() can only be compiler
magic.
> > Please explain exactly what the semantics of getattr are, and exactly
> > which modules it is supposed to be able to see. Remember, it is not a
> > compiler construct or an operator. It is a function, and it lives in
> > its own module (the builtins).
>
> You seem to think that getattr being a function makes a difference. Why?
>
> Aside from the possibility that it might be shadowed or deleted from
> builtins, can you give me any examples where `obj.attr` and
> `getattr(obj. 'attr')` behave differently? Even *one* example?
That is *precisely* the possibility. That is exactly why it is magical
by your definition, and nonmagical by the current definition. At the
moment, getattr() is just a function, hasattr() is just a function. I
can do things like this:
def ga(obj, attr): return getattr(obj, attr)
Or this:
ga = getattr
Or this:
PyObject *my_getattr(PyObject *obj, PyObject *attr) {return
PyObject_GetAttr(obj, attr);}
But if it's possible to do a source code transformation from
getattr(obj, "attr") to obj.attr, then it is no longer possible to do
*ANY* of this. You can't have an alias for getattr, you can't have a
wrapper around it, you can't write your own version of it.
In some languages, this is acceptable and unsurprising, because the
getattr-like feature is actually an operator. (For instance,
JavaScript fundamentally defines obj.attr as being equivalent to
obj["attr"], so if you want dynamic lookups, you just use square
brackets.) In Python, that is simply not the case. Are you proposing
to break backward compatibility and all consistency just for the sake
of this?
> > > Not a rhetorical question: is that how it works in something like Swift,
> > > or Kotlin?
> >
> > I have no idea. I'm just asking how you intend it to work in Python.
> > If you want to cite other languages, go ahead, but I'm not assuming
> > that they already have the solution, because they are different
> > languages. Also not a rhetorical question: Is their getattr equivalent
> > actually an operator or compiler construct, rather than being a
> > regular function? Because if it is, then the entire problem doesn't
> > exist.
>
> I really don't know why you think getattr being a function makes any
> difference here. It's a builtin function, written in C, and can and does
> call the same internal C routines used by dot notation.
But if it's just a builtin function, then how is it going to know the
execution context it's supposed to look up attributes in? If
getattr(obj, "attr") can be implemented by a third party, show me how
you would write the function such that it knows which extension
methods to look for.
You keep coming back to this assumption that it has to be
fundamentally equivalent to obj.attr, but that's the exact problem -
there is no way to define a function that can know the caller's
context (barring shenanigans with sys._getframe), so it has to be
compiler magic instead, which means it is *not a function any more*.
> > > > And what about this?
> > > >
> > > > f = functools.partial(getattr, stuff)
> > > > f("in_order")
> > > >
> > > > NOW which extension methods should apply? Those registered here? Those
> > > > registered in the builtins? Those registered in functools?
> > >
> > > partial is just a wrapper around its function argument, so that should
> > > behave *exactly* the same as `getattr(stuff, 'in_order')`.
> >
> > So if it behaves exactly the same way that getattr would, then is it
> > exactly the same as fetch2 and fetch4? If not, how is it different?
>
> Okay, let's look at the partial object:
>
>
> >>> import functools
> >>> f = functools.partial(getattr, [10, 20])
> >>> f('index')(20)
> 1
>
> Partial objects like f don't seem to have anything like a __globals__
> attribute that allow me to tell what the execution context would be. I
> *think* that for Python functions (def or lambda) they just inherit the
> execution context from the function. For builtins, I'm not sure. I
> presume their execution context will be the current scope.
Do you see the problem, then? The partial object has to somehow pass
along the execution context. Otherwise, functools.partial(getattr,
obj)("attr") won't behave identically to obj.attr.
> Right now, I've already spent multiple hours on these posts, and I have
> more important things to do now than argue about the minutia of
> partial's behaviour. But if you wanted to do an experiment, you could do
> something like comparing the behaviour of:
>
>
> # module A.py
> f = lambda: globals()
> g = partial(globals)
>
> # module B.py
> from A import f, g
> f()
> g()
>
>
> and see whether f and g behave identically. I expect that f would return
> A's globals regardless of where it was called from, but I'm not sure
> what g would do. It might very well return the globals of the calling
> site.
Correct on both counts - f() naturally has to return the globals from
where it is, and g() uses the calling site.
Whichever way you do it, somewhere, you're going to have a disconnect
between obj.attr and the various dynamic ways of looking it up. It is
going to happen. So why are you fighting so hard for getattr() to
become magical in this way?
> > What about other functions implemented in C? If I write a C module
> > that calls PyObject_GetAttr, does it behave as if dot notation were
> > used in the module that called me, or does it use my module's
> > extension methods?
>
> That depends. If you write a C module that calls PyObject_GetAttr right
> now, is that *exactly* the same as dot notation in pure-Python code?
>
> The documentation is terse:
>
> https://docs.python.org/3.8/c-api/object.html#c.PyObject_GetAttr
>
> but if it is correct that it is precisely equivalent to dot syntax, then
> the same rules will apply. Has the current module opted in? If so, then
> does the class have an extension method of the requested name?
The "current module", logically, would be the extension module.
> Same applies to code objects evaluated without a function, or whatever
> other exotic corner cases you think of. Whatever you think of, the
> answer will always be the same:
>
> - if the execution context is a module that has opted to use
> extension methods, then attribute access will see extension methods;
>
> - if not, then it won't.
>
> If you think of a scenario where you are executing code where there is
> no module scope at all, and all global lookups fail, then "no module"
> cannot opt in to use extension methods and so the code won't see them.
>
> If you can think of a scenario where you are executing code where there
> are multiple module scopes that fight for supremacy using their two
> weapons of fear, surprise and a fanatical devotion to the Pope, then the
> winner will determine the result.
>
> *wink*
Multiple scopes can definitely be plausible, but there'll just have to
be some definition for which one wins. My guess would be that the
function object wins, but if not, the code object should know its own
context, and if not that, then there is a null context with no
extension methods. But that's nothing more than a guess, and there's
no rush on pinning that part down precisely.
I am wholeheartedly against this proposal if it means that getattr has
to become magical. If, however, getattr simply ignores extension
methods, and the ONLY thing changed is the way that dot lookup is
done, I would be more able to see its value. (Though not so much that
I'd actually be using this myself. I don't think it'd benefit any of
my current projects. But I can see its potential value for the
language.)
ChrisA
_______________________________________________
Python-ideas mailing list -- python-ideas@python.org
To unsubscribe send an email to python-ideas-le...@python.org
https://mail.python.org/mailman3/lists/python-ideas.python.org/
Message archived at
https://mail.python.org/archives/list/python-ideas@python.org/message/OU2XEX3ARHHRWSCTCMSBGRDEUKGTYSIH/
Code of Conduct: http://python.org/psf/codeofconduct/
