Re: [Python-ideas] Restore the __members__ behavior to python3 for C extension writers
On 2017-06-18 20:10, Barry Scott wrote: > What is the way to tell python about 'value' in the python3 world? Implement a __dir__ method. This should call the superclass' (e.g. object's) __dir__ and add whatever it is you want to add to the list. In general I'd recommend using properties for this kind of thing rather than messing with __getattr__, __setattr__ and __dir__, especially in pure Python. I'm no expert on the C API, but I think this should be possible by setting PyTypeObject.tp_getset https://docs.python.org/3/c-api/typeobj.html#c.PyTypeObject.tp_getset -- Thomas ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] ImportError raised for a circular import
On 19 June 2017 at 04:47, Mahmoud Hashemi wrote: > Barry, that kind of circular import is actually fine in many (if not most) > cases. Modules are immediately created and importable, thenincrementally > populated. The problem arises when you try to do something with contents of > the module that have not been populated, usually manifesting in the > AttributeError above. > > If you'd like to test this yourself, I've made a tiny demo with a little bit > of documentation: > https://gist.github.com/mahmoud/32fd056a3d4d1cd03a4e8aeff6b5ee70 > > Long story short, circular imports can be a code smell, but they're by no > means universally an error condition. :) Indeed, and this is why we've been begrudgingly giving ground and ensuring that the cases that *can* be made to work actually succeed in practice. We'll let code linters complain about those cycles, rather than having the interpreter continue to get confused :) However, permitting resolvable circular imports means that for the remaining cases that are genuinely irresolvably broken, any custom detection logic needs to live in some combination of the module attribute lookup code and the IMPORT_FROM opcode implementation, rather than being able to isolate the changes to the import machinery itself. Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Restore the __members__ behavior to python3 for C extension writers
On 19 June 2017 at 04:10, Barry Scott wrote:
> The code does this:
>
> Py::Object getattro( const Py::String &name_ )
> {
> std::string name( name_.as_std_string( "utf-8" ) );
>
> if( name == "value" )
> {
> return m_value;
> }
> else
> {
> return genericGetAttro( name_ );
> }
> }
>
> Where getattro is called (indirectly) from tp_getattro.
>
> In the python 2 I can tell python that 'value' exists because I provide a
> value of __members__.
>
> What is the way to tell python about 'value' in the python3 world?
OK, I think I may understand the confusion now.
As Thomas noted, the preferred way of informing Python of data
attributes for types implemented in C is to ask the interpreter to
automatically create the appropriate descriptor objects by setting the
`tp_members` slot on the C level *type*, rather than setting
`__members__` on the instance:
https://docs.python.org/3/c-api/typeobj.html#c.PyTypeObject.tp_members
That approach also works for new-style classes in Python 2:
https://docs.python.org/2/c-api/typeobj.html#c.PyTypeObject.tp_members
I believe this is actually an old-/new-style class difference, so the
relevant Python 3 change is the fact that the old-style approach
simply isn't available any more.
So if you use tp_members and tp_getset to request the creation of
suitable descriptors, then the interpreter will automatically take
care of populating the results of `dir()` correctly. However, if
you're genuinely dynamically adding attributes in `__getattr__`, then
you're going to need to add code to report them from `__dir__` as
well.
Cheers,
Nick.
--
Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-ideas] the error that raises an AttributeError should be passed to __getattr__
Hi everyone, A while back I had a conversation with some folks over on python-list. I was having issues implementing error handling of `AttributeError`s using `__getattr__`. My problem is that it is currently impossible for a `__getattr__` in Python to know which method raised the `AttributeError` that was caught by `__getattr__` if there are nested methods. For example, we cannot tell the difference between `A.x` not existing (which would raise an AttributeError) and some attribute inside `A.x` not existing (which also raises an AttributeError). This is evident from the stack trace that gets printed to screen, but `__getattr__` doesn't get that stack trace. I propose that the error that triggers an `AttributeError` should get passed to `__getattr__` (if `__getattr__` exists of course). Then, when handling errors, users could dig into the problematic error if they so desire. What do you think? Best, Jason ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] socket module: plain stuples vs named tuples
+1 ;-) for example to get IP address of all interfaces we have to use the third (indexed as 2) member of "something" it is much more beatiful to get ip_address: *import *socket *for *ip *in *socket.gethostbyname_ex(socket.gethostname())[2]: print(ip) BR, George 2017-06-13 22:13 GMT+02:00 Thomas Güttler : > AFAIK the socket module returns plain tuples in Python3: > > https://docs.python.org/3/library/socket.html > > Why not use named tuples? > > Regards, > Thomas Güttler > > -- > I am looking for feedback for my personal programming guidelines: > https://github.com/guettli/programming-guidelines > ___ > Python-ideas mailing list > Python-ideas@python.org > https://mail.python.org/mailman/listinfo/python-ideas > Code of Conduct: http://python.org/psf/codeofconduct/ > ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] socket module: plain stuples vs named tuples
Hi,
2017-06-13 22:13 GMT+02:00 Thomas Güttler :
> AFAIK the socket module returns plain tuples in Python3:
>
> https://docs.python.org/3/library/socket.html
>
> Why not use named tuples?
For technical reasons: the socket module is mostly implemented in the
C language, and define a "named tuple" in C requires to implement a
"sequence" time which requires much more code than creating a tuple.
In short, create a tuple is as simple as Py_BuildValue("OO", item1, item2).
Creating a "sequence" type requires something like 50 lines of code,
maybe more, I don't know exactly.
Victor
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] socket module: plain stuples vs named tuples
There are examples in timemodule.c which went through a similar conversion
from plain tuples to (sort-of) named tuples. I agree that upgrading the
tuples returned by the socket module to named tuples would be nice, but
it's a low priority project. Maybe someone interested can create a PR?
(First open an issue stating that you're interested; point to this email
from me to prevent that some other core dev just closes it again.)
On Mon, Jun 19, 2017 at 2:24 PM, Victor Stinner
wrote:
> Hi,
>
> 2017-06-13 22:13 GMT+02:00 Thomas Güttler :
> > AFAIK the socket module returns plain tuples in Python3:
> >
> > https://docs.python.org/3/library/socket.html
> >
> > Why not use named tuples?
>
> For technical reasons: the socket module is mostly implemented in the
> C language, and define a "named tuple" in C requires to implement a
> "sequence" time which requires much more code than creating a tuple.
>
> In short, create a tuple is as simple as Py_BuildValue("OO", item1, item2).
>
> Creating a "sequence" type requires something like 50 lines of code,
> maybe more, I don't know exactly.
>
> Victor
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
--
--Guido van Rossum (python.org/~guido)
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] socket module: plain stuples vs named tuples
Oh, about the cost of writing C code, we started to enhance the socket
module in socket.py but keep the C code unchanged. I am thinking to the
support of enums. Some C functions are wrapped in Python.
Victor
Le 19 juin 2017 11:59 PM, "Guido van Rossum" a écrit :
> There are examples in timemodule.c which went through a similar conversion
> from plain tuples to (sort-of) named tuples. I agree that upgrading the
> tuples returned by the socket module to named tuples would be nice, but
> it's a low priority project. Maybe someone interested can create a PR?
> (First open an issue stating that you're interested; point to this email
> from me to prevent that some other core dev just closes it again.)
>
> On Mon, Jun 19, 2017 at 2:24 PM, Victor Stinner
> wrote:
>
>> Hi,
>>
>> 2017-06-13 22:13 GMT+02:00 Thomas Güttler :
>> > AFAIK the socket module returns plain tuples in Python3:
>> >
>> > https://docs.python.org/3/library/socket.html
>> >
>> > Why not use named tuples?
>>
>> For technical reasons: the socket module is mostly implemented in the
>> C language, and define a "named tuple" in C requires to implement a
>> "sequence" time which requires much more code than creating a tuple.
>>
>> In short, create a tuple is as simple as Py_BuildValue("OO", item1,
>> item2).
>>
>> Creating a "sequence" type requires something like 50 lines of code,
>> maybe more, I don't know exactly.
>>
>> Victor
>> ___
>> Python-ideas mailing list
>> Python-ideas@python.org
>> https://mail.python.org/mailman/listinfo/python-ideas
>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] Run length encoding
On 19/06/17 02:47, David Mertz wrote: As an only semi-joke, I have created a module on GH that meets the needs of this discussion (using the spelling I think are most elegant): https://github.com/DavidMertz/RLE It's a shame you have to build that list when encoding. I tried to work out a way to get the number of items in an iterable without having to capture all the values (on the understanding that if the iterable is already an iterator, it would be consumed). The best I came up with so far (not general purpose, but it works in this scenario) is: from iterator import groupby from operator import countOf def rle_encode(it): return ((k, countOf(g, k)) for k, g in groupby(it)) In your test code, this speeds things up quite a bit over building the list, but that's presumably only because both groupby() and countOf() will use the standard class comparison operator methods which in the case of ints will short-circuit with a C-level pointer comparison first. For user-defined classes with complicated comparison methods, getting the length of the group by comparing the items will probably be worse. Is there a better way of implementing a general-purpose "ilen()"? I tried a couple of other things, but they all required at least one lambda function and slowed things down by about 50% compared to the list-building version. (I agree this is sort of a joke, but it's still an interesting puzzle ...). Regards, E. ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] the error that raises an AttributeError should be passed to __getattr__
On Mon, Jun 19, 2017 at 04:06:56PM -0500, Jason Maldonis wrote:
> Hi everyone,
>
> A while back I had a conversation with some folks over on python-list. I
> was having issues implementing error handling of `AttributeError`s using
> `__getattr__`.
[...]
> For example, we cannot tell the difference between `A.x` not existing
> (which would raise an AttributeError) and some attribute inside `A.x` not
> existing (which also raises an AttributeError).
I didn't understand what you were talking about here at first. If you
write something like
A.x.y
where y doesn't exist, it's A.x.__getattr__ that is called, not
A.__getattr__. But I went and looked at the thread in Python-Ideas and
discovered that you're talking about the case where A.x is a descriptor,
not an ordinary attribute, and the descriptor leaks AttributeError.
Apparently you heavily use properties, and __getattr__, and find that
the two don't interact well together when the property getters and
setters themselves raise AttributeError. I think that's relevant
information that helps explain the problem you are hoping to fix.
So I *think* this demonstrates the problem:
class A(object):
eggs = "text"
def __getattr__(self, name):
if name == 'cheese':
return "cheddar"
raise AttributeError('%s missing' % name)
@property
def spam(self):
return self.eggs.uper() # Oops.
a = A()
a.spam
Which gives us
Traceback (most recent call last):
File "", line 1, in
File "", line 6, in __getattr__
AttributeError: spam missing
But you go on to say that:
> This is evident from the
> stack trace that gets printed to screen, but `__getattr__` doesn't get that
> stack trace.
I can't reproduce that! As you can see from the above, the stack trace
doesn't say anything about the actual missing attribute 'uper'.
So I must admit I don't actually understand the problem you are hoping
to solve. It seems to be different from my understanding of it.
> I propose that the error that triggers an `AttributeError` should get
> passed to `__getattr__` (if `__getattr__` exists of course). Then, when
> handling errors, users could dig into the problematic error if they so
> desire.
What precisely will be passed to __getattr__? The exception instance?
The full traceback object? The name of the missing attribute? Something
else? It is hard to really judge this proposal without more detail.
I think the most natural thing to pass would be the exception instance,
but AttributeError instances don't record the missing attribute name
directly (as far as I can tell). Given:
try:
''.foo
except AttributeError as e:
print(e.???)
there's nothing in e we can inspect to get the name of the missing
exception, 'foo'. (As far as I can see.) We must parse the error message
itself, which we really shouldn't do, because the error message is not
part of the exception API and could change at any time.
So... what precisely should be passed to __getattr__, and what exactly
are you going to do with it?
Having said that, there's another problem: adding this feature (whatever
it actually is) to __getattr__ will break every existing class that uses
__getattr__. The problem is that everyone who writes a __getattr__
method writes it like this:
def __getattr__(self, name):
not:
def __getattr__(self, name, error):
so the class will break when the method receives two arguments
(excluding self) but only has one parameter.
*If* we go down this track, it would probably require a __future__
import for at least one release, probably more:
- in 3.7, use `from __future__ import extra_getattr_argument`
- in 3.8, deprecate the single-argument form of __getattr__
- in 3.9 or 4.0 no longer require the __future__ import.
That's a fairly long and heavy process, and will be quite annoying to
those writing cross-version code using __getattr__, but it can be done.
But only if it actually helps solve the problem. I'm not convinced that
it does. It comes down to the question of what this second argument is,
and how do you expect to use it?
--
Steve
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] the error that raises an AttributeError should be passed to __getattr__
On Tue, Jun 20, 2017 at 10:18 AM, Steven D'Aprano wrote:
> Apparently you heavily use properties, and __getattr__, and find that
> the two don't interact well together when the property getters and
> setters themselves raise AttributeError. I think that's relevant
> information that helps explain the problem you are hoping to fix.
>
> So I *think* this demonstrates the problem:
>
> class A(object):
> eggs = "text"
> def __getattr__(self, name):
> if name == 'cheese':
> return "cheddar"
> raise AttributeError('%s missing' % name)
> @property
> def spam(self):
> return self.eggs.uper() # Oops.
I'm quoting Steven's post, but I'm addressing the OP.
One good solution to this is a "guard point" around your property functions.
def noleak(*exc):
def deco(func):
@functools.wraps(func)
def wrapper(*a, **kw):
try: return func(*a, **kw)
except exc: raise RuntimeError
return wrapper
return deco
@property
@noleak(AttributeError)
def spam(self):
return self.eggs.uper()
In fact, you could make this into a self-wrapping system if you like:
def property(func, *, _=property):
return _(noleak(AttributeError)(func))
Now, all your @property functions will be guarded: any AttributeErrors
they raise will actually bubble as RuntimeErrors instead.
Making this work with setters and deleters is left as an exercise for
the reader.
ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] the error that raises an AttributeError should be passed to __getattr__
On Tue, Jun 20, 2017 at 10:18 AM, Steven D'Aprano wrote: > Having said that, there's another problem: adding this feature (whatever > it actually is) to __getattr__ will break every existing class that uses > __getattr__. The problem is that everyone who writes a __getattr__ > method writes it like this: > > def __getattr__(self, name): > > not: > > def __getattr__(self, name, error): > > so the class will break when the method receives two arguments > (excluding self) but only has one parameter. Why not just write cross-version-compatible code as def __getattr__(self, name, error=None): ? Is there something special about getattr? ChrisA ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] socket module: plain stuples vs named tuples
Namedtuple in Python make startup time slow.
So I'm very conservative to convert tuple to namedtuple in Python.
INADA Naoki
On Tue, Jun 20, 2017 at 7:27 AM, Victor Stinner
wrote:
> Oh, about the cost of writing C code, we started to enhance the socket
> module in socket.py but keep the C code unchanged. I am thinking to the
> support of enums. Some C functions are wrapped in Python.
>
> Victor
>
> Le 19 juin 2017 11:59 PM, "Guido van Rossum" a écrit :
>>
>> There are examples in timemodule.c which went through a similar conversion
>> from plain tuples to (sort-of) named tuples. I agree that upgrading the
>> tuples returned by the socket module to named tuples would be nice, but it's
>> a low priority project. Maybe someone interested can create a PR? (First
>> open an issue stating that you're interested; point to this email from me to
>> prevent that some other core dev just closes it again.)
>>
>> On Mon, Jun 19, 2017 at 2:24 PM, Victor Stinner
>> wrote:
>>>
>>> Hi,
>>>
>>> 2017-06-13 22:13 GMT+02:00 Thomas Güttler :
>>> > AFAIK the socket module returns plain tuples in Python3:
>>> >
>>> > https://docs.python.org/3/library/socket.html
>>> >
>>> > Why not use named tuples?
>>>
>>> For technical reasons: the socket module is mostly implemented in the
>>> C language, and define a "named tuple" in C requires to implement a
>>> "sequence" time which requires much more code than creating a tuple.
>>>
>>> In short, create a tuple is as simple as Py_BuildValue("OO", item1,
>>> item2).
>>>
>>> Creating a "sequence" type requires something like 50 lines of code,
>>> maybe more, I don't know exactly.
>>>
>>> Victor
>>> ___
>>> Python-ideas mailing list
>>> Python-ideas@python.org
>>> https://mail.python.org/mailman/listinfo/python-ideas
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>
>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>
>
> ___
> Python-ideas mailing list
> Python-ideas@python.org
> https://mail.python.org/mailman/listinfo/python-ideas
> Code of Conduct: http://python.org/psf/codeofconduct/
>
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] the error that raises an AttributeError should be passed to __getattr__
On 06/19/2017 06:31 PM, Chris Angelico wrote: On Tue, Jun 20, 2017 at 10:18 AM, Steven D'Aprano wrote: Having said that, there's another problem: adding this feature (whatever it actually is) to __getattr__ will break every existing class that uses __getattr__. The problem is that everyone who writes a __getattr__ method writes it like this: def __getattr__(self, name): not: def __getattr__(self, name, error): so the class will break when the method receives two arguments (excluding self) but only has one parameter. Why not just write cross-version-compatible code as def __getattr__(self, name, error=None): ? Is there something special about getattr? The point was existing code would fail until that change was made. And a lot of existing code uses __getattr__. -- ~Ethan~ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] the error that raises an AttributeError should be passed to __getattr__
On Tue, Jun 20, 2017 at 11:31:34AM +1000, Chris Angelico wrote: > Why not just write cross-version-compatible code as > > def __getattr__(self, name, error=None): > > ? Is there something special about getattr? You've still got to write it in the first place. That's a pain, especially since (1) it doesn't do you any good before 3.7 if not later, and (2) even if this error parameter is useful (which is yet to be established), it's a pretty specialised use. Most of the time, you already know the name that failed (its the one being looked up). Perhaps a better approach is to prevent descriptors from leaking AttributeError in the first place? Change the protocol so that if descriptor.__get__ raises AttributeError, it is caught and re-raised as RuntimeError, similar to StopIteration and generators. Or maybe we decide that it's actually a feature, not a problem, for an AttributeError inside self.attr.__get__ to look like self.attr is missing. I don't know. (Also everything I said applies to __setattr__ and __delattr__ as well.) -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] the error that raises an AttributeError should be passed to __getattr__
On 06/19/2017 07:26 PM, Steven D'Aprano wrote: Or maybe we decide that it's actually a feature, not a problem, for an AttributeError inside self.attr.__get__ to look like self.attr is missing. It's a feature. It's why Enum classes can have members named 'value' and 'name'. -- ~Ethan~ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] the error that raises an AttributeError should be passed to __getattr__
On Mon, Jun 19, 2017 at 07:36:09PM -0700, Ethan Furman wrote: > On 06/19/2017 07:26 PM, Steven D'Aprano wrote: > > >Or maybe we decide that it's actually a feature, not a problem, for an > >AttributeError inside self.attr.__get__ to look like self.attr is > >missing. > > It's a feature. It's why Enum classes can have members named 'value' and > 'name'. Can you explain further? What's special about value and name? -- Steve ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] socket module: plain stuples vs named tuples
On 20 June 2017 at 12:05, INADA Naoki wrote: > Namedtuple in Python make startup time slow. > So I'm very conservative to convert tuple to namedtuple in Python. Aye, I don't think a Python level wrapper would be the right way to go here - while namedtuple is designed to be as cheap as normal tuples in *use*, the same can't be said for the impact on startup time. As context for anyone not familiar with the time module precedent that Guido mentioned, we have a C level `PyStructSequence` that provides some of the most essential namedtuple features, but not all of them: https://github.com/python/cpython/blob/master/Objects/structseq.c So there's potentially a case to be made for: 1. Including the struct sequence header from "Python.h" and making it part of the stable ABI 2. Documenting it in the C API reference The main argument against doing so is that the initialisation API for it is pretty weird by the standards of the rest of the Python C API - it's mainly designed for use as a C level type *factory*, rather than intended to be used directly. That's also why there isn't a Python level API for it - it's designed around accepting C level structs as inputs, which doesn't translate well to pure Python code (whereas the collections.namedtuple API works well in pure Python code, but doesn't translate well to C code). Cheers, Nick. -- Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] the error that raises an AttributeError should be passed to __getattr__
On Tue, Jun 20, 2017 at 12:26 PM, Steven D'Aprano wrote:
> On Tue, Jun 20, 2017 at 11:31:34AM +1000, Chris Angelico wrote:
>
>> Why not just write cross-version-compatible code as
>>
>> def __getattr__(self, name, error=None):
>>
>> ? Is there something special about getattr?
>
> You've still got to write it in the first place. That's a pain,
> especially since (1) it doesn't do you any good before 3.7 if not later,
> and (2) even if this error parameter is useful (which is yet to be
> established), it's a pretty specialised use. Most of the time, you
> already know the name that failed (its the one being looked up).
Gotcha, yep. I was just confused by your two-parter that made it look
like it would be hard (or impossible) to write code that would work on
both 3.6 and the new protocol.
> Perhaps a better approach is to prevent descriptors from leaking
> AttributeError in the first place? Change the protocol so that if
> descriptor.__get__ raises AttributeError, it is caught and re-raised as
> RuntimeError, similar to StopIteration and generators.
This can't be done globally, because that's how a descriptor can be
made conditional (it raises AttributeError to say "this attribute does
not, in fact, exist"). But it's easy enough - and safe enough - to do
it just for your own module, where you know in advance that any
AttributeError is a leak. The way generators and StopIteration
interact was more easily fixed, because generators have two legitimate
ways to emit data (yield and return), but there's no easy way for a
magic method to say "I don't have anything to return" other than an
exception.
Well, that's not strictly true. In JavaScript, they don't raise
StopIteration from iterators - they always return a pair of values
("done" and the actual value, where "done" is either false for a yield
or true for a StopIteration). That complicates the normal case but it
does make the unusual case a bit easier. Also, it's utterly and
fundamentally incompatible with the current system, so it'd have to be
a brand new competing protocol.
ChrisA
___
Python-ideas mailing list
Python-ideas@python.org
https://mail.python.org/mailman/listinfo/python-ideas
Code of Conduct: http://python.org/psf/codeofconduct/
Re: [Python-ideas] the error that raises an AttributeError should be passed to __getattr__
First, I apologize for the poor post. Your corrections were exactly
correct: This is only relevant in the context of properties/descriptors,
and the property swallows the error message and it isn't printed to
screen. I should not be typing without testing.
> So... what precisely should be passed to __getattr__, and what exactly are
you going to do with it?
I'd say the error instance that caused __getattr__ to be raised should be
passed into __getattr__, but I don't have a strong opinion on that. I'll
assume that's true for the rest of this post, however.
To clarify my mistakes in my first post, your example illustrates what I
wanted to show:
class A(object):
eggs = "text"
def __getattr__(self, name):
if name == 'cheese':
return "cheddar"
raise AttributeError('%s missing' % name)
@property
def spam(self):
return self.eggs.uper() # Oops.
a = A()
a.spam
Traceback (most recent call last):
File "", line 1, in
File "", line 6, in __getattr__
AttributeError: spam missing
This swallows the AttributeError from `eggs.uper()` and it isn't available.
Even if it were available, I see your point that it may not be especially
useful. In one iteration of my code I was looking through the stack trace
using the traceback module to find the error I wanted, but I quickly
decided that was a bad idea because I couldn't reliably find the error.
However, with the full error, it would (I think) be trivial to find the
relevant error in the stack trace. With the full stack trace, I would hope
that you could properly do any error handling you wanted.
However, if the error was available in __getattr__, we could at least
`raise from` so that the error isn't completely swallowed. I.e. your
example would be slightly modified like this:
class A(object):
eggs = "text"
def __getattr__(self, name, error):
if name == 'cheese':
return "cheddar"
raise AttributeError('%s missing' % name) from error
...
which I think is useful.
> That's a fairly long and heavy process, and will be quite annoying to
> those writing cross-version code using __getattr__, but it can be done.
This is another thing I hadn't considered, and I have no problem saying
this non-backwards-compatible-change just isn't worth it. Maybe if there
are multiple small updates to error handling it could be worth it (which I
believe I read was something the devs care quite a bit about atm), but I
don't think that this change is a huge deal.
> One good solution to this is a "guard point" around your property
functions.
I am using a very similar decorator (modified from the one you gave me a
month or so ago) in my code now, and functionally it works great. There is
something that feels a bit off / like a hack to me though, and maybe it's
that I am simply "renaming" the error from one I can't handle (due to name
conflicts) to one I can. But the modification is just name change -- if I
can handle the RuntimeError correctly, I feel like I should have just been
able to handle the original AttributeError correctly (because in practice
they should be raising an error in response to the exact same problem).
That said, your decorator works great and gives me the functionality I
needed.
> > It's a feature. It's why Enum classes can have members named 'value'
and
> > 'name'.
> Can you explain further? What's special about value and name?
I'm also very curious about this. I've read the Enum code a few times but
it hasn't clicked yet.
> > Perhaps a better approach is to prevent descriptors from leaking
> > AttributeError in the first place? Change the protocol so that if
> > descriptor.__get__ raises AttributeError, it is caught and re-raised as
> > RuntimeError, similar to StopIteration and generators.
> This can't be done globally, because that's how a descriptor can be
> made conditional (it raises AttributeError to say "this attribute does
> not, in fact, exist"). But it's easy enough - and safe enough - to do
> it just for your own module, where you know in advance that any
> AttributeError is a leak.
I have a pretty large codebase with a few different functionalities, so I'm
hesitant to say "any AttributeError is a leak" in an object that might
affect / be affected by other packages/functionalities. One other thing I
really like about the `noleak` decorator is that I can change the
AttributeError to a custom MyCustomError, which allows me to handle
MyCustomError precisely where it should be handled. Also, maybe I'm just
not fully understanding the locality of re-raising another error from
descriptor.__get__ because I haven't completely wrapped my head around it
yet.
On Mon, Jun 19, 2017 at 10:10 PM, Chris Angelico wrote:
> On Tue, Jun 20, 2017 at 12:26 PM, Steven D'Aprano
> wrote:
> > On Tue, Jun 20, 2017 at 11:31:34AM +1000, Chris Angelico wrote:
> >
> >> Why not just write cross-version-compatible code as
> >>
> >> def __getattr__(self, name, error=None):
> >>
> >> ? Is there
Re: [Python-ideas] the error that raises an AttributeError should be passed to __getattr__
On 06/19/2017 07:44 PM, Steven D'Aprano wrote: On Mon, Jun 19, 2017 at 07:36:09PM -0700, Ethan Furman wrote: On 06/19/2017 07:26 PM, Steven D'Aprano wrote: Or maybe we decide that it's actually a feature, not a problem, for an AttributeError inside self.attr.__get__ to look like self.attr is missing. It's a feature. It's why Enum classes can have members named 'value' and 'name'. Can you explain further? What's special about value and name? value and name are attributes of every Enum member; to be specific, they are descriptors, and so live in the class namespace. Enum members also live in the class namespace, so how do you get both a member name value and the value descriptor to both live in the class namespace? Easy. ;) Have the "value" and "name" descriptors check to see if they are being called on an instance, or an the class. If called on an instance they behave normally, returning the "name" or "value" data; but if called on the class the descriptor raises AttributeError, which causes Python to try the Enum class' __getattr__ method, which can find the member and return it... or raise AttributeError again if there is no "name" or "value" member. Here's the docstring from the types.DynamicClassAttribute in question: class DynamicClassAttribute: """Route attribute access on a class to __getattr__. This is a descriptor, used to define attributes that act differently when accessed through an instance and through a class. Instance access remains normal, but access to an attribute through a class will be routed to the class's __getattr__ method; this is done by raising AttributeError. This allows one to have properties active on an instance, and have virtual attributes on the class with the same name (see Enum for an example). """ -- ~Ethan~ ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
