Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
On Thu, Jan 17, 2019, 22:11 Steve Dower For everyone who managed to reply *hours* after Eryk Sun posted the > correct answer and still get it wrong, here it is again in full. > > As a bonus, here's a link to the place where this answer appears in the > documentation: > https://docs.python.org/3/library/ctypes.html#ctypes.py_object Eryk's answer is actually much more useful than the documentation. I've read that documentation many times, but always decided not to use py_object because I couldn't figure out what it would actually do... (I still probably won't use it because IME by the time I'm using ctypes and PyObject* together I usually need manual control over refcounts, but it's nice to know what it actually does.) -n ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
On Thu, Jan 17, 2019 at 11:37:13AM +0100, Antoine Pitrou wrote: I said: > > The id() function is documented as returning an abstract ID number. In > > CPython, that happens to have been implemented as the address of the > > object. > > > > I understand that the only way to pass the address of an object to > > ctypes is to use that id. Is that intentional? Antoine: > Can you explain in detail what you're doing? Code-wise, I'm not doing anything with ctypes. Language-wise, I'm trying to get a definitive answer of whether or not id() returning the address of the object should be a guaranteed feature or not. Across the entire Python ecosystem, no it isn't, as Jython and IronPython return consecutive integers. But should we consider it an intentional part of the CPython API? There are developers who insist that when it comes to CPython, id() returning the object address is an intentional feature that they can and do rely on, because (so I was told by one of them) that using id() is the only way to get the address of an object from pure-Python. According to this claim, using id() to get the address for use in ctypes is the correct and only way to do it, and this is a deliberate design choice by the core devs rather than an accident of the implementation. So long as you know you are using CPython, this is (so I was told) completely safe. In the grand scheme of things this may be a pretty minor issue. But I suspect that it could be a pain point for implementations like PyPy that support both objects that move and a ctypes emulation. -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
On Fri, 18 Jan 2019 at 09:52, Steven D'Aprano wrote: > Code-wise, I'm not doing anything with ctypes. > > Language-wise, I'm trying to get a definitive answer of whether or not > id() returning the address of the object should be a guaranteed feature > or not. > > Across the entire Python ecosystem, no it isn't, as Jython and > IronPython return consecutive integers. But should we consider it an > intentional part of the CPython API? > > There are developers who insist that when it comes to CPython, id() > returning the object address is an intentional feature that they can and > do rely on, because (so I was told by one of them) that using id() is > the only way to get the address of an object from pure-Python. > > According to this claim, using id() to get the address for use in ctypes > is the correct and only way to do it, and this is a deliberate design > choice by the core devs rather than an accident of the implementation. > So long as you know you are using CPython, this is (so I was told) > completely safe. > > In the grand scheme of things this may be a pretty minor issue. But I > suspect that it could be a pain point for implementations like PyPy that > support both objects that move and a ctypes emulation. As per Eryk Sun's reply, the "correct" way to get an object address is by using ctypes.py_object. Supporting py_object may be a pain point for other implementations that emulate ctypes, but then again, so is supporting the whole CPython C API (which is where the py_object type is needed, so it's basically the same problem). So to answer your question, I'd say that no, id() returning the object address is not, and should not be, a guaranteed aspect of CPython, and the motivating issue of ctypes is solved within ctypes itself by using py_object. Paul ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
On Thu, Jan 17, 2019 at 04:48:38PM -0800, Gregory P. Smith wrote: > I've heard that libraries using ctypes, cffi, or cython code of various > sorts in the real world wild today does abuse the unfortunate side effect > of CPython's implementation of id(). I don't have specific instances of > this in mind but trust what I've heard: that it is happening. Indeed -- I've been told by one developer in no uncertain terms that using id() in this fashion is the only way to get the address of an object for use in ctypes. I don't know enough about ctypes to judge whether that is correct or not. The sample code I've been shown is this: pointer_to_obj = id(obj) from_deref = ctypes.cast(pointer_to_obj, ctypes.py_object).value from_deref is obj # True > id() should never be considered to be the PyObject*. In as much as code > shouldn't assume it is running on top of a specific CPython implementation. > If there is a _need_ to get a pointer to a C struct handle referencing a > CPython C API PyObject, we should make an explicit API for that rather than > the id() hack. That way code can be explicit about its need, and code that > is just doing a funky form of identity tracking without using is and is not > can continue using id() without triggering regressive behavior on VMs that > don't have a CPython compatible PyObject under the hood by default. +1 to all of this. -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
On Fri, Jan 18, 2019 at 1:51 AM Steven D'Aprano wrote: > Across the entire Python ecosystem, no it isn't, as Jython and > IronPython return consecutive integers. But should we consider it an > intentional part of the CPython API? It's always worked, there's substantial code in the wild that depends on it, and AFAICT it doesn't cause any real harm, so to me it seems like the only possible conclusion is that CPython will continue to guarantee this. For this argument I don't think it matters whether it was originally intentional, or whether there's some other alternative that people could use in theory. -n -- Nathaniel J. Smith -- https://vorpus.org ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
Thanks for the detailed answer. A further question below. On Thu, Jan 17, 2019 at 07:50:51AM -0600, eryk sun wrote: > On 1/17/19, Steven D'Aprano wrote: > > > > I understand that the only way to pass the address of an object to > > ctypes is to use that id. Is that intentional? > > It's kind of dangerous to pass an object to C without an increment of > its reference count. "Kind of dangerous?" How dangerous? If I am reading this correctly, I think you are saying that using id() in this way is never(?) correct. -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
On Fri, 18 Jan 2019 20:49:26 +1100 Steven D'Aprano wrote: > > Language-wise, I'm trying to get a definitive answer of whether or not > id() returning the address of the object should be a guaranteed feature > or not. For me, the definitive answer is "yes, it's a CPython feature". However, it's obviously not a PyPy feature, and I'm not sure about other implementations. Anything with an object model that can eliminate in-memory objects in favour of in-register values (for example using tagged pointers or type specialization + lifetime analysis) is obviously not able to hold the promise that id() returns the /address/ of the "object". That doesn't mean the CPython feature has to live forever. We may want to deprecate it at some point (though it's not obvious how to warn the user: just because you're using id() doesn't mean you're interested in the actual /address/, rather than some arbitrary unique id). > According to this claim, using id() to get the address for use in ctypes > is the correct and only way to do it I don't know why you keep repeating that. You were already explained that it's /not/ the correct and only way to get the address for use in ctypes. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
On Fri, 18 Jan 2019 00:18:17 -0800 Nathaniel Smith wrote: > On Thu, Jan 17, 2019, 22:11 Steve Dower > > For everyone who managed to reply *hours* after Eryk Sun posted the > > correct answer and still get it wrong, here it is again in full. > > > > As a bonus, here's a link to the place where this answer appears in the > > documentation: > > https://docs.python.org/3/library/ctypes.html#ctypes.py_object > > > Eryk's answer is actually much more useful than the documentation. I've > read that documentation many times, but always decided not to use py_object > because I couldn't figure out what it would actually do... +1 Needless to say, this is an opportunity to improve the documentation ;-) Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
On Fri, 18 Jan 2019 03:00:54 + MRAB wrote: > On 2019-01-18 00:48, Gregory P. Smith wrote: > > I've heard that libraries using ctypes, cffi, or cython code of various > > sorts in the real world wild today does abuse the unfortunate side > > effect of CPython's implementation of id(). I don't have specific > > instances of this in mind but trust what I've heard: that it is happening. > > > > id() should never be considered to be the PyObject*. In as much as code > > shouldn't assume it is running on top of a specific CPython implementation. > > If there is a _need_ to get a pointer to a C struct handle referencing a > > CPython C API PyObject, we should make an explicit API for that rather > > than the id() hack. That way code can be explicit about its need, and > > code that is just doing a funky form of identity tracking without using > > is and is not can continue using id() without triggering regressive > > behavior on VMs that don't have a CPython compatible PyObject under the > > hood by default. > > > > [who uses id() anyways?] > > > I use it in some of my code. > > If I want to cache some objects, I put them in a dict, using the id as > the key. If I wanted to locate an object in a cache and didn't have > id(), I'd have to do a linear search for it. Indeed. I've used it for the same purpose in the past (identity-dict). Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
On Thu, 17 Jan 2019 22:18:13 -0800 Steve Dower wrote: > I feel like I should clarify - not everyone who posted got it wrong, and > I understand there's a side discussion among those who are also > interested/participants in > https://discuss.python.org/t/demoting-the-is-operator-to-avoid-an-identity-crisis/86/ > - but there was no of acknowledgement of Eryk Sun's correct and useful > answer which I find very disappointing and a great way to discourage > contributions. > > We can, and should, do better, at least by thanking the person for their > response before running down a barely related side track. I can certainly thank Eryk for posting a much better answer than mine. Regards Antoine. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
On Thu, Jan 17, 2019 at 10:09:36PM -0800, Steve Dower wrote: > For everyone who managed to reply *hours* after Eryk Sun posted the > correct answer and still get it wrong, here it is again in full. Sorry, I'm confused by your response here. As far as I can see, nobody except Eryk Sun gave any technical details about how to correctly pass objects to ctypes, so I'm not sure what sense of "get it wrong" you mean. A couple of people offered the opinion that we ought to offer an explicit ctypes API for getting the address of an object, decoupling that functionality from id(). Do you mean "wrong" in the sense that such an API would be unnecessary, given the existing solution Eryk Sun quoted? > As a bonus, here's a link to the place where this answer appears in the > documentation: > https://docs.python.org/3/library/ctypes.html#ctypes.py_object Thanks for the link, that's useful. -- Steve ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
On 18 Jan 2019, at 11:57, Antoine Pitrou wrote: On Fri, 18 Jan 2019 03:00:54 + MRAB wrote: On 2019-01-18 00:48, Gregory P. Smith wrote: I've heard that libraries using ctypes, cffi, or cython code of various sorts in the real world wild today does abuse the unfortunate side effect of CPython's implementation of id(). I don't have specific instances of this in mind but trust what I've heard: that it is happening. id() should never be considered to be the PyObject*. In as much as code shouldn't assume it is running on top of a specific CPython implementation. If there is a _need_ to get a pointer to a C struct handle referencing a CPython C API PyObject, we should make an explicit API for that rather than the id() hack. That way code can be explicit about its need, and code that is just doing a funky form of identity tracking without using is and is not can continue using id() without triggering regressive behavior on VMs that don't have a CPython compatible PyObject under the hood by default. [who uses id() anyways?] I use it in some of my code. If I want to cache some objects, I put them in a dict, using the id as the key. If I wanted to locate an object in a cache and didn't have id(), I'd have to do a linear search for it. Indeed. I've used it for the same purpose in the past (identity-dict). Its useful in all situations where you do topology preserving transformations, for example pickling (i.e. object serialization) or a deep copy of some object structures. In these cases you need a way to record and quickly detect whether you've handled a specific object before. In Python we can do that with a dictionary that has object ids as keys. Java provides IdentityHashMap for that. Javascript provides neither, so deep-copying objects in Javascript seems to be impossible. Regards Antoine. Servus, Walter ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
On Fri, Jan 18, 2019, 5:55 AM Antoine Pitrou > > id() returning the address of the object should be a guaranteed feature > > For me, the definitive answer is "yes, it's a CPython feature". > That doesn't mean the CPython feature has to live forever. We may want > to deprecate it at some point Whenever I've taught Python (quite a bit between writing, in person, and webinars), I have been very explicit in stating that id(obj) returns some unique number for each object, and mentioned that for MANY Python objects CPython users an implementation convenience of using the memory address. Every time I've explained it I've said not to rely on that implementation detail. It's not true for small integers, for example, even in CPython. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
Oh, bracket my brain glitch on small integers. Yes, they still give id() of memory address, they just get reused, which is different. Nonetheless, I never teach id(obj) == ctypes.c_void_p.from_buffer(ctypes.py_object(b)).value ... and not only because I only learned the latter spelling from eryk sun. On Fri, Jan 18, 2019 at 10:29 AM David Mertz wrote: > On Fri, Jan 18, 2019, 5:55 AM Antoine Pitrou >> >> > id() returning the address of the object should be a guaranteed feature >> >> For me, the definitive answer is "yes, it's a CPython feature". >> That doesn't mean the CPython feature has to live forever. We may want >> to deprecate it at some point > > > Whenever I've taught Python (quite a bit between writing, in person, and > webinars), I have been very explicit in stating that id(obj) returns some > unique number for each object, and mentioned that for MANY Python objects > CPython users an implementation convenience of using the memory address. > > Every time I've explained it I've said not to rely on that implementation > detail. It's not true for small integers, for example, even in CPython. > -- Keeping medicines from the bloodstreams of the sick; food from the bellies of the hungry; books from the hands of the uneducated; technology from the underdeveloped; and putting advocates of freedom in prisons. Intellectual property is to the 21st century what the slave trade was to the 16th. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2019-01-11 - 2019-01-18) Python tracker at https://bugs.python.org/ To view or respond to any of the issues listed below, click on the issue. Do NOT respond to this message. Issues counts and deltas: open6932 ( +0) closed 40595 (+58) total 47527 (+58) Open issues with patches: 2766 Issues opened (43) == #35701: [uuid] 3.8 breaks weak references for UUIDs https://bugs.python.org/issue35701 reopened by serhiy.storchaka #35720: Memory leak in Modules/main.c:pymain_parse_cmdline_impl when u https://bugs.python.org/issue35720 opened by Lucas Cimon #35721: _UnixSubprocessTransport leaks socket pair if Popen fails https://bugs.python.org/issue35721 opened by niklasf #35722: disable_existing_loggers does not apply to the root logger https://bugs.python.org/issue35722 opened by maggyero #35723: Add "time zone index" cache to datetime objects https://bugs.python.org/issue35723 opened by p-ganssle #35724: Check for main interpreter when checking for "main" thread (fo https://bugs.python.org/issue35724 opened by eric.snow #35726: QueueHandler formating affects other handlers https://bugs.python.org/issue35726 opened by David Ruggles #35727: sys.exit() in a multiprocessing.Process does not align with Py https://bugs.python.org/issue35727 opened by chrahunt #35728: Tkinter font nametofont requires default root https://bugs.python.org/issue35728 opened by terry.reedy #35731: Modify to support multiple urls in webbrowser.open https://bugs.python.org/issue35731 opened by arlenyu #35733: isinstance(ast.Constant(value=True), ast.Num) should be False https://bugs.python.org/issue35733 opened by Anthony Sottile #35736: [xml.minidom] Missing component in table after getElementsByTa https://bugs.python.org/issue35736 opened by MiKr41 #35737: crypt AuthenticationError introduced with new Linux kernel https://bugs.python.org/issue35737 opened by icycle #35739: Enable verbose of tests during PGO build on amd64 platforms https://bugs.python.org/issue35739 opened by neyuru #35740: openssl version 1.1.1 need to be there in cpython-source-deps https://bugs.python.org/issue35740 opened by ossdev07 #35741: unittest.skipUnless(time._STRUCT_TM_ITEMS == 11, "needs tm_zon https://bugs.python.org/issue35741 opened by jianxu3 #35745: Add import statement in dataclass code snippet https://bugs.python.org/issue35745 opened by Windson Yang #35746: [ssl][CVE-2019-5010] TALOS-2018-0758 Denial of Service https://bugs.python.org/issue35746 opened by Talos #35748: urlparse library detecting wrong hostname leads to open redire https://bugs.python.org/issue35748 opened by nsonaniya2010 #35749: Rewrite asyncio signal handler https://bugs.python.org/issue35749 opened by asvetlov #35751: traceback.clear_frames manages to deadlock a background task https://bugs.python.org/issue35751 opened by tvoinarovskyi #35752: test_buffer fails on ppc64le: memoryview pack_single() is misc https://bugs.python.org/issue35752 opened by vstinner #35754: When writing/closing a closed Popen.stdin, I get OSError vs. B https://bugs.python.org/issue35754 opened by jimbo1qaz_ #35755: Remove current directory from posixpath.defpath to enhance sec https://bugs.python.org/issue35755 opened by vstinner #35756: Using `return value` in a generator function skips the returne https://bugs.python.org/issue35756 opened by Bryan Koch #35758: Disable x87 control word for MSVC ARM compiler https://bugs.python.org/issue35758 opened by Minmin.Gong #35759: inspect module does not implement introspection API for asynch https://bugs.python.org/issue35759 opened by tkren #35760: test_asyncio: test_async_gen_asyncio_gc_aclose_09() race condi https://bugs.python.org/issue35760 opened by vstinner #35761: Allow dataclasses to be updated in place https://bugs.python.org/issue35761 opened by theophile #35762: subprocess.Popen with universal_newlines and nonblocking strea https://bugs.python.org/issue35762 opened by sambayer #35763: IDLE calltips: make positional note less obtrusive https://bugs.python.org/issue35763 opened by terry.reedy #35764: IDLE: revise calltip doc https://bugs.python.org/issue35764 opened by terry.reedy #35765: Document references object x but doesn't show it in the exampl https://bugs.python.org/issue35765 opened by Patrick Rice #35766: Merge typed_ast back into CPython https://bugs.python.org/issue35766 opened by gvanrossum #35767: unittest loader doesn't work with partial test functions https://bugs.python.org/issue35767 opened by fried #35768: IDLE: Auto measure font fixed pitch characteristics https://bugs.python.org/issue35768 opened by terry.reedy #35769: IDLE: change new file name from ''Untitled" to "untitled" https://bugs.python.org/issue35769 opened by terry.reedy #35770: IDLE: python -m idlelib fails on master on Mac OS 10.10.4 https://bugs.python.org/issue35770 opened by xtreak #35771: IDLE: Fix tooltip Hovertiptest failure https://bugs.py
Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
Chris Angelico wrote: I would be strongly in favour of ctypes gaining a "get address of object" function, which happens (in current CPythons) to return the same value as id() does, but is specifically tied to ctypes. Isn't this what the ctypes.py_object type is for? Also, any code that does anything with the address of an object other than just pass it around is going to depend heavily on the Python implementation being used, so the idea of an implementation-independent way to deal with object addresses seems problematic. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
On Sat, Jan 19, 2019 at 9:58 AM Greg Ewing wrote: > > Chris Angelico wrote: > > I would be strongly in favour of ctypes gaining a "get address of > > object" function, which happens (in current CPythons) to return the > > same value as id() does, but is specifically tied to ctypes. > > Isn't this what the ctypes.py_object type is for? I didn't know about it when I posted that (as, I suspect, others also didn't), and as others have pointed out, this is a prime target for a docs update. Scanning the docs as of today does not suggest a better way to do things. ChrisA ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
MRAB wrote: If I want to cache some objects, I put them in a dict, using the id as the key. If I wanted to locate an object in a cache and didn't have id(), I'd have to do a linear search for it. That sounds dangerous. An id() is only valid as long as the object it came from still exists, after which it can get re-used for a different object. So when an object is flushed from your cache, you would have to chase down all the places its id is being stored and eliminate them. Are you sure you couldn't achieve the same thing more safely using weak references? -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
[MRAB] >> If I want to cache some objects, I put them in a dict, using the id as >> the key. If I wanted to locate an object in a cache and didn't have >> id(), I'd have to do a linear search for it. [Greg Ewing ] > That sounds dangerous. An id() is only valid as long as the object > it came from still exists, after which it can get re-used for a different > object. The objects are the values in such a dict. thedict[id(obj)] is obj Therefore the objects can't become garbage before id(obj) is deleted from the dict. > So when an object is flushed from your cache, you would have > to chase down all the places its id is being stored and eliminate them. The dict itself keeps the objects alive. > Are you sure you couldn't achieve the same thing more safely using > weak references? I can't say exactly what MRAB is doing. I've done things "like that" for decades, though, and have happily almost never used weakrefs. I wouldn't call my uses "caches", though - more like using dicts to associate info with arbitrary objects (via using the object id as the dict key), where the object implementations are out of my control and don't properly support being used as dict keys. This sometimes includes builtin mutable objects, like lists, or even other dicts. No such uses care about object addresses, though - just that id(obj) returns a value usable as a dict key, unique among all reachable objects at the time `id()` is called. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
On 2019-01-18 23:02, Greg Ewing wrote: MRAB wrote: If I want to cache some objects, I put them in a dict, using the id as the key. If I wanted to locate an object in a cache and didn't have id(), I'd have to do a linear search for it. That sounds dangerous. An id() is only valid as long as the object it came from still exists, after which it can get re-used for a different object. So when an object is flushed from your cache, you would have to chase down all the places its id is being stored and eliminate them. Are you sure you couldn't achieve the same thing more safely using weak references? I'm not storing the id anywhere else. I could've used a list for the cache, but then when I wanted to remove an object I'd have to search for it, O(n). Using a dict makes it O(1). ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
Tim Peters wrote: The dict itself keeps the objects alive. Yes, but the idea of a cache is that you're free to flush things out of it to make room for something else without breaking anything. It sounds like MRAB is using ids as weak references, without the assurance actual weak references give you that they become invalidated when the refefenced object goes away, No such uses care about object addresses, though - just that id(obj) returns a value usable as a dict key, unique among all reachable objects at the time `id()` is called. Yep. In hindsight it was probably a mistake for the docs to talk about addresses in relation to id() -- it seems to have given some people unrealistic expectations. -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
Steven D'Aprano wrote: The sample code I've been shown is this: pointer_to_obj = id(obj) from_deref = ctypes.cast(pointer_to_obj, ctypes.py_object).value from_deref is obj # True There's no need to use id() or casting to create a ctypes.py_object instance, you can just call it: >>> obj = (1,2,3) >>> obj (1, 2, 3) >>> p = ctypes.py_object(obj) >>> p py_object((1, 2, 3)) >>> p.value (1, 2, 3) >>> p.value is obj True -- Greg ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?
On 2019-01-19 00:28, Greg Ewing wrote: Tim Peters wrote: The dict itself keeps the objects alive. Yes, but the idea of a cache is that you're free to flush things out of it to make room for something else without breaking anything. It sounds like MRAB is using ids as weak references, without the assurance actual weak references give you that they become invalidated when the refefenced object goes away, "Cache" was the wrong word for what it does. I'm not using the id as a weak reference. Sometimes I might want to store a collection of objects and their order isn't important. I can add an object to the collection, or remove an object from it. If I used a list, adding would be quick, but removing would require searching the list. By putting them in a dict, keyed by the id, I can remove an object in O(1). Trust me, I'm not doing anything that's unreliable! (And I _have_ done programming in C with the Python API, so I know all about refcounts...) :-) No such uses care about object addresses, though - just that id(obj) returns a value usable as a dict key, unique among all reachable objects at the time `id()` is called. Yep. In hindsight it was probably a mistake for the docs to talk about addresses in relation to id() -- it seems to have given some people unrealistic expectations. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
