Re: [Tutor] object size in python is in what units?
On 23/07/13 19:27, Jim Mooney wrote:
And yet I still get 36 with this:
bigKronk = Kronk('supercalifracilisticexpalidocious, even though the
sound of it is really quite atrocious')
sys.getsizeof(bigKronk)
36
So I guess the object is only pointing to its contents.
Python namespaces are effectively dictionaries binding a name
with an object reference. The object in turn is like a dictionary
referring to the values. So I guess you are seeing the object
container size.
means an object is just a bunch of names bound to pointers.
Pretty much everything in Python looks like a dictionary
so I'd guess you are correct.
--
Alan G
Author of the Learn to Program web site
http://www.alan-g.me.uk/
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] object size in python is in what units?
On 23 July 2013 07:19, Steven D'Aprano wrote: > Some day, I'll invent my own version of Python, and make object IDs be > consecutive prime numbers. That'll teach people to rely on them. > --- Why "consecutive?" That's still way too predictable (Yes, the obvious flaw already occurred to me ;') Jim ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] object size in python is in what units?
On 23 July 2013 07:21, Oscar Benjamin wrote:
>
>
> You can check the size in bytes of an object with sys.getsizeof e.g.:
>
> >>> import sys
> >>> sys.getsizeof(lardKronk)
> 36
>
And yet I still get 36 with this:
bigKronk = Kronk('supercalifracilisticexpalidocious, even though the sound
of it is really quite atrocious')
sys.getsizeof(bigKronk)
36
So I guess the object is only pointing to its contents. Which I assume
means an object is just a bunch of names bound to pointers.
Jim
An excellent animation/graph of the accelerating disappearance of arctic
sea ice, which controls
climate stability in the Northern hemisphere:
https://www.youtube.com/watch?v=YgiMBxaL19M
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] object size in python is in what units?
On 23/07/13 18:17, Jim Mooney wrote: On 23 July 2013 00:40, Steven D'Aprano wrote: No no no, a thousand times no!!! IDs are just numeric IDs, that is all, like your social security number or driver's licence number. Don't think of them as having any meaning at all, except that they are unique during the lifetime of the object. Okay, ID stands for a location fixed until the object disappears, but we don't know where that location is. No, it does not stand for a location. It is an identification number, that is all. In CPython, every value (lists, strings, integers, custom classes) are objects which exist in one location until destroyed. In Jython and IronPython, values are objects which can move from location to location, in order to free up memory for new objects. In PyPy, not only can objects move, but they can also be invisibly turned into low-level data structures for efficient processing. So in CPython, Jython and IronPython, an object's identity is fixed. In PyPy, object identity is an illusion, and according to the PyPy developers, keeping that illusion working requires a lot of effort. But what about object offsets from self? That's always zero, since self is the object :-) You can read up about the implementation of CPython here: http://docs.python.org/2/c-api/ but this describes only the C interface, not the concrete implementation. For that you pretty much have to read the source code. The C source code. Is the beginning of self.spam always the same distance from the beginning of self.eggs? Or can I just forget the hard ground of assembler-metaphors entirely as I float off into abstractville? I guess the fixed lengths I kept getting on re-runs were coincidental but not to be relied on. Understanding a little of the concrete implementation details will help you understand some of Python's design, and its limitations. But apart from that, it's not necessary, and sometimes it is actually counter-productive. Thinking in terms of low-level data can lead you wrong. For instance, when working in low-level languages, it is normal to write code to minimize moving data from place to place. For example, when sorting in C or assembly, comparing two values is cheap, but swapping them is expensive. In Python it is the opposite: swapping two values in a list just requires swapping two pointers, which is cheap, but the comparison is quite expensive and could call arbitrary user code. In CPython, objects are implemented in C, and are a block of memory that contains various fields, and potentially pointers to other objects. The nature of those fields will depend on which type of object they are. In Jython, objects are implemented in Java, and in IronPython they are implemented in C#, but otherwise are similar. But the layout of the fields will be different, and the fields themselves, except perhaps by accident. In PyPy, objects are implemented in RPython, a restricted and simplified version of Python, which is then automatically translated by the optimizing compiler into some low-level data structure behind the scenes. Since Python is defined by its behaviour, not any sort of concrete implementation, in principle one could invent a "Python Engine", like Babbage's Difference Engine, only thousands of times more complex, entirely out of clockwork. Or by water flowing through pipes, or reproducing DNA in a test tube, or some exotic quantum field. Or simply simulate a Python interpreter in your head, perhaps aided by pencil and paper. -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] object size in python is in what units?
On 24/07/13 00:21, Oscar Benjamin wrote: The reason for the patterns that you observe are related to CPython's memory pool allocator. You can read the long source comment describing this here: http://hg.python.org/cpython/file/a5681f50bae2/Objects/obmalloc.c#l367 [...] Nice analysis! Thanks Oscar! -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] object size in python is in what units?
On 23/07/13 18:19, Marc Tompkins wrote: A couple of clarifications I wish I'd made before hitting Send: I shouldn't have said that IDs are "derived" or "calculated"; they're "assigned", presumably on a first-come, first-served basis; Actually, in CPython they are derived, from the memory location, which annoys me no end :-) But in Jython and IronPython they are assigned, when and as requested. Some day, I'll invent my own version of Python, and make object IDs be consecutive prime numbers. That'll teach people to rely on them. :-) -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] object size in python is in what units?
Speaking of brain dead, I was taking a break from Python by looking at a video on finite state machines and the very first example was tennis scoring. I think the $#@! insane tennis scoring was harder to understand than the subject. That got me laughing pretty good, since I too was recently trying to learn Finite State Machines. I play tennis occasionally and still forget how to score properly! Jim ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] object size in python is in what units?
On 23 July 2013 01:13, Marc Tompkins wrote: When I saved your code as a file on my machine (Python 2.7 on Win8 64-bit) > and ran it from the command line, I got the same result as you - three IDs, > 40 apart every time. However, I initially pasted your code into > PyScripter, and when I ran it inside PyScripter the results were different: > if the first ID was x, then the second was x+160, and the third was x+200. > (160 is obviously divisible by 40... I have no idea what conclusion to draw > from that.) > I like PyScripter, but I suspect it injects a lot of behind-the-scenes code. At times it does something odd or just chokes, so if something looks funny, I run it from Wing 101, which nearly always gives the same result as the command line and has yet to choke. And the command line, of course, is the final arbiter once a program is complete. So far, Wing 101 has fulfilled my expectations as a basic Python IDE, so it's the one I'll buy if I fall into money. PyCharm might be good but they don't have a free version and I'm not learning a tryout I might have to ditch. Which means the folks at Wing are smarter marketers by giving away the basic version ;') I think that Joel guy at .joelonsoftware.com mentioned something to that effect. An excellent site to peruse when I'm feeling brain-dead and need a break. Speaking of brain dead, I was taking a break from Python by looking at a video on finite state machines and the very first example was tennis scoring. I think the $#@! insane tennis scoring was harder to understand than the subject. Jim ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] object size in python is in what units?
On Tue, Jul 23, 2013 at 1:17 AM, Jim Mooney wrote: > On 23 July 2013 00:40, Steven D'Aprano wrote: > >> >> No no no, a thousand times no!!! IDs are just numeric IDs, that is all, >> like your social security number or driver's licence number. Don't think of >> them as having any meaning at all, except that they are unique during the >> lifetime of the object. >> > > Okay, ID stands for a location fixed until the object disappears, but we > don't know where that location is. But what about object offsets from self? > Is the beginning of self.spam always the same distance from the beginning > of self.eggs? Or can I just forget the hard ground of assembler-metaphors > entirely as I float off into abstractville? I guess the fixed lengths I > kept getting on re-runs were coincidental but not to be relied on. > As Steven said: they are NOT locations, they are IDs. There is no distance involved. You CAN find the size of an object, and you can find its location in memory (but you really shouldn't bother, or rely on it) but neither of those things has any particular relation to the ID, except accidentally. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] object size in python is in what units?
On Tue, Jul 23, 2013 at 1:13 AM, Marc Tompkins wrote: > As an experiment, I added a couple of extra methods and attributes to the > Kronk class, but it didn't change anything - IDs still incremented by 40 > each time. eryksun will no doubt chime in to tell us exactly how object > IDs are derived in cPython, but I'll go out on a limb and say that they > have nothing to do with the size of the object; also that one would be very > silly to make a program rely in any way on predicting the next ID, since > external factors may throw off the calculations (as when I invoked your > original script from inside PyScripter). All that you can rely on is that > each unique object (i.e. doesn't pass an "is" comparison with any other > object) has a unique ID... and, for all I know, not even that. > A couple of clarifications I wish I'd made before hitting Send: I shouldn't have said that IDs are "derived" or "calculated"; they're "assigned", presumably on a first-come, first-served basis; when I said "external factors may throw off the calculations" I meant "may make your calculations non-congruent with reality". ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] object size in python is in what units?
On 23 July 2013 00:40, Steven D'Aprano wrote: > > No no no, a thousand times no!!! IDs are just numeric IDs, that is all, > like your social security number or driver's licence number. Don't think of > them as having any meaning at all, except that they are unique during the > lifetime of the object. > Okay, ID stands for a location fixed until the object disappears, but we don't know where that location is. But what about object offsets from self? Is the beginning of self.spam always the same distance from the beginning of self.eggs? Or can I just forget the hard ground of assembler-metaphors entirely as I float off into abstractville? I guess the fixed lengths I kept getting on re-runs were coincidental but not to be relied on. -- Jim An excellent animation/graph of the accelerating disappearance of arctic sea ice, which controls climate in the Northern hemisphere: https://www.youtube.com/watch?v=YgiMBxaL19M It is incontrovertible that Something is happening. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] object size in python is in what units?
On Tue, Jul 23, 2013 at 12:09 AM, Jim Mooney wrote: > I've noticed that when I create a number of objects from a class, one > after another, they're at different IDs, but the IDs all appear to be > equidistant, so that they all appear to have the same size. But what does > that size represent? is it bytes? ie - if I run this over and over I get > different id numbers but they are always 40 apart. Or is that distance > machine-specific as to units?: > > > 41599624 - different results for each run but always 40 apart > 41599664 > 41599704 > ''' > When I saved your code as a file on my machine (Python 2.7 on Win8 64-bit) and ran it from the command line, I got the same result as you - three IDs, 40 apart every time. However, I initially pasted your code into PyScripter, and when I ran it inside PyScripter the results were different: if the first ID was x, then the second was x+160, and the third was x+200. (160 is obviously divisible by 40... I have no idea what conclusion to draw from that.) I simplified the test-harness portion as follows: kronkList = [] for kronknum in xrange(0,100): kronkList.append(Kronk(kronknum)) print(kronkList[kronknum].getzar()) print(kronkList[kronknum].getself()) and now the ID is incremented by 40 each time regardless how I run the script. Each time I run it, the last digit stays the same throughout the run (as you would expect, given that we're adding 40 each time), but the last digits are NOT consistent _between_ runs. It appears to always be an even number, though. As an experiment, I added a couple of extra methods and attributes to the Kronk class, but it didn't change anything - IDs still incremented by 40 each time. eryksun will no doubt chime in to tell us exactly how object IDs are derived in cPython, but I'll go out on a limb and say that they have nothing to do with the size of the object; also that one would be very silly to make a program rely in any way on predicting the next ID, since external factors may throw off the calculations (as when I invoked your original script from inside PyScripter). All that you can rely on is that each unique object (i.e. doesn't pass an "is" comparison with any other object) has a unique ID... and, for all I know, not even that. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] object size in python is in what units?
On 23/07/13 17:09, Jim Mooney wrote: I've noticed that when I create a number of objects from a class, one after another, they're at different IDs, but the IDs all appear to be equidistant, so that they all appear to have the same size. But what does that size represent? is it bytes? No no no, a thousand times no!!! IDs are just numeric IDs, that is all, like your social security number or driver's licence number. Don't think of them as having any meaning at all, except that they are unique during the lifetime of the object. Some Python implementations never reuse IDs, and they are numbered consecutively. IronPython numbers them from 40 (if memory serves me right), so you get 40, 41, 42, 43, ... while Jython numbers them consecutively from 1, 2, 3, ... PyPy does something similar. CPython does reuse IDs, and uses the memory address of the base of the object, as reported by the operating system, which leads to annoying people saying that id() returns the address of the object. This is WRONG. It does not. id() returns an arbitrary ID number. ie - if I run this over and over I get different id numbers but they are always 40 apart. Or is that distance machine-specific as to units?: Yes. It depends on whether CPython gets a chance to reuse the same memory address or not, whether you have a 32-bit or 64-bit Python compiler, whether any other objects are created in between each one, and many other factors. -- Steven ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] object size in python is in what units?
On Tue, Jul 23, 2013 at 9:09 AM, Jim Mooney wrote: > I've noticed that when I create a number of objects from a class, one > after another, they're at different IDs, but the IDs all appear to be > equidistant, so that they all appear to have the same size. But what does > that size represent? is it bytes? ie - if I run this over and over I get > different id numbers but they are always 40 apart. Or is that distance > machine-specific as to units?: > > In Python, the id is just a number that identifies the object. You can't count on it to represent object size, or anything else for that matter. The only requirement for python implementations is that different objects can not have the same id. The CPython implementation uses the memory address of the object as its id because it's simple and satisfies the above requirement. Still, i'd say it's very unsafe to try to infer object sizes from this. It depends on how memory addresses work on your machine, what type of memory allocator is used and how much overhead it has, and many more factors. Furthermore, other python implementations (I believe Jython and IronPython among them) do not even use memory addresses as id numbers, but simply assign consecutive integers to each created object. In general, relying on the id number to tell you anything other than whether two variables point to the same object is highly unreliable at best and impossible in the general case. HTH, Hugo ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
[Tutor] object size in python is in what units?
I've noticed that when I create a number of objects from a class, one after
another, they're at different IDs, but the IDs all appear to be
equidistant, so that they all appear to have the same size. But what does
that size represent? is it bytes? ie - if I run this over and over I get
different id numbers but they are always 40 apart. Or is that distance
machine-specific as to units?:
#Using Python 2.7 on Win 7
from __future__ import division, print_function
class Kronk:
def __init__(self, zar):
self.zar = zar
def getzar(self):
return self.zar
def getself(self):
return id(self)
lardKronk = Kronk('ziggle')
newKronk = Kronk('barf')
budKronk = Kronk('baloney')
print(lardKronk.getzar())
print(newKronk.getzar())
print(budKronk.getzar())
print(lardKronk.getself())
print(newKronk.getself())
print(budKronk.getself())
'''result:
ziggle
barf
baloney
41599624 - different results for each run but always 40 apart
41599664
41599704
'''
--
Jim
When I was young dad told me if a moth fluttered onto my sleeping lips at
night
it would suck out my soul.
Dad was such a kidder.
But I still flee from moths.
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
http://mail.python.org/mailman/listinfo/tutor
