Re: Surprise using the 'is' operator
>> I don't pretend to understand the reasons for all of this, >> but if it is possible to use the same integer object whenever >> it is referenced, then why should there be a limit on range? > > Preallocating all the 4 billion 4-byte ints would overfill the memory of > most machines ;-) Sure, I just meant that once a given integer was used, it should be referenced the next time that value is required. As this thread exposed however, there is a trade off for lookup time. -- Posted via a free Usenet account from http://www.teranews.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
Terry Reedy wrote: > "Georg Brandl" <[EMAIL PROTECTED]> wrote in message > news:[EMAIL PROTECTED] >> tobiah wrote: >>> Suppose I fill an list with 100 million random integers in the range >>> of 1 - 65535. Wouldn't I save much memory if all of the ocurrances >>> of '12345' pointed to the same integer object? Why should more be made, >>> when they all do the same thing, and are not subject to change? >> >> Because for typical usage of integers (which doesn't include your >> example), >> it is more expensive to check if there's already an integer with that >> specific >> value out there than to create a new one. > > I am not sure what you are saying. Every (C) int is checked to see if it > is in the preallocated range before making a new Python int object. The > reason for the low limit is the various time-space tradeoffs applied across > the entire range of programs. Maybe I phrased that misleadingly (expensive refering to memory space). In the end, I wanted to say the same thing you did: it's a tradeoff between space and time. Georg -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
"Georg Brandl" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > tobiah wrote: >> Suppose I fill an list with 100 million random integers in the range >> of 1 - 65535. Wouldn't I save much memory if all of the ocurrances >> of '12345' pointed to the same integer object? Why should more be made, >> when they all do the same thing, and are not subject to change? > > Because for typical usage of integers (which doesn't include your > example), > it is more expensive to check if there's already an integer with that > specific > value out there than to create a new one. I am not sure what you are saying. Every (C) int is checked to see if it is in the preallocated range before making a new Python int object. The reason for the low limit is the various time-space tradeoffs applied across the entire range of programs. Terry Jan Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
"tobiah" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > wesley chun wrote: >>> "Terry Reedy" <[EMAIL PROTECTED]> wrote: >>> > as a side note, the ints that are cached (for current versions of >>> > Python) are in range(-1, 100)... is this documented somewhere? >>> Not true for at least 2.4 and 2.5. The cached range has expanded > I don't pretend to understand the reasons for all of this, > but if it is possible to use the same integer object whenever > it is referenced, then why should there be a limit on range? Preallocating all the 4 billion 4-byte ints would overfill the memory of most machines ;-) So there is a tradeoff between startup time, required minimal space usage by *every* program, needed or not, and runtime savings of time and space by programs that do make frequent use of 'small' ints. The expansion from 99 or 100 to 257 is partly a look ahead to a new 'bytes' type for 3.0 and maybe 2.x. > Suppose I fill an list with 100 million random integers in the range > of 1 - 65535. Wouldn't I save much memory if all of the ocurrances > of '12345' pointed to the same integer object? Yes > Why should more be made, > when they all do the same thing, and are not subject to change? Because such a situation is a special case and the delivered code is for everyone. If you compile your own binary, you can change the #define XXX 257 (where XXX is something obvious like MAX_SMALL_INT) in intobject.c to 65535 to make a special-purpose binary for such a use as you describe. Terry J. Reedy -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
tobiah wrote: > Simon Brunning wrote: > >>On 9/27/06, tobiah <[EMAIL PROTECTED]> wrote: >> >>>Suppose I fill an list with 100 million random integers in the range >>>of 1 - 65535. Wouldn't I save much memory if all of the ocurrances >>>of '12345' pointed to the same integer object? Why should more be made, >>>when they all do the same thing, and are not subject to change? >> >>If you were to drop that list, then to generate another large list of >>integers, you'd want to re-use the memory from the first lot, wouldn't >>you? >> >>(BTW, AFAIK, integers are kept seperate from other objects >>memory-wise, so memory used for integers won'tr be re-used for other >>object types. but memory used for integers can be re-used for *other* >>integers. I think.) >> > > > I'm confused now, but yes, I would want to reuse the memory for > the other integers. That's why I understand why I get the same > id back for small integers, but why limit that to (-5, 257)? > > Thanks, > > Toby > It's what's called an "implementation detail". Don't even worry about it until you need to shave every microsecond off your program's execution time, as reliance on such details reduces portability. regards Steve -- Steve Holden +44 150 684 7255 +1 800 494 3119 Holden Web LLC/Ltd http://www.holdenweb.com Skype: holdenweb http://holdenweb.blogspot.com Recent Ramblings http://del.icio.us/steve.holden -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
Georg Brandl <[EMAIL PROTECTED]> wrote: >> Suppose I fill an list with 100 million random integers in the range >> of 1 - 65535. Wouldn't I save much memory if all of the ocurrances >> of '12345' pointed to the same integer object? Why should more be >> made, when they all do the same thing, and are not subject to change? > > Because for typical usage of integers (which doesn't include your > example), it is more expensive to check if there's already an integer > with that specific value out there than to create a new one. Ok, so here's a follow-on question: why doesn't intern() accept integer arguments? If it did then you could do the memory optimisation anywhere it seemed worthwhile (and all constant integers could be interned when code is first loaded at effectively no expense). -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
tobiah wrote: >>> Suppose I fill an list with 100 million random integers in the range >>> of 1 - 65535. Wouldn't I save much memory if all of the ocurrances >>> of '12345' pointed to the same integer object? Why should more be made, >>> when they all do the same thing, and are not subject to change? >> >> Because for typical usage of integers (which doesn't include your example), >> it is more expensive to check if there's already an integer with that >> specific >> value out there than to create a new one. >> >> >> Georg > > I see. I assume then, that the lookup performance hit is acceptable > as a trade off against memory usage for the quite commonly used range > of (-5, 257). Is that the idea? Yes, indeed. Georg -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
>> Suppose I fill an list with 100 million random integers in the range >> of 1 - 65535. Wouldn't I save much memory if all of the ocurrances >> of '12345' pointed to the same integer object? Why should more be made, >> when they all do the same thing, and are not subject to change? > > Because for typical usage of integers (which doesn't include your example), > it is more expensive to check if there's already an integer with that > specific > value out there than to create a new one. > > > Georg I see. I assume then, that the lookup performance hit is acceptable as a trade off against memory usage for the quite commonly used range of (-5, 257). Is that the idea? Thanks, Tobiah -- Posted via a free Usenet account from http://www.teranews.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
Simon Brunning wrote: > On 9/27/06, tobiah <[EMAIL PROTECTED]> wrote: >> Suppose I fill an list with 100 million random integers in the range >> of 1 - 65535. Wouldn't I save much memory if all of the ocurrances >> of '12345' pointed to the same integer object? Why should more be made, >> when they all do the same thing, and are not subject to change? > > If you were to drop that list, then to generate another large list of > integers, you'd want to re-use the memory from the first lot, wouldn't > you? > > (BTW, AFAIK, integers are kept seperate from other objects > memory-wise, so memory used for integers won'tr be re-used for other > object types. but memory used for integers can be re-used for *other* > integers. I think.) > I'm confused now, but yes, I would want to reuse the memory for the other integers. That's why I understand why I get the same id back for small integers, but why limit that to (-5, 257)? Thanks, Toby -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
tobiah wrote: > wesley chun wrote: >>> "Terry Reedy" <[EMAIL PROTECTED]> wrote: >>> > as a side note, the ints that are cached (for current versions of >>> > Python) are in range(-1, 100)... is this documented somewhere? >>> Not true for at least 2.4 and 2.5. The cached range has expanded >> >> oops, apologies to all... it really *is* subject to change. ;-) it's >> now range(-5, 257). >> >> -wesley > > I don't pretend to understand the reasons for all of this, > but if it is possible to use the same integer object whenever > it is referenced, then why should there be a limit on range? > > Suppose I fill an list with 100 million random integers in the range > of 1 - 65535. Wouldn't I save much memory if all of the ocurrances > of '12345' pointed to the same integer object? Why should more be made, > when they all do the same thing, and are not subject to change? Because for typical usage of integers (which doesn't include your example), it is more expensive to check if there's already an integer with that specific value out there than to create a new one. Georg -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
On 9/27/06, tobiah <[EMAIL PROTECTED]> wrote: > Suppose I fill an list with 100 million random integers in the range > of 1 - 65535. Wouldn't I save much memory if all of the ocurrances > of '12345' pointed to the same integer object? Why should more be made, > when they all do the same thing, and are not subject to change? If you were to drop that list, then to generate another large list of integers, you'd want to re-use the memory from the first lot, wouldn't you? (BTW, AFAIK, integers are kept seperate from other objects memory-wise, so memory used for integers won'tr be re-used for other object types. but memory used for integers can be re-used for *other* integers. I think.) -- Cheers, Simon B, [EMAIL PROTECTED] -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
wesley chun wrote: >> "Terry Reedy" <[EMAIL PROTECTED]> wrote: >> > as a side note, the ints that are cached (for current versions of >> > Python) are in range(-1, 100)... is this documented somewhere? >> Not true for at least 2.4 and 2.5. The cached range has expanded > > oops, apologies to all... it really *is* subject to change. ;-) it's > now range(-5, 257). > > -wesley I don't pretend to understand the reasons for all of this, but if it is possible to use the same integer object whenever it is referenced, then why should there be a limit on range? Suppose I fill an list with 100 million random integers in the range of 1 - 65535. Wouldn't I save much memory if all of the ocurrances of '12345' pointed to the same integer object? Why should more be made, when they all do the same thing, and are not subject to change? Thanks, Tobiah -- Posted via a free Usenet account from http://www.teranews.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
> "Terry Reedy" <[EMAIL PROTECTED]> wrote: > > as a side note, the ints that are cached (for current versions of > > Python) are in range(-1, 100)... is this documented somewhere? > Not true for at least 2.4 and 2.5. The cached range has expanded oops, apologies to all... it really *is* subject to change. ;-) it's now range(-5, 257). -wesley -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
"wesley chun" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED] > as a side note, the ints that are cached (for current versions of > Python) are in range(-1, 100)... is this documented somewhere? Not true for at least 2.4 and 2.5. The cached range has expanded > i know it's subject to change. Yes ;-) tjr -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
Wesley> as a side note, the ints that are cached (for current versions Wesley> of Python) are in range(-1, 100)... is this documented Wesley> somewhere? Other than the code, probably not. Since it's an implementation optimization, I see no particular reason that it should be documented elsewhere. Hmmm... It would be interesting if we had a buildbot which through a series of slightly twisty C macros built an interpreter that eliminates the free list altogether to make sure nothing depends on its presence or size. Skip -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
as a side note, the ints that are cached (for current versions of Python) are in range(-1, 100)... is this documented somewhere? i know it's subject to change. and as already mentioned, you probably wouldn't use "is" this way in real code... i know you are just reinforcing your learning of the difference between object *value* comparison vs. object *identity* comparison. what you've discovered, however, is important in learning Python... i have material in my intro course and the book dedicated to this subject. cheers, -- wesley - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - "Core Python Programming", Prentice Hall, (c)2007,2001 http://corepython.com wesley.j.chun :: wescpy-at-gmail.com python training and technical consulting cyberweb.consulting : silicon valley, ca http://cyberwebconsulting.com -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
Antoon Pardon wrote: > > I find this a bit oddly worded. Now the "may always reuse" phrase > suggests this is not an obligation and I can certainly understand > that in the case of integers. But when you enumerate examples you > include None and Booleans, creating the suggestion these don't > have to be implemented as singletons either and that there can be > more than one None, True and False object. Now I probably just > misunderstand, but I'm wondering, is there somewhere in the language > reference that specifies these have to be singletons? Yes. The topic "The standard Type Hierarchy" in the Language reference specifies this exactly. John Roth > > -- > Antoon Pardon -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
On 2006-09-26, John Roth <[EMAIL PROTECTED]> wrote: > > Antoon Pardon wrote: >> >> I find this a bit oddly worded. Now the "may always reuse" phrase >> suggests this is not an obligation and I can certainly understand >> that in the case of integers. But when you enumerate examples you >> include None and Booleans, creating the suggestion these don't >> have to be implemented as singletons either and that there can be >> more than one None, True and False object. Now I probably just >> misunderstand, but I'm wondering, is there somewhere in the language >> reference that specifies these have to be singletons? > > Yes. The topic "The standard Type Hierarchy" in the > Language reference specifies this exactly. Thank you very much. -- Antoon Pardon -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
"codefire" <[EMAIL PROTECTED]> writes: > I was just trying to check if objects were the same (object), didn't > know Integers were a special case. They're not. The Python runtime environment can do whatever it likes underneath the hood; the language gives no promises about any relationship between 'is' and '=='. What you may be missing is that 'c = 10' is not "putting the value 10 into the box named c". It is rather "putting the label c onto the object 10". Whether there are one, two or twelve instances of an object '10' is entirely up to the Python runtime implementation. You can use 'is' to check whether two objects are identical; you can use '==' to check whether they are equal in value; you cannot generalise anything about the relationship between those two. -- \ "I still have my Christmas Tree. I looked at it today. Sure | `\enough, I couldn't see any forests." -- Steven Wright | _o__) | Ben Finney -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
Thanks for that Fredrik, that's clear. That's actually a pretty nice feature as it's nicely optimised. >>> a = 10 >>> c = 10 >>> a is c True >>> c = c +1 >>> a is c False >>> Cheers, Tony -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
Haha! OK thanks guys. I was just trying to check if objects were the same (object), didn't know Integers were a special case. Thanks, Tony -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
On 2006-09-26, Fredrik Lundh <[EMAIL PROTECTED]> wrote: > codefire wrote: > >> I was just trying to check if objects were the same (object), didn't >> know Integers were a special case. > > they're not, really; "is" works the same way for all objects. > > when you ask for a new immutable object, a Python implementation may > always reuse an existing object, if it wants to. > > the CPython implementation does this for small integers, booleans, None, > empty tuples, certain strings, type objects, etc. this also applies to > library code; for example, string methods often return a reference to > "self" if the method didn't actually change anything. I find this a bit oddly worded. Now the "may always reuse" phrase suggests this is not an obligation and I can certainly understand that in the case of integers. But when you enumerate examples you include None and Booleans, creating the suggestion these don't have to be implemented as singletons either and that there can be more than one None, True and False object. Now I probably just misunderstand, but I'm wondering, is there somewhere in the language reference that specifies these have to be singletons? -- Antoon Pardon -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
codefire a écrit : > Haha! > > OK thanks guys. > > I was just trying to check if objects were the same (object), didn't > know Integers were a special case. They are not a special case so much. It's just that "is" is extremly unreliable unless you're the one who created the objects. In the case of integers, it's the CPython implementation which constructs them as it sees fit and so, you cannot expect any kind of reliable "is" behavior. When you are manipulating let's say lists, it gets much more reliable. You do know that the list you create by doing a = [] is not and will never be the same than the one you'll create later by doing b = [] -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
codefire wrote: > I was just trying to check if objects were the same (object), didn't > know Integers were a special case. they're not, really; "is" works the same way for all objects. when you ask for a new immutable object, a Python implementation may always reuse an existing object, if it wants to. the CPython implementation does this for small integers, booleans, None, empty tuples, certain strings, type objects, etc. this also applies to library code; for example, string methods often return a reference to "self" if the method didn't actually change anything. -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
codefire a écrit : > I thought the 'is' operator was used to identify identical objects, > whereas the '==' operator checked equality. Well, I got a surprise > here: > > IDLE 1.1.3 a = 10 b = a a is b > True a == b > True c = 10 a == c > True a is c > True > > I was NOT expecting the last statement to return True! The answer is : Why not? Does it even matter for integers? Never use "is" on integers, always == ( and on strings too for that matter ) -- http://mail.python.org/mailman/listinfo/python-list
Re: Surprise using the 'is' operator
codefire wrote: > I thought the 'is' operator was used to identify identical objects, > whereas the '==' operator checked equality. Well, I got a surprise > here: > > IDLE 1.1.3 a = 10 b = a a is b > True a == b > True c = 10 a == c > True a is c > True > > I was NOT expecting the last statement to return True! > > What am I missing? One of the most severely beaten dead horses of python - small integers are cached by the interpreter. Maybe. Thus the above results. Google for a bazillion discussions on this. Diez -- http://mail.python.org/mailman/listinfo/python-list
