Re: [Tutor] lists, name semantics
On 04/19/2015 12:07 AM, boB Stepp wrote: . Before Peter changed one of these changeable objects, he had: a = [1, [x, y], 3] b = a[:] Now BOTH a[1] and b[1] now identify the location of the inner list object, [x, y] . Apparently, Python, in its ever efficient memory management fashion, when it creates the new object/piece of data a[:], it sees no need to duplicate the inner list object, [x, y], but instead creates another identifier/pointer/reference to this object's location. But since this inner list object is mutable, when you change y to hello! in b, you also change it in a because both a[1][1] and b[1][1] reference/point to the exact same storage location where this element of the inner list is actually stored. I hope this is helpful, and, if there are any misstepps, that when they are revealed both of our understandings will be enhanced! Some of your knowledge of other languages is leaking into your explanation. When we talk of the language Python, we need to distinguish between how CPython happens to be implemented, how other Python implementations happen to be created, and how C++ (for example) implements similar things. Some of the above use pointers, some do not. The language Python does not. So especially when talking of inner lists, we need to clarify a few things. An object has an identity, not a location. That identity can be checked with the 'is' operator, or the id() function. But it exists all the time. Variables, as you say, do not contain an object, they reference it. And the formal term for that is binding. A name is bound to an object, to one object, at a time. Now some objects have attributes, which is to say names, and those attributes are bound to other objects. So if we define a class, and have an instance of that class, and the instance has attributes, we can do something like: obj.inst_name and get the particular attribute. Still other objects have unnamed bindings. The canonical example is a list. A list object has a bunch of bindings to other objects. Even though each binding doesn't have a specific name, it nevertheless exists. And in this case we use integers to specify which of those bindings we want to follow. And we use a special syntax (the square bracket) to indicate which of these we want. So let's take the simplest example: mylist = [a, b] the name mylist is bound to a list object, which has two numbered bindings, called 0 and 1. That object's [0] binding is to a separate object a. And the [1] binding is to a separate object b There is another shorthand called a slice, which looks like [start, len, step] which lets us construct a *new* list from our list. And using defaults for all three terms lets us copy the list. But let's look at what the copy means: newlist = mylist[:] -- mylist[0, 2, 1] This constructs a new list from the present one, where the zeroth location is bound to whatever object the first list's zeroth location was bound to. And so on for the oneth location, the twoth, etc. Only one new list object is built, and no new objects are made for it. It just gets new bindings to the same things the first list had. At this point, it should be clear what a shallow copy means. If the original list's oneth item was a binding to another list object, *that* list object does NOT get copied. I don't like the term inner list, but I don't know if it's incorrect. It's just misleading, since to the slice operation, the fact that it's a list is irrelevant. It's just an object whose binding is to be copied. -- DaveA ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
On 04/19/2015 03:08 PM, boB Stepp wrote: On Sun, Apr 19, 2015 at 6:47 AM, Dave Angel da...@davea.name wrote: On 04/19/2015 12:07 AM, boB Stepp wrote: [...] I hope this is helpful, and, if there are any misstepps, that when they are revealed both of our understandings will be enhanced! Some of your knowledge of other languages is leaking into your explanation. When we talk of the language Python, we need to distinguish between how CPython happens to be implemented, how other Python implementations happen to be created, and how C++ (for example) implements similar things. Some of the above use pointers, some do not. The language Python does not. I actually was being deliberately *imprecise* in my use of technical terminology. But I see below that there are some nuances I need to learn... So especially when talking of inner lists, we need to clarify a few things. An object has an identity, not a location. That identity can be checked with the 'is' operator, or the id() function. But it exists all the time. But the object, in order to exist, must be stored in RAM somewhere, doesn't it? Or in a swap file, a disk file, or some other media. Or is the real point that we are adding an abstraction layer so we don't even have to think about where objects are physically stored in RAM? Somebody keeps track, but the address is not necessarily constant, and not necessarily stored in any references. The references (bindings) are abstract, and the details are unimportant to the user. For example, the jython system does not use addresses at all. And an object gets moved around from time to time without its references knowing it. So I am the object referenced by boB and we don't care what my precise (x, y, z) coordinates are relative to the planet Earth. If we need to find me or modify me we use my label, boB, to access me? No, we use one of your many labels to find you. And if no labels exist, you quietly cease to exist (garbage collection). But understand that a label in this sense need not be some alpha string. It might be a slot in some collection, like the way I described list. Variables, as you say, do not contain an object, they reference it. And the formal term for that is binding. A name is bound to an object, to one object, at a time. Now some objects have attributes, which is to say names, and those attributes are bound to other objects. So if we define a class, and have an instance of that class, and the instance has attributes, we can do something like: obj.inst_name and get the particular attribute. Still other objects have unnamed bindings. The canonical example is a list. A list object has a bunch of bindings to other objects. Even though each binding doesn't have a specific name, it nevertheless exists. And in this case we use integers to specify which of those bindings we want to follow. And we use a special syntax (the square bracket) to indicate which of these we want. Ah, unnamed bindings was the concept I was talking around. I realized these were there and were referenced by the square bracket syntax, but I did not know what to call the concept. [...] At this point, it should be clear what a shallow copy means. If the original list's oneth item was a binding to another list object, *that* list object does NOT get copied. I don't like the term inner list, but I don't know if it's incorrect. It's just misleading, since to the slice operation, the fact that it's a list is irrelevant. It's just an object whose binding is to be copied. So the real point here is that there are two distinct copying mechanisms, deep and shallow. The inner list could just have been any other type of object, though if it had been an immutable type it would not have made Peter's original interesting point. The distinction between deep and shallow was already made, by better people than I. I was trying to explain it in terms of a working model that would let you predict what will happen in each circumstance. A slice only copies the bindings in the list, it doesn't care what they are bound to. It's shallow because it's shallow. -- DaveA ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
On Sun, Apr 19, 2015 at 6:23 PM, boB Stepp robertvst...@gmail.com wrote: On Sun, Apr 19, 2015 at 4:05 PM, Dave Angel da...@davea.name wrote: On 04/19/2015 03:08 PM, boB Stepp wrote: Or is the real point that we are adding an abstraction layer so we don't even have to think about where objects are physically stored in RAM? Somebody keeps track, but the address is not necessarily constant, and not necessarily stored in any references. The references (bindings) are abstract, and the details are unimportant to the user. For example, the jython system does not use addresses at all. And an object gets moved around from time to time without its references knowing it. The last sentence in this paragraph has me intrigued. Why would an object, once it has been created, be moved? What practical benefit does doing this give? boB I'm guessing memory management. You want to have large contiguous blocks of memory available for large objects. If a small object is surrounded by available memory, you might want to move it to some smaller empty spot. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor -- Joel Goldstick http://joelgoldstick.com ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
In a message of Sun, 19 Apr 2015 17:23:13 -0500, boB Stepp writes: On Sun, Apr 19, 2015 at 4:05 PM, Dave Angel da...@davea.name wrote: abstract, and the details are unimportant to the user. For example, the jython system does not use addresses at all. And an object gets moved around from time to time without its references knowing it. The last sentence in this paragraph has me intrigued. Why would an object, once it has been created, be moved? What practical benefit does doing this give? boB One great reason is called 'garbage collection'. We find all the objects we are using, stuff them someplace in memory, and free up all the space they left. Now we are free to lay down a whole lot of other things, in contiguous order. When you want to go access an array, for instance, it matters' a whole lot if you can actually say: for x for y tell_me (x,y) rather than a lot of 'oops, I would love to tell you but when I went to an address I got told, out space, go look over hear instead. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
On Sun, Apr 19, 2015 at 12:24 AM, Cameron Simpson c...@zip.com.au wrote: On 19Apr2015 15:09, Cameron Simpson c...@zip.com.au wrote: On 18Apr2015 23:26, boB Stepp robertvst...@gmail.com wrote: On Sat, Apr 18, 2015 at 11:08 PM, Cameron Simpson c...@zip.com.au wrote: [...] Two problems often exist with deep copy operations that don’t exist with shallow copy operations: Recursive objects (compound objects that, directly or indirectly, contain a reference to themselves) may cause a recursive loop. Because deep copy copies everything it may copy too much, e.g., administrative data structures that should be shared even between copies. If I am understanding things correctly, should not that last sentence read instead: ...structures that should *not* be shared even between copies. ??? No, the text is correct. Um, my explaination was incomplete. [...] The second sentence Because deep copy copies everything it may copy too much, e.g., administrative data structures that should be shared even between copies is an issuewhich is addressed lower down when mentioning the __copy_ and __deepcopy__ methods. And it is not necessarily obvious. Suppose you've got a data structure of objects, which should broadly be copied. However, _internally_, these objects may use some external facility. Further, suppose that facility can't be copied; perhaps it is a reference to a databse or something built around something like that. Like this: G - A - B - _open_database When you deepcopy that data sctructure you want to copy everything, but _not_ copy the external facility object. So for the above example, after the deepcopy you want this: Gcopy - Acopy - Bcopy - _open_database i.e. the same reference to the database, but copies of A and B. Ah! This example clarifies things for me. This provides a specific context for the phrase, ...that should be shared even between copies... That now makes sense. If you give G's class a .__deepcopy__ method, then that will be called by deepcopy() to make a copy of G by calling G.__deepcopy__(memodict). And G's class will define its __deepcopy__ method to copy A and B but not _open_database. Most classes do not need this and deepcopy() just copies everything. Does this clarify things? Very much! Thanks! -- boB ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
On Sun, Apr 19, 2015 at 6:47 AM, Dave Angel da...@davea.name wrote: On 04/19/2015 12:07 AM, boB Stepp wrote: [...] I hope this is helpful, and, if there are any misstepps, that when they are revealed both of our understandings will be enhanced! Some of your knowledge of other languages is leaking into your explanation. When we talk of the language Python, we need to distinguish between how CPython happens to be implemented, how other Python implementations happen to be created, and how C++ (for example) implements similar things. Some of the above use pointers, some do not. The language Python does not. I actually was being deliberately *imprecise* in my use of technical terminology. But I see below that there are some nuances I need to learn... So especially when talking of inner lists, we need to clarify a few things. An object has an identity, not a location. That identity can be checked with the 'is' operator, or the id() function. But it exists all the time. But the object, in order to exist, must be stored in RAM somewhere, doesn't it? Or is the real point that we are adding an abstraction layer so we don't even have to think about where objects are physically stored in RAM? So I am the object referenced by boB and we don't care what my precise (x, y, z) coordinates are relative to the planet Earth. If we need to find me or modify me we use my label, boB, to access me? Variables, as you say, do not contain an object, they reference it. And the formal term for that is binding. A name is bound to an object, to one object, at a time. Now some objects have attributes, which is to say names, and those attributes are bound to other objects. So if we define a class, and have an instance of that class, and the instance has attributes, we can do something like: obj.inst_name and get the particular attribute. Still other objects have unnamed bindings. The canonical example is a list. A list object has a bunch of bindings to other objects. Even though each binding doesn't have a specific name, it nevertheless exists. And in this case we use integers to specify which of those bindings we want to follow. And we use a special syntax (the square bracket) to indicate which of these we want. Ah, unnamed bindings was the concept I was talking around. I realized these were there and were referenced by the square bracket syntax, but I did not know what to call the concept. [...] At this point, it should be clear what a shallow copy means. If the original list's oneth item was a binding to another list object, *that* list object does NOT get copied. I don't like the term inner list, but I don't know if it's incorrect. It's just misleading, since to the slice operation, the fact that it's a list is irrelevant. It's just an object whose binding is to be copied. So the real point here is that there are two distinct copying mechanisms, deep and shallow. The inner list could just have been any other type of object, though if it had been an immutable type it would not have made Peter's original interesting point. -- boB ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
On 04/19/2015 06:28 PM, Joel Goldstick wrote: On Sun, Apr 19, 2015 at 6:23 PM, boB Stepp robertvst...@gmail.com wrote: On Sun, Apr 19, 2015 at 4:05 PM, Dave Angel da...@davea.name wrote: On 04/19/2015 03:08 PM, boB Stepp wrote: Or is the real point that we are adding an abstraction layer so we don't even have to think about where objects are physically stored in RAM? Somebody keeps track, but the address is not necessarily constant, and not necessarily stored in any references. The references (bindings) are abstract, and the details are unimportant to the user. For example, the jython system does not use addresses at all. And an object gets moved around from time to time without its references knowing it. The last sentence in this paragraph has me intrigued. Why would an object, once it has been created, be moved? What practical benefit does doing this give? boB I'm guessing memory management. You want to have large contiguous blocks of memory available for large objects. If a small object is surrounded by available memory, you might want to move it to some smaller empty spot. Good answer. The java jvm garbage collector is free to move blocks around to defrag the free space. FWIW, I'm told the ID value used is a simple integer, that indexes a list containing the actual addresses. So in this case, the binding value is an integer, not an address. -- DaveA ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
In a message of Sun, 19 Apr 2015 19:19:27 -0400, Dave Angel writes: Good answer. The java jvm garbage collector is free to move blocks around to defrag the free space. Correct. FWIW, I'm told the ID value used is a simple integer, that indexes a list containing the actual addresses. Also correct for the majority of cases. You would have to be an expert in GC to care about when this is not correct, which is what I happen to be, but only a pedant would object to what you said. So in this case, the binding value is an integer, not an address. Utterly wrong. The binding value has to be an address. All addresses are integers. I have no clue what you meant by this. Please resubmit and try again. That some of the adresses are indirectly referred to through a list means _nothing_. Very puzzled, your brain is thinking wrong thoughts, but mostly I can guess what is meant, here I cannot at all, Sorry about that, Laura ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
In a message of Sun, 19 Apr 2015 17:23:13 -0500, boB Stepp writes: The last sentence in this paragraph has me intrigued. Why would an object, once it has been created, be moved? What practical benefit does doing this give? boB If you have more than enough memory in your system, you never do this because there is never any need. But if you would like to lay down an array of x by y for the fastest access, you would like it to fit perfectly in memory. Which can mean that the best you can do is pick up a bunch of smaller objects, move them someplace (anyplace) else, and then let your bigger object fit in memory in contiguous space. See 'garbage collection' for more useful ideas. Laura ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
On 20/04/15 00:29, Laura Creighton wrote: So in this case, the binding value is an integer, not an address. Utterly wrong. The binding value has to be an address. I think it depends on how you define 'binding' value. In Python binding is the connection between a name and an object. So in a handle type system the binding between a name and object is the objects handle, which is usually an integer. But the address of the object is the thing the VM uses to fetch the actual object based on the handle. So we have two bindings at work. The one the user sees between the name and the object (which is the handle) and the one the VM uses between the handle and the actual object, which is its address at that particular moment in time. Its the VMs job to keep track of how that second binding changes but the users view of binding (the handle) never changes. All addresses are integers. Usually, but not always. In segmented architectures they are tuples of integers, and in clustered Virtual Memory implementations they may be vectors. But in modern machines they usually resolve to an integer. I have no clue what you meant by this. Please resubmit and try again. That some of the adresses are indirectly referred to through a list means _nothing_. It means a huge amount to the user(programmer), especially if he/she erroneously tries to use a handle as an address. And of course there is a third layer of abstraction here because often times the address that compiler and VM writers think of as the physical address is nothing of the sort its just a bit of virtual space created by the OS which then does another binding to the real RAM chips (or magnetic tape, or optical drive or ferrite core or whatever technology is providing the physical storage) -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
On Sun, Apr 19, 2015 at 4:05 PM, Dave Angel da...@davea.name wrote: On 04/19/2015 03:08 PM, boB Stepp wrote: Or is the real point that we are adding an abstraction layer so we don't even have to think about where objects are physically stored in RAM? Somebody keeps track, but the address is not necessarily constant, and not necessarily stored in any references. The references (bindings) are abstract, and the details are unimportant to the user. For example, the jython system does not use addresses at all. And an object gets moved around from time to time without its references knowing it. The last sentence in this paragraph has me intrigued. Why would an object, once it has been created, be moved? What practical benefit does doing this give? boB ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
Everyone that responded, Thanks very much for the excellent explanations! The distinction between a reference to an object and a seperate copy of the object is quite clear now. --Bill On Apr 18, 2015 1:44 AM, Alan Gauld alan.ga...@btinternet.com wrote: On 18/04/15 04:16, Bill Allen wrote: If I have a list defined as my_list = ['a','b','c'], what is the is differnce between refering to it as my_list or my_list[:]? These seem equivalent to me. Is that the case? Is there any nuance I am missing here? Situations where one form should be used as opposed to the other? Others have already given some good explanations. I'll add a slightly different take. Your confusion starts with your first statement: I have a list defined as my_list = ['a','b','c'] What you should be saying is I have a list defined as ['a', 'b', 'c'] Thats the list object that you are working with. The object is completely separate from the name that you choose to associate with it. You then bound that list to a name: my_list. You could bind it to any number of names but there would still only be one object: foo = my_list bar = foo baz = my_list Now I have 4 names all referring to the same list object. The next source of confusion comes from another mist-statement: differnce between refering to it as my_list or my_list[:] The [:] at the end is an operator that returns a copy of the list. So when you use it you are NOT referring to the original list at all. You are creating a new copy. So if we now take one of our previous names, say foo, and do: foo = my_list[:] foo no longer refers to your original list ['a','b','c'] but to a completely new copy of that list. If you modify my_list the changes will show up when you look at bar and baz as well. But foo will be unchanged my_list[0] = 'z' print baz - prints ['z','b','c'] - the same list as my_list print foo - prints ['a','b','c'] - a different list object Understanding the separation of names from objects in Python is essential to understanding how it works. It is different to many other languages in this respect. And understanding the difference between identity and value is also important. Two completely different objects can have the same value and so appear the same but they are in fact entirely different. Think about two drivers who both buy the exact same model of car. They may look identical, but they are two separate cars. HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
Bill Allen wrote: Everyone that responded, Thanks very much for the excellent explanations! The distinction between a reference to an object and a seperate copy of the object is quite clear now. You can test your newfound knowledge by predicting the output of the following script: a = [1, [x, y], 3] b = a[:] a[1][1] = hello! print(a) # [1, ['x', 'hello!'], 3] print(b) # what will that print? Think twice before you answer. What is copied, what is referenced? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
print(b) will print the original copy of a which b now references which is [1, [x, y], 3] On Apr 18, 2015 7:50 AM, Peter Otten __pete...@web.de wrote: Bill Allen wrote: Everyone that responded, Thanks very much for the excellent explanations! The distinction between a reference to an object and a seperate copy of the object is quite clear now. You can test your newfound knowledge by predicting the output of the following script: a = [1, [x, y], 3] b = a[:] a[1][1] = hello! print(a) # [1, ['x', 'hello!'], 3] print(b) # what will that print? Think twice before you answer. What is copied, what is referenced? ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
On Sat, Apr 18, 2015 at 3:28 PM, Bill Allen walle...@gmail.com wrote: On Apr 18, 2015 7:50 AM, Peter Otten __pete...@web.de wrote: Bill Allen wrote: Everyone that responded, Thanks very much for the excellent explanations! The distinction between a reference to an object and a seperate copy of the object is quite clear now. You can test your newfound knowledge by predicting the output of the following script: a = [1, [x, y], 3] b = a[:] a[1][1] = hello! print(a) # [1, ['x', 'hello!'], 3] print(b) # what will that print? Think twice before you answer. What is copied, what is referenced? print(b) will print the original copy of a which b now references which is [1, [x, y], 3] Uh, oh! You should have checked your work in the interpreter before replying! Peter is being very tricky!! (At least for me...) Look again at that list inside of a list and... boB P.S.: Watch out for top-posting. That tends to get peopled riled. I moved your response back into the normal flow of the interleaved conversation. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
On Apr 18, 2015 4:11 PM, boB Stepp robertvst...@gmail.com wrote: On Sat, Apr 18, 2015 at 3:28 PM, Bill Allen walle...@gmail.com wrote: On Apr 18, 2015 7:50 AM, Peter Otten __pete...@web.de wrote: Bill Allen wrote: Everyone that responded, Thanks very much for the excellent explanations! The distinction between a reference to an object and a seperate copy of the object is quite clear now. You can test your newfound knowledge by predicting the output of the following script: a = [1, [x, y], 3] b = a[:] a[1][1] = hello! print(a) # [1, ['x', 'hello!'], 3] print(b) # what will that print? Think twice before you answer. What is copied, what is referenced? print(b) will print the original copy of a which b now references which is [1, [x, y], 3] Uh, oh! You should have checked your work in the interpreter before replying! Peter is being very tricky!! (At least for me...) Look again at that list inside of a list and... boB P.S.: Watch out for top-posting. That tends to get peopled riled. I moved your response back into the normal flow of the interleaved boB, Ok, just tried it out. In this example b=a and b=a[:] seem to yield the same results even after the change to a, which I do not understand. Should not b be a copy of a and not reflect the change? --bill ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
On Sat, Apr 18, 2015 at 10:03 PM, Bill Allen walle...@gmail.com wrote: On Apr 18, 2015 4:11 PM, boB Stepp robertvst...@gmail.com wrote: On Sat, Apr 18, 2015 at 3:28 PM, Bill Allen walle...@gmail.com wrote: On Apr 18, 2015 7:50 AM, Peter Otten __pete...@web.de wrote: Bill Allen wrote: [...] You can test your newfound knowledge by predicting the output of the following script: a = [1, [x, y], 3] b = a[:] a[1][1] = hello! print(a) # [1, ['x', 'hello!'], 3] print(b) # what will that print? Think twice before you answer. What is copied, what is referenced? print(b) will print the original copy of a which b now references which is [1, [x, y], 3] Uh, oh! You should have checked your work in the interpreter before replying! Peter is being very tricky!! (At least for me...) Look again at that list inside of a list and... [...] Ok, just tried it out. In this example b=a and b=a[:] seem to yield the same results even after the change to a, which I do not understand. Should not b be a copy of a and not reflect the change? Like you, I am on the path to learning Python, so I may or may not get all the technical details correct, but here goes (I'm certain that if I take any misstepps --pun intended!--that the ever-helpful crew of professionals will set us both straight): So far the emphasis on your original question has been on the differences between 'references to objects' and the actual 'objects'. I think that for the purpose of your question you can think about 'objects' as some sort of data stored someplace, though this is not technically correct. When an item of data is stored, it is more efficient to store it once and then from that point on use identifiers (Which we are used to thinking of in most instances as 'variables'.) to point to the storage location of that particular item of data. So when you originally said something like: my_list = ['a', 'b', 'c'] The ['a', 'b', 'c'] is the item of data stored and my_list is the identifier identifying *where* this particular item of data is stored. If you then do things like say: some_other_identifier = my_list then you just created a new identifier which gives the same location information to exactly the same piece of data. However, fortunately (Or, unfortunately, depending on your point of view.) you picked a type of data -- a list -- that is *mutable*. Like mutations in genetics, this just means that this item of data is capable of being changed in place, i.e., where it is actually stored in memory. So if Peter had said instead something like (Using the above statements.): some_other_identifier[1] = 'Hello!' I think you understand now that the originally *identified* list would now be ['a', 'Hello!', 'c'] . But Peter's actual example had a list inside of a list and BOTH of these are objects (In our discussion, items of data.) and BOTH of these have this property of being *mutable*. So he stuck an object inside of another object, so to speak. And this inner object has identifiers associated with it, too! Before Peter changed one of these changeable objects, he had: a = [1, [x, y], 3] b = a[:] Now BOTH a[1] and b[1] now identify the location of the inner list object, [x, y] . Apparently, Python, in its ever efficient memory management fashion, when it creates the new object/piece of data a[:], it sees no need to duplicate the inner list object, [x, y], but instead creates another identifier/pointer/reference to this object's location. But since this inner list object is mutable, when you change y to hello! in b, you also change it in a because both a[1][1] and b[1][1] reference/point to the exact same storage location where this element of the inner list is actually stored. I hope this is helpful, and, if there are any misstepps, that when they are revealed both of our understandings will be enhanced! boB Stepp ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
On 18Apr2015 22:03, Bill Allen walle...@gmail.com wrote: On Apr 18, 2015 4:11 PM, boB Stepp robertvst...@gmail.com wrote: On Sat, Apr 18, 2015 at 3:28 PM, Bill Allen walle...@gmail.com wrote: On Apr 18, 2015 7:50 AM, Peter Otten __pete...@web.de wrote: You can test your newfound knowledge by predicting the output of the following script: a = [1, [x, y], 3] b = a[:] a[1][1] = hello! print(a) # [1, ['x', 'hello!'], 3] print(b) # what will that print? Think twice before you answer. What is copied, what is referenced? print(b) will print the original copy of a which b now references which is [1, [x, y], 3] Uh, oh! You should have checked your work in the interpreter before replying! Peter is being very tricky!! (At least for me...) Look again at that list inside of a list and... [...] Ok, just tried it out. In this example b=a and b=a[:] seem to yield the same results even after the change to a, which I do not understand. Should not b be a copy of a and not reflect the change? Because is it a _shallow_ copy. Doing this: b = a[:] produces a new list, referenced by b, with the _same_ references in it as a. a[1] is a reference to this list: [x, y] b[1] is a reference to the _same_ list. So a[:] makes what is called a shallow copy, a new list with references to the same deeper structure. So this: a[1][1] = hello! affects the original x-y list, and both a and b reference it, so printing either shows the change. By contrast, this: a[1] = [x, hello] _replaces_ the reference in a with a reference to a new, different, list. Sometimes you want a deep copy, where b would have got a copy of the iriginal x-y list. See the copy module's deepcopy function, which supplies this for when it is needed: https://docs.python.org/3/library/copy.html#copy.deepcopy Cheers, Cameron Simpson c...@zip.com.au Draw little boxes with arrows. It helps. - Michael J. Eager ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
On Sat, Apr 18, 2015 at 11:08 PM, Cameron Simpson c...@zip.com.au wrote: Sometimes you want a deep copy, where b would have got a copy of the iriginal x-y list. See the copy module's deepcopy function, which supplies this for when it is needed: https://docs.python.org/3/library/copy.html#copy.deepcopy In this reference, part of it states: Two problems often exist with deep copy operations that don’t exist with shallow copy operations: Recursive objects (compound objects that, directly or indirectly, contain a reference to themselves) may cause a recursive loop. Because deep copy copies everything it may copy too much, e.g., administrative data structures that should be shared even between copies. If I am understanding things correctly, should not that last sentence read instead: ...structures that should *not* be shared even between copies. ??? -- boB ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
On 18Apr2015 23:26, boB Stepp robertvst...@gmail.com wrote: On Sat, Apr 18, 2015 at 11:08 PM, Cameron Simpson c...@zip.com.au wrote: Sometimes you want a deep copy, where b would have got a copy of the iriginal x-y list. See the copy module's deepcopy function, which supplies this for when it is needed: https://docs.python.org/3/library/copy.html#copy.deepcopy In this reference, part of it states: Two problems often exist with deep copy operations that don’t exist with shallow copy operations: Recursive objects (compound objects that, directly or indirectly, contain a reference to themselves) may cause a recursive loop. Because deep copy copies everything it may copy too much, e.g., administrative data structures that should be shared even between copies. If I am understanding things correctly, should not that last sentence read instead: ...structures that should *not* be shared even between copies. ??? No, the text is correct. Suppose you have a graph of objects where some single object A is referenced in mulitple places: /-- O1 v G A \-- O2 ^ If I make a deepcopy of G I want this: /-- O1copy v Gcopy Acopy \-- O2copy ^ i.e. a _single_ copy Acopy of A, again referenced from 2 places in the copied graph. Without deepcopy()'s object tracking you would get something like this: /-- O1copy Acopy1 Gcopy \-- O2copy Acopy2 i.e. two distinct copies of A. Changes to Acopy1 would not affect Acopy2. Cheers, Cameron Simpson c...@zip.com.au They said it couldn't be done/they said nobody could do it/ But he tried the thing that couldn't be done!/He tried - and he couldn't do it. ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
On 19Apr2015 15:09, Cameron Simpson c...@zip.com.au wrote: On 18Apr2015 23:26, boB Stepp robertvst...@gmail.com wrote: On Sat, Apr 18, 2015 at 11:08 PM, Cameron Simpson c...@zip.com.au wrote: Sometimes you want a deep copy, where b would have got a copy of the iriginal x-y list. See the copy module's deepcopy function, which supplies this for when it is needed: https://docs.python.org/3/library/copy.html#copy.deepcopy In this reference, part of it states: Two problems often exist with deep copy operations that don’t exist with shallow copy operations: Recursive objects (compound objects that, directly or indirectly, contain a reference to themselves) may cause a recursive loop. Because deep copy copies everything it may copy too much, e.g., administrative data structures that should be shared even between copies. If I am understanding things correctly, should not that last sentence read instead: ...structures that should *not* be shared even between copies. ??? No, the text is correct. Um, my explaination was incomplete. The first sentence quoted explains why it is necessary for deepcopy keep track of copied objects in order to correctly duplicate objects just once instead of twice or more, let alone in an unbounded way if there is a loop (A-B-C-A). It does this with a memo dictionary of already copied objects so it can know them when it sees them again. The second sentence Because deep copy copies everything it may copy too much, e.g., administrative data structures that should be shared even between copies is an issuewhich is addressed lower down when mentioning the __copy_ and __deepcopy__ methods. And it is not necessarily obvious. Suppose you've got a data structure of objects, which should broadly be copied. However, _internally_, these objects may use some external facility. Further, suppose that facility can't be copied; perhaps it is a reference to a databse or something built around something like that. Like this: G - A - B - _open_database When you deepcopy that data sctructure you want to copy everything, but _not_ copy the external facility object. So for the above example, after the deepcopy you want this: Gcopy - Acopy - Bcopy - _open_database i.e. the same reference to the database, but copies of A and B. If you give G's class a .__deepcopy__ method, then that will be called by deepcopy() to make a copy of G by calling G.__deepcopy__(memodict). And G's class will define its __deepcopy__ method to copy A and B but not _open_database. Most classes do not need this and deepcopy() just copies everything. Does this clarify things? Cheers, Cameron Simpson c...@zip.com.au No wonder these courtroom dramas are so popular. Half the population are lawyers, judges, or some other kind of criminal.- Jake Vest 1986 ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
On 18/04/15 04:16, Bill Allen wrote: If I have a list defined as my_list = ['a','b','c'], what is the is differnce between refering to it as my_list or my_list[:]? These seem equivalent to me. Is that the case? Is there any nuance I am missing here? Situations where one form should be used as opposed to the other? Others have already given some good explanations. I'll add a slightly different take. Your confusion starts with your first statement: I have a list defined as my_list = ['a','b','c'] What you should be saying is I have a list defined as ['a', 'b', 'c'] Thats the list object that you are working with. The object is completely separate from the name that you choose to associate with it. You then bound that list to a name: my_list. You could bind it to any number of names but there would still only be one object: foo = my_list bar = foo baz = my_list Now I have 4 names all referring to the same list object. The next source of confusion comes from another mist-statement: differnce between refering to it as my_list or my_list[:] The [:] at the end is an operator that returns a copy of the list. So when you use it you are NOT referring to the original list at all. You are creating a new copy. So if we now take one of our previous names, say foo, and do: foo = my_list[:] foo no longer refers to your original list ['a','b','c'] but to a completely new copy of that list. If you modify my_list the changes will show up when you look at bar and baz as well. But foo will be unchanged my_list[0] = 'z' print baz - prints ['z','b','c'] - the same list as my_list print foo - prints ['a','b','c'] - a different list object Understanding the separation of names from objects in Python is essential to understanding how it works. It is different to many other languages in this respect. And understanding the difference between identity and value is also important. Two completely different objects can have the same value and so appear the same but they are in fact entirely different. Think about two drivers who both buy the exact same model of car. They may look identical, but they are two separate cars. HTH -- Alan G Author of the Learn to Program web site http://www.alan-g.me.uk/ http://www.amazon.com/author/alan_gauld Follow my photo-blog on Flickr at: http://www.flickr.com/photos/alangauldphotos ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
Ben Finney ben+pyt...@benfinney.id.au writes: Bill Allen walle...@gmail.com writes: If I have a list defined as my_list = ['a','b','c'], what is the is differnce between refering to it as my_list or my_list[:]? ‘my_list’ is a reference to the object you've already described (the existing object ‘['a', 'b', 'c']’). ‘my_list[:]’ is an operation that takes the original object and creates a new one by slicing. In this case, the new object happens to be equal to (but probably not identical to) the original, because of the slice you specified. To demonstrate how identity differs from equality, use the appropriate comparison operators:: $ python3 foo = ['a', 'b', 'c'] bar = foo # Bind the name ‘bar’ to the same object. bar # Show me the object referred to by ‘bar’. ['a', 'b', 'c'] bar == foo# Is the object ‘bar’ equal to the object ‘foo’? True bar is foo# Is ‘bar’ referring to the same object as ‘foo’? True baz = foo[:] # Slice ‘foo’, creating a new list; bind ‘baz’ to that. baz # Show me the object referred to by ‘baz’. ['a', 'b', 'c'] baz == foo# Is the object ‘baz’ equal to the object ‘foo’? True baz is foo# Is ‘baz’ referring to the same object as ‘foo’? False References which compare identical *are the same* object, guaranteed. Object identity almost always implies equality. (“almost always” because some object types have unusual behaviour like “the object is not equal to itself”. Don't fret about that though, these exceptions are clearly defined when you find them.) References which compare equal *may under some conditions* be identical. This is *not* ever a promise, though, and you should never rely on it, not even in the same session of a program. Some of Python's internal optimisations depend on the fact that object equality *does not* imply object identity. If you happen to notice some operations producing the same object at one point in time, but the documentation doesn't promise it, then treat that as an unpredictable implementation detail and don't rely on it in your code. -- \ “The best in us does not require the worst in us: Our love of | `\ other human beings does not need to be nurtured by delusion.” | _o__) —Sam Harris, at _Beyond Belief 2006_ | Ben Finney ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
On 04/17/2015 11:51 PM, Ben Finney wrote: Ben Finney ben+pyt...@benfinney.id.au writes: Bill Allen walle...@gmail.com writes: If I have a list defined as my_list = ['a','b','c'], what is the is differnce between refering to it as my_list or my_list[:]? ‘my_list’ is a reference to the object you've already described (the existing object ‘['a', 'b', 'c']’). ‘my_list[:]’ is an operation that takes the original object and creates a new one by slicing. In this case, the new object happens to be equal to (but probably not identical to) the original, because of the slice you specified. To demonstrate how identity differs from equality, use the appropriate comparison operators:: $ python3 foo = ['a', 'b', 'c'] bar = foo # Bind the name ‘bar’ to the same object. bar # Show me the object referred to by ‘bar’. ['a', 'b', 'c'] bar == foo# Is the object ‘bar’ equal to the object ‘foo’? True bar is foo# Is ‘bar’ referring to the same object as ‘foo’? True baz = foo[:] # Slice ‘foo’, creating a new list; bind ‘baz’ to that. baz # Show me the object referred to by ‘baz’. ['a', 'b', 'c'] baz == foo# Is the object ‘baz’ equal to the object ‘foo’? True baz is foo# Is ‘baz’ referring to the same object as ‘foo’? False References which compare identical *are the same* object, guaranteed. Object identity almost always implies equality. (“almost always” because some object types have unusual behaviour like “the object is not equal to itself”. Don't fret about that though, these exceptions are clearly defined when you find them.) References which compare equal *may under some conditions* be identical. This is *not* ever a promise, though, and you should never rely on it, not even in the same session of a program. Some of Python's internal optimisations depend on the fact that object equality *does not* imply object identity. If you happen to notice some operations producing the same object at one point in time, but the documentation doesn't promise it, then treat that as an unpredictable implementation detail and don't rely on it in your code. Great description, if the OP really wants that level of detail. But I think what he needs to know first is: If you change foo, like (untested) foo[1] = new then you'll see that bar also changes (since it's just another name for the exact same object). bar ['a', 'new', 'c'] But baz does not, since it's bound to a copy of the object. baz ['a', 'b', 'c'\ -- DaveA ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
Bill Allen walle...@gmail.com writes: If I have a list defined as my_list = ['a','b','c'], what is the is differnce between refering to it as my_list or my_list[:]? ‘my_list’ is a reference to the object you've already described (the existing object ‘['a', 'b', 'c']’). ‘my_list[:]’ is an operation that takes the original object and creates a new one by slicing. In this case, the new object happens to be equal to (but probably not identical to) the original, because of the slice you specified. Is there any nuance I am missing here? Situations where one form should be used as opposed to the other? You need to understand, when writing code, whether you are intending to refer to the original object, or to create a new one. Neither is better, they are both common but different operations. -- \ “Our products just aren't engineered for security.” —Brian | `\ Valentine, senior vice-president of Microsoft Windows | _o__)development, 2002 | Ben Finney ___ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: https://mail.python.org/mailman/listinfo/tutor
Re: [Tutor] lists, name semantics
Good evening Bill,
If I have a list defined as my_list = ['a','b','c'], what is the
is differnce between refering to it as my_list or my_list[:]?
These seem equivalent to me. Is that the case? Is there any
nuance I am missing here? Situations where one form should be
used as opposed to the other?
Yes, there is a difference. It can (conceivably) by subtle, but it
is simple.
Let's start with the following. I will call the list by the
variable name l. So, Python ('the computer') has stored the list in
memory. After executing this command, if I want to retrieve that
data, I can call it by the variable named 'l':
l = ['a', 'b', 'c']
Now consider the following two statements.
case A:k = l
case B:m = l[:]
These perform two different operations. The first simply says Hey,
when I refer to variable k, I want it to behave exactly like I had
used the variable named l. The second says Please give me a COPY
of everything contained in the variable named l and let me refer to
that as the variable named m.
What is the nuance of difference? Where are the data stored in
memory? Let's take case A:
l = ['a', 'b', 'c']
k = l
k == l
True
k is l
True
So, you see, not only are k and l refer to the same list contents,
but they point to the same place in computer memory where the list
is stored. The variables named l and k are just different names for
the same thing (data).
Now, let's see what happens when we use case B, copying the list
(strictly speaking you are slicing the entire list, see Python
documentation).
m = l[:]
m == l
True
m is l
False
What's different here? Well, the variables m and l have the same
contents, and are therefore equal (this will compare all elements of
the list for equality). But, m and l are not the same thing (data)!
Though they contain the same data, the list contents are stored in
different places in computer memory.
This subtlety means that you probably do not want to say my_list[:]
unless you really want to use up all the memory to store the same
data twice. You may wish to do that, but given your question about
nuance, I would point out that this is no nuance but a significant
feature which can surprise you later if you do not understand what
is happening with the slicing notation.
Best of luck and enjoy a fried slice of Python!
-Martin
[0]
https://docs.python.org/3/library/stdtypes.html#sequence-types-list-tuple-range
--
Martin A. Brown
http://linux-ip.net/
___
Tutor maillist - Tutor@python.org
To unsubscribe or change subscription options:
https://mail.python.org/mailman/listinfo/tutor
