RE: Popping key causes dict derived from object to revert to object (3/25/2024)
Hello, all. I guess I'm just not seeing this dictionary pop() problem you're having. What version of Python are you using? Is there something I'm missing? If I've made an obvious oversight, please forgive me and explain. Thanks! PyCharm (2023.1.1) REPL running Python 3.12: Mike -- F. M. (Mike) Covington, PhD, PCPP, MCT Automation Consultant, Developer, Instructor -- https://mail.python.org/mailman/listinfo/python-list
RE: Popping key causes dict derived from object to revert to object
Lori, The list comprehension you are thinking of does work if you change things a bit. But it is not a great idea as a main purpose of a dict is that using a hash means things are found in linear time. A comprehension iterates on all values. If you wanted to select just some items to keep in a list, your code could be modified from: dict_list = [d.pop('a') for d in dict_list] to have an IF clause that would specify something like comparing it to the item you do not want to keep. But your idiom might be better done to make another dictionaly, not list with something like: New_dict = {key:value for key in dict if key != "whatever"} Or variants on that. It builds a new dictionary, at nontrivial expense, as compared to using del on an existing dictionary. -Original Message- From: Python-list On Behalf Of Loris Bennett via Python-list Sent: Monday, March 25, 2024 2:56 AM To: python-list@python.org Subject: Re: Popping key causes dict derived from object to revert to object Grant Edwards writes: > On 2024-03-22, Loris Bennett via Python-list wrote: > >> Yes, I was mistakenly thinking that the popping the element would >> leave me with the dict minus the popped key-value pair. > > It does. Indeed, but I was thinking in the context of dict_list = [d.pop('a') for d in dict_list] and incorrectly expecting to get a list of 'd' without key 'a', instead of a list of the 'd['a]'. >> Seem like there is no such function. > > Yes, there is. You can do that with either pop or del: > > >>> d = {'a':1, 'b':2, 'c':3} > >>> d > {'a': 1, 'b': 2, 'c': 3} > >>> d.pop('b') > 2 > >>> d > {'a': 1, 'c': 3} > > > >>> d = {'a':1, 'b':2, 'c':3} > >>> del d['b'] > >>> d > {'a': 1, 'c': 3} > > In both cases, you're left with the dict minus the key/value pair. > > In the first case, the deleted value printed by the REPL because it > was returned by the expression "d.pop('b')" (a method call). > > In the second case is no value shown by the REPL because "del d['b']" > is a statement not an expression. Thanks for pointing out 'del'. My main problem, however, was failing to realise that the list comprehension is populated by the return value of the 'pop', not the popped dict. Cheers, Loris -- This signature is currently under constuction. -- https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
RE: Popping key causes dict derived from object to revert to object
I am glad, Lori, you found a solution another way. Actually, Lori, I think you were right in looking for a built-in method that complements pop() by returning everything else other than the item mentioned. There are philosophical and practical considerations that were no doubt considered and a reality that the functionality did exist albeit not in a pipelined format. Consider a language like LISP which sort of processed lists of things in which a major concept was getting the first item or getting all the rest. Lots of LISP programs had complex structures using CAR() and CDR() nested various ways to say extract the third item as CAR(CDR(CDR(X))) to the point where some commonly used combos became functions with names like CADDR(). There are again many languages where functions or methods are available that include an exclusion variant from a collection perhaps by doing something as simple as using a minus sign to indicate what to remove, as is commonly done in R to remove a column in a data.frame while keeping the order of the remaining columns the same. Lots of languages had similar concepts about ordered data structures but dictionaries may be a bit of something else and initially in Python were not guaranteed to have any kind of order. Python dicts are more like unordered sets. So although there remains a concept of not first/rest but this/rest, I suspect there was some thought about the process that ended in deciding not to supply some functionality. When you use pop() on something like a large dictionary, the original is left intact and is ignored and a copy of a single element is made and returned. To do the opposite and return the rest has two choices. One is to make a big copy of the rest of the dictionary and the other is to use del internally and return the modified dict. The functions you could write do the latter. So why not add one or more methods to do that? Who knows? But I think some may have considered it not needed including some who felt no need for a pipeline method when del would do. Another consideration was the common idiom for iterating on a collection. Besides pop() you can get lists of dictionary entries, keys or values that you can work with and you can even iterate with "for key in dict: ..." Given how many ways common things can be done, and given that adding too many methods has costs including new users not understanding all the nuanced differences, this fairly useful functionality was left out. Unfortunately, I think they were wrong here as instead we hear often from people like you who assume things would work other ways. I still think it would be simple enough to have had a .removekeys(keys) that would work in a pipeline to modify the dict by removing one or more items and perhaps another .removevalues(values) but at some point you may keep adding methods nobody ever uses. The reality is that many trivial one-liner comprehensions can easily do many such things using iteration. But many simply do not work well in pipelined fashion and thus may need to be embedded in a method of your own by subclassing dict or rolling your own. -Original Message- From: Loris Bennett Sent: Monday, March 25, 2024 2:45 AM To: avi.e.gr...@gmail.com Cc: python-list@python.org Subject: Re: Popping key causes dict derived from object to revert to object writes: > Loris wrote: > > "Yes, I was mistakenly thinking that the popping the element would leave > me with the dict minus the popped key-value pair. Seem like there is no > such function." > > Others have tried to explain and pointed out you can del and then use the > changed dict. > > But consider the odd concept of writing your own trivial function. > > def remaining(adict, anitem): > _ = adict.pop(anitem) > # alternatively duse del on dict and item > return adict > > >>>> remaining({"first": 1, "second": 2, "third": 3}, "second") > {'first': 1, 'third': 3} > > > Or do you want to be able to call it as in dict.remaining(key) by > subclassing your own variant of dict and adding a similar method? No, 'del' does indeed do what I wanted, although I have now decided I want something else :-) Nevertheless it is good to know that 'del' exists, so that I don't have to reinvent it. Cheers, Loris -- This signature is currently under constuction. -- https://mail.python.org/mailman/listinfo/python-list
Re: Popping key causes dict derived from object to revert to object
On 2024-03-25, Loris Bennett via Python-list wrote: > Grant Edwards writes: > >> On 2024-03-22, Loris Bennett via Python-list wrote: >> >>> Yes, I was mistakenly thinking that the popping the element would >>> leave me with the dict minus the popped key-value pair. >> >> It does. > > Indeed, but I was thinking in the context of > > dict_list = [d.pop('a') for d in dict_list] > > and incorrectly expecting to get a list of 'd' without key 'a', instead > of a list of the 'd['a]'. So when you say "leave me with", you mean "return the same dictionary with"? There's an important difference between what a function returns and what global/local state it "leaves you with". > Thanks for pointing out 'del'. My main problem, however, was > failing to realise that the list comprehension is populated by the > return value of the 'pop', not the popped dict. OK, so perhaps you weren't execting the original dict objects to be mutated, but rather that the pop method would return a new dict object without the "popped" element. The whole point of the 'pop method is to return the popped value, otherwise it wouldn't be needed. The 'del' statement would suffice. -- https://mail.python.org/mailman/listinfo/python-list
Re: Popping key causes dict derived from object to revert to object
"Michael F. Stemper" writes: > On 25/03/2024 01.56, Loris Bennett wrote: >> Grant Edwards writes: >> >>> On 2024-03-22, Loris Bennett via Python-list wrote: >>> Yes, I was mistakenly thinking that the popping the element would leave me with the dict minus the popped key-value pair. >>> >>> It does. >> Indeed, but I was thinking in the context of >>dict_list = [d.pop('a') for d in dict_list] >> and incorrectly expecting to get a list of 'd' without key 'a', >> instead >> of a list of the 'd['a]'. > I apologize if this has already been mentioned in this thread, but are > you aware of "d.keys()" and "d.values"? > > >>> d = {} > >>> d['do'] = 'a deer, a female deer' > >>> d['re'] = 'a drop of golden sunshine' > >>> d['mi'] = 'a name I call myself' > >>> d['fa'] = 'a long, long way to run' > >>> d.keys() > ['fa', 'mi', 'do', 're'] > >>> d.values() > ['a long, long way to run', 'a name I call myself', 'a deer, a female deer', > 'a drop of golden sunshine'] > >>> Yes, I am, thank you. However, I didn't want either the keys or the values. Instead I wanted to remove a key within a list comprehension. Cheers, Loris PS: "a drop of golden *sun*" - rhymes with "a long, long way to run" -- This signature is currently under constuction. -- https://mail.python.org/mailman/listinfo/python-list
Re: Popping key causes dict derived from object to revert to object
On 2024-03-25, Loris Bennett wrote: > "Michael F. Stemper" writes: > >> On 25/03/2024 01.56, Loris Bennett wrote: >>> Grant Edwards writes: >>> On 2024-03-22, Loris Bennett via Python-list wrote: > Yes, I was mistakenly thinking that the popping the element would > leave me with the dict minus the popped key-value pair. It does. >>> Indeed, but I was thinking in the context of >>>dict_list = [d.pop('a') for d in dict_list] >>> and incorrectly expecting to get a list of 'd' without key 'a', >>> instead >>> of a list of the 'd['a]'. >> I apologize if this has already been mentioned in this thread, but are >> you aware of "d.keys()" and "d.values"? >> >> >>> d = {} >> >>> d['do'] = 'a deer, a female deer' >> >>> d['re'] = 'a drop of golden sunshine' >> >>> d['mi'] = 'a name I call myself' >> >>> d['fa'] = 'a long, long way to run' >> >>> d.keys() >> ['fa', 'mi', 'do', 're'] >> >>> d.values() >> ['a long, long way to run', 'a name I call myself', 'a deer, a female >> deer', 'a drop of golden sunshine'] >> >>> > > Yes, I am, thank you. However, I didn't want either the keys or the > values. Instead I wanted to remove a key within a list comprehension. Do you mean something like: [my_dict[key] for key in my_dict if key != 'a'] ? -- https://mail.python.org/mailman/listinfo/python-list
Re: Popping key causes dict derived from object to revert to object
On 25/03/2024 01.56, Loris Bennett wrote: Grant Edwards writes: On 2024-03-22, Loris Bennett via Python-list wrote: Yes, I was mistakenly thinking that the popping the element would leave me with the dict minus the popped key-value pair. It does. Indeed, but I was thinking in the context of dict_list = [d.pop('a') for d in dict_list] and incorrectly expecting to get a list of 'd' without key 'a', instead of a list of the 'd['a]'. I apologize if this has already been mentioned in this thread, but are you aware of "d.keys()" and "d.values"? >>> d = {} >>> d['do'] = 'a deer, a female deer' >>> d['re'] = 'a drop of golden sunshine' >>> d['mi'] = 'a name I call myself' >>> d['fa'] = 'a long, long way to run' >>> d.keys() ['fa', 'mi', 'do', 're'] >>> d.values() ['a long, long way to run', 'a name I call myself', 'a deer, a female deer', 'a drop of golden sunshine'] >>> -- Michael F. Stemper Exodus 22:21 -- https://mail.python.org/mailman/listinfo/python-list
Re: Popping key causes dict derived from object to revert to object
Grant Edwards writes: > On 2024-03-22, Loris Bennett via Python-list wrote: > >> Yes, I was mistakenly thinking that the popping the element would >> leave me with the dict minus the popped key-value pair. > > It does. Indeed, but I was thinking in the context of dict_list = [d.pop('a') for d in dict_list] and incorrectly expecting to get a list of 'd' without key 'a', instead of a list of the 'd['a]'. >> Seem like there is no such function. > > Yes, there is. You can do that with either pop or del: > > >>> d = {'a':1, 'b':2, 'c':3} > >>> d > {'a': 1, 'b': 2, 'c': 3} > >>> d.pop('b') > 2 > >>> d > {'a': 1, 'c': 3} > > > >>> d = {'a':1, 'b':2, 'c':3} > >>> del d['b'] > >>> d > {'a': 1, 'c': 3} > > In both cases, you're left with the dict minus the key/value pair. > > In the first case, the deleted value printed by the REPL because it > was returned by the expression "d.pop('b')" (a method call). > > In the second case is no value shown by the REPL because "del d['b']" > is a statement not an expression. Thanks for pointing out 'del'. My main problem, however, was failing to realise that the list comprehension is populated by the return value of the 'pop', not the popped dict. Cheers, Loris -- This signature is currently under constuction. -- https://mail.python.org/mailman/listinfo/python-list
Re: Popping key causes dict derived from object to revert to object
writes: > Loris wrote: > > "Yes, I was mistakenly thinking that the popping the element would leave > me with the dict minus the popped key-value pair. Seem like there is no > such function." > > Others have tried to explain and pointed out you can del and then use the > changed dict. > > But consider the odd concept of writing your own trivial function. > > def remaining(adict, anitem): > _ = adict.pop(anitem) > # alternatively duse del on dict and item > return adict > > remaining({"first": 1, "second": 2, "third": 3}, "second") > {'first': 1, 'third': 3} > > > Or do you want to be able to call it as in dict.remaining(key) by > subclassing your own variant of dict and adding a similar method? No, 'del' does indeed do what I wanted, although I have now decided I want something else :-) Nevertheless it is good to know that 'del' exists, so that I don't have to reinvent it. Cheers, Loris -- This signature is currently under constuction. -- https://mail.python.org/mailman/listinfo/python-list
RE: Popping key causes dict derived from object to revert to object
Loris wrote: "Yes, I was mistakenly thinking that the popping the element would leave me with the dict minus the popped key-value pair. Seem like there is no such function." Others have tried to explain and pointed out you can del and then use the changed dict. But consider the odd concept of writing your own trivial function. def remaining(adict, anitem): _ = adict.pop(anitem) # alternatively duse del on dict and item return adict >>> remaining({"first": 1, "second": 2, "third": 3}, "second") {'first': 1, 'third': 3} Or do you want to be able to call it as in dict.remaining(key) by subclassing your own variant of dict and adding a similar method? -- https://mail.python.org/mailman/listinfo/python-list
Re: Popping key causes dict derived from object to revert to object
On 2024-03-22, Loris Bennett via Python-list wrote: > Yes, I was mistakenly thinking that the popping the element would > leave me with the dict minus the popped key-value pair. It does. > Seem like there is no such function. Yes, there is. You can do that with either pop or del: >>> d = {'a':1, 'b':2, 'c':3} >>> d {'a': 1, 'b': 2, 'c': 3} >>> d.pop('b') 2 >>> d {'a': 1, 'c': 3} >>> d = {'a':1, 'b':2, 'c':3} >>> del d['b'] >>> d {'a': 1, 'c': 3} In both cases, you're left with the dict minus the key/value pair. In the first case, the deleted value printed by the REPL because it was returned by the expression "d.pop('b')" (a method call). In the second case is no value shown by the REPL because "del d['b']" is a statement not an expression. -- https://mail.python.org/mailman/listinfo/python-list
Re: Popping key causes dict derived from object to revert to object
Mark Bourne writes: > Loris Bennett wrote: >> Hi, >> I am using SQLAlchemy to extract some rows from a table of 'events'. >> From the call to the DB I get a list of objects of the type >>sqlalchemy.orm.state.InstanceState >> I would like to print these rows to the terminal using the >> 'tabulate' >> package, the documentation for which says >>The module provides just one function, tabulate, which takes a >> list of >>lists or another tabular data type as the first argument, and outputs >>a nicely formatted plain-text table >> So as I understand it, I need to convert the InstanceState-objects >> to, >> say, dicts, in order to print them. However I also want to remove one >> of the keys from the output and assumed I could just pop it off each >> event dict, thus: >>event_dicts = [vars(e) for e in events] >> print(type(event_dicts[0])) >> event_dicts = [e.pop('_sa_instance_state', None) for e in event_dicts] >> print(type(event_dicts[0])) > > vars() returns the __dict__ attribute of the object. It may not be a > good idea to modify that dictionary directly (it will also affect the > object), although it might be OK if you're not going to do anything > else with the original objects. To be safer, you could copy the event > objects: > event_dicts = [dict(vars(e)) for e in events] > or: > event_dicts = [vars(e).copy()] Thanks for making this clear to me. However, in the end I actually decided to use the list comprehension without either 'dict()' or 'vars(). Instead I just select the keys I want and so don't need to pop the unwanted key later and can simultaneously tweak the names of the key for better printing to the terminal. >> However, this prints >> >> >> If I comment out the third line, which pops the unwanted key, I get >> >> >> Why does popping one of the keys cause the elements of the list to >> revert back to their original class? > > As Dieter pointed out, the main problem here is that pop() returns the > value removed, not the dictionary with the rest of the values. You > probably want something more like: > for e in event_dicts: > del e['_sa_instance_state'] > (There's not really any point popping the value if you're not going to > do anything with it - just delete the key from the dictionary) Yes, I was mistakenly thinking that the popping the element would leave me with the dict minus the popped key-value pair. Seem like there is no such function. Cheers, Loris -- This signature is currently under constuction. -- https://mail.python.org/mailman/listinfo/python-list
Re: Popping key causes dict derived from object to revert to object
Loris Bennett wrote: Hi, I am using SQLAlchemy to extract some rows from a table of 'events'. From the call to the DB I get a list of objects of the type sqlalchemy.orm.state.InstanceState I would like to print these rows to the terminal using the 'tabulate' package, the documentation for which says The module provides just one function, tabulate, which takes a list of lists or another tabular data type as the first argument, and outputs a nicely formatted plain-text table So as I understand it, I need to convert the InstanceState-objects to, say, dicts, in order to print them. However I also want to remove one of the keys from the output and assumed I could just pop it off each event dict, thus: event_dicts = [vars(e) for e in events] print(type(event_dicts[0])) event_dicts = [e.pop('_sa_instance_state', None) for e in event_dicts] print(type(event_dicts[0])) vars() returns the __dict__ attribute of the object. It may not be a good idea to modify that dictionary directly (it will also affect the object), although it might be OK if you're not going to do anything else with the original objects. To be safer, you could copy the event objects: event_dicts = [dict(vars(e)) for e in events] or: event_dicts = [vars(e).copy()] However, this prints If I comment out the third line, which pops the unwanted key, I get Why does popping one of the keys cause the elements of the list to revert back to their original class? As Dieter pointed out, the main problem here is that pop() returns the value removed, not the dictionary with the rest of the values. You probably want something more like: for e in event_dicts: del e['_sa_instance_state'] (There's not really any point popping the value if you're not going to do anything with it - just delete the key from the dictionary) -- Mark. -- https://mail.python.org/mailman/listinfo/python-list
Re: Popping key causes dict derived from object to revert to object
Loris Bennett wrote at 2024-3-21 10:56 +0100: > ... >So as I understand it, I need to convert the InstanceState-objects to, >say, dicts, in order to print them. However I also want to remove one >of the keys from the output and assumed I could just pop it off each >event dict, thus: > >event_dicts = [vars(e) for e in events] >print(type(event_dicts[0])) >event_dicts = [e.pop('_sa_instance_state', None) for e in event_dicts] >print(type(event_dicts[0])) > >However, this prints > > `vars` typically returns a `dict`. > This is what you have popped. > >If I comment out the third line, which pops the unwanted key, I get Then you do not change `event_dicts`. You problem likely is: `pop` does not return the `dict` after the removal of a key but the removed value. -- https://mail.python.org/mailman/listinfo/python-list