Re: Pre-PEP: Dictionary accumulator methods
Greg Ewing wrote: Steven Bethard wrote: py def defaultdict(*args, **kwargs): ... defaultfactory, args = args[0], args[1:] which can be written more succinctly as def defaultdict(defaultfactory, *args, **kwargs): ... Not if you want to allow the defaultfactory to be called with a keyword argument 'defaultfactory'. Compare my code: py def defaultdict(*args, **kwargs): ... defaultfactory, args = args[0], args[1:] ... print defaultfactory, args, kwargs ... py defaultdict(dict, defaultfactory=True) type 'dict' () {'defaultfactory': True} with the code you suggested: py def defaultdict(defaultfactory, *args, **kwargs): ... print defaultfactory, args, kwargs ... py defaultdict(dict, defaultfactory=True) Traceback (most recent call last): File interactive input, line 1, in ? TypeError: defaultdict() got multiple values for keyword argument 'defaultfactory' Uncommon, sure, but I'd rather not rule it out if there's no need to. STeVe -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
Steven Bethard wrote: I'd argue that for the same reasons that dict.fromkeys is a dict classmethod, the itertools methods could be iter classmethods (or staticmethods). The basic idea being that it's nice to place the methods associated with a type in that type's definiton. The parallel's a little weaker here because calling iter doesn't always produce objects of type iter: Indeed, I see iter() as being more like len(), which is clearly a function, not a constructor. Making iter() a type and giving it class methods would be strange. -- Greg Ewing, Computer Science Dept, University of Canterbury, Christchurch, New Zealand http://www.cosc.canterbury.ac.nz/~greg -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
Ville Vainio wrote: The issue that really bothers me here is bloating the builtin space. We already have an uncomfortable amount of builtin functions. Maybe what we're really after here is the notion of a builtin module that's pre-imported into the builtin namespace. -- Greg Ewing, Computer Science Dept, University of Canterbury, Christchurch, New Zealand http://www.cosc.canterbury.ac.nz/~greg -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
Ville Vainio wrote: The issue that really bothers me here is bloating the builtin space. We already have an uncomfortable amount of builtin functions. Of course the additions that have been suggested would not pollute the builtin namespace, but they would still be there, taking space. I'd rather see a more modular and 'slimmer' Python, what with the advent of Python for S60 and other embedded uses. Certainly a valid point. How would you feel about adding just a select few itertools functions, perhaps just islice, chain and tee? These functions provide the operations that exist for lists but don't, by default, exist for iterators: slicing, concatenation and copying. STeVe -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
Ville Vainio wrote: A minimal set would not be that offensive, yes. But then we would have two places to look for itertools functionality, which may not be desirable. True, though this is currently necessary with str objects if you want to use, say string.maketrans, so it's not without some precedent. If it's necessary to leave anything in itertools, my suggestion would be that the documentation for the iter type have a clear see also link to the itertools module. One thing that might be worth keeping in mind is that some of itertools functionality is going to become obsolete come py3k (izip-zip), and some is already (imap). At least such operations should not be dumped into the builtin iter. Yeah, maps and filters are basically obsolete as of generator expressions. The list of itertools functions that don't seem obsolete (and won't be made obsolete by Python 3.0): chain count cycle dropwhile groupby islice repeat takewhile tee As I suggested, I think that chain, islice and tee are tightly coupled with iterator objects, providing concatenation, slicing and copying operations. This leaves: count cycle dropwhile groupby repeat takewhile None of these really have analogs in sequence objects, so I consider them less tightly tied to iter. I'd probahbly say that these are more along the lines of alternate constructors, ala dict.fromkeys. While they're certainly useful at times, I'd be happy enough to leave them in itertools if that was the general feeling. Of course I guess I'd be happy enough to leave everything in itertools if that was the general feeling (or the BDFL pronouncement). ;) STeVe -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
[Jack Diederich] itertools to iter transition, huh? I slipped that one in, I mentioned it to Raymond at PyCon and he didn't flinch. It would be nice not to have to sprinkle 'import itertools as it' in code. iter could also become a type wrapper instead of a function, so an iter instance could be a wrapper that figures out whether to call .next or __getitem__ depending on it's argument. for item in iter(mylist).imap: print item or for item in iter.imap(mylist): print item [Steven Bethard] Very cool idea. I think the transition from itertools.XXX(iterable, *args, **kwargs) to iter.XXX(iterable, *args, **kwargs) ought to be pretty easy. Just to make sure you guys can live with your proposed syntax, trying using it for a month or so and report back on whether the experience was pleasant. Try dropping the following into your setup.py def wrapiter(): import __builtin__, itertools orig = __builtin__.iter def iter(*args): return orig(*args) for name in ('__doc__', '__name__'): setattr(iter, name, getattr(orig, name)) vars(iter).update(vars(itertools)) __builtin__.iter = iter wrapiter() If the experience works out, then all you're left with is the trivial matter of convincing Guido that function attributes are a sure cure for the burden of typing import statements. Raymond Hettinger -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
Raymond == Raymond Hettinger [EMAIL PROTECTED] writes: Raymond If the experience works out, then all you're left with is Raymond the trivial matter of convincing Guido that function Raymond attributes are a sure cure for the burden of typing Raymond import statements. For one thing, it would make it harder to find the functions from the docs. It's easy to find the doc for 'itertools', but iter object methods would require browsing that infamous Chapter 2 of the documentation... Apart from that, I don't really see the advantage in moving away from itertools. -- Ville Vainio http://tinyurl.com/2prnb -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
Ville Vainio wrote: Raymond == Raymond Hettinger [EMAIL PROTECTED] writes: Raymond If the experience works out, then all you're left with is Raymond the trivial matter of convincing Guido that function Raymond attributes are a sure cure for the burden of typing Raymond import statements. For one thing, it would make it harder to find the functions from the docs. It's easy to find the doc for 'itertools', but iter object methods would require browsing that infamous Chapter 2 of the documentation... Well, it would only make them as hard to find as, say, dict.fromkeys, which is probably the best parallel here. Of course iter would have to be documented as a builtin type. I don't find the argument builtin type methods are hard to find convincing -- the solution here is to fix the documentation, not refuse to add builtin types. Apart from that, I don't really see the advantage in moving away from itertools. True it's not a huge win. But I'd argue that for the same reasons that dict.fromkeys is a dict classmethod, the itertools methods could be iter classmethods (or staticmethods). The basic idea being that it's nice to place the methods associated with a type in that type's definiton. The parallel's a little weaker here because calling iter doesn't always produce objects of type iter: py class C(object): ... def __iter__(self): ... yield 1 ... py iter(C()) generator object at 0x011805A8 But note that iter does produce 'iterator' objects for the old __getitem__ protocol: py class C(object): ... def __getitem__(self, index): ... if index 5: ... raise IndexError ... return index ... py iter(C()) iterator object at 0x01162EF0 I guess the real questions are[1]: * How much does iter feel like a type? * How closely are the itertools functions associated with iter? STeVe [1] There's also the question of how much you believe in OO tenets like functions closely associated with a type should be members of that type... -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
Steven Bethard wrote: Ville Vainio wrote: Raymond == Raymond Hettinger [EMAIL PROTECTED] writes: Raymond If the experience works out, then all you're left with is Raymond the trivial matter of convincing Guido that function Raymond attributes are a sure cure for the burden of typing Raymond import statements. For one thing, it would make it harder to find the functions from the docs. It's easy to find the doc for 'itertools', but iter object methods would require browsing that infamous Chapter 2 of the documentation... Well, it would only make them as hard to find as, say, dict.fromkeys, which is probably the best parallel here. Of course iter would have to be documented as a builtin type. I don't find the argument builtin type methods are hard to find convincing -- the solution here is to fix the documentation, not refuse to add builtin types. Apart from that, I don't really see the advantage in moving away from itertools. True it's not a huge win. But I'd argue that for the same reasons that dict.fromkeys is a dict classmethod, the itertools methods could be iter classmethods (or staticmethods). The basic idea being that it's nice to place the methods associated with a type in that type's definiton. The parallel's a little weaker here because calling iter doesn't always produce objects of type iter: py class C(object): ... def __iter__(self): ... yield 1 ... py iter(C()) generator object at 0x011805A8 But note that iter does produce 'iterator' objects for the old __getitem__ protocol: py class C(object): ... def __getitem__(self, index): ... if index 5: ... raise IndexError ... return index ... py iter(C()) iterator object at 0x01162EF0 I guess the real questions are[1]: * How much does iter feel like a type? * How closely are the itertools functions associated with iter? STeVe [1] There's also the question of how much you believe in OO tenets like functions closely associated with a type should be members of that type... While we're on the topic, what do you think of having unary, non-summary builtins automatically map themselves when called with an iterable that would otherwise be an illegal argument: e.g., int(iterable) - (int(i) for i in iterable) ord(iterable) - (ord(i) for i in iterable) This would be unambiguous, I think, in the cases of bool, int, callable, chr, float, hex, id, long, oct, ord, vars... It would shorten the common cases of: for char in somestring: ordchar = ord(char) # do something with ordchar, but not char to for ordchar in ord(somestring): ... It would not work for summarizing functions or those that can accept an iterable today e.g., len, repr Michael -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
Steven Bethard [EMAIL PROTECTED] wrote: [snip] I guess the real questions are[1]: * How much does iter feel like a type? * How closely are the itertools functions associated with iter? STeVe [1] There's also the question of how much you believe in OO tenets like functions closely associated with a type should be members of that type... I would answer positively for both: iter does feel like a type conceptually and (most, if not all) itertools would be suitable methods for such a type. Here I am referring to 'type' more as an interface (or protocol; i'm not sure of the difference) rather than a concrete class, so whether the result of iter is an iterator or a generator object is of little importance as long as it works as expected (that it, whether it makes calls to next() or __getitem__() becomes a hidden implementation detail). If iter was a type, it would also be neat to replace some itertool callables with special methods, as it has been mentioned in another thread (http://tinyurl.com/6mmmf), so that: iter(x)[a:b:c] := itertools.islice(iter(x),a,b,c) iter(x) + iter(y) := itertools.chain(iter(x), iter(y)) iter(x) * 3 := itertools.chain(* itertools.tee(iter(x), 3)) George -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
On Tue, Mar 29, 2005 at 12:38:42PM +0300, Ville Vainio wrote: Raymond == Raymond Hettinger [EMAIL PROTECTED] writes: Raymond If the experience works out, then all you're left with is Raymond the trivial matter of convincing Guido that function Raymond attributes are a sure cure for the burden of typing Raymond import statements. For one thing, it would make it harder to find the functions from the docs. It's easy to find the doc for 'itertools', but iter object methods would require browsing that infamous Chapter 2 of the documentation... Apart from that, I don't really see the advantage in moving away from itertools. I only use itertools when I have to currently, which isn't necessarily bad (premature optimization etc) but I do use lists when I just need an iterator - simply because 'list()' is easier to type than '^spacehome^n^nimport itertools as itCR^x^x' (emacsen to mark HERE, jump to the top, import itertools, and jump back). If itertools methods were handier I'd use them when I just want to iterate. As an anecdote I use generator comprehensions[1] more often than list comprehensions. I'll give the builtin manipulations a try but since I have to deal with many machines I can't promise to flex it much. -jack [1] aside, I didn't care too much about upgrading machines 2.2 = 2.3, but when 2.4 came along with set as a builtin and generator comprehensions it was compelling. -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
On Tue, 29 Mar 2005 11:32:33 -0800, Michael Spencer [EMAIL PROTECTED] wrote: [...] While we're on the topic, what do you think of having unary, non-summary builtins automatically map themselves when called with an iterable that would otherwise be an illegal argument: That last otherwise is pretty important for strings as in int('1234') ;-) e.g., int(iterable) - (int(i) for i in iterable) ord(iterable) - (ord(i) for i in iterable) This would be unambiguous, I think, in the cases of bool, int, callable, chr, float, hex, id, long, oct, ord, vars... It would shorten the common cases of: for char in somestring: ordchar = ord(char) # do something with ordchar, but not char But wouldn't you really currently write the - form from above? I.e., for ordchar in (ord(c) for c in somestring): ... to compare with to for ordchar in ord(somestring): ... So it's not _that_ much shorter ;-) It would not work for summarizing functions or those that can accept an iterable today e.g., len, repr I like concise expression, so I'm willing to try it. I guess it would be enough to override __builtins__ to get a taste, e.g., (not thought through): class itint(int): ... oldint = __builtins__.int ... def __new__(cls, arg): ... try: return cls.oldint(arg) ... except (TypeError, ValueError): ... oi = cls.oldint ... return (oi(item) for item in arg) ... __builtins__.int = itint int('1234') 1234 for x in int('1 23 456'.split()): print x, ... 1 23 456 for x in int(range(1,8,2)): print x, ... 1 3 5 7 for x in int('123x'): print x, ... 1 2 3 Traceback (most recent call last): File stdin, line 1, in ? File stdin, line 7, in generator expression ValueError: invalid literal for int(): x Hm, ... ;-) Regards, Bengt Richter -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
Terry Reedy wrote: But if classmethods are intended to provide alternate constructors But I do not remember that being given as a reason for classmethod(). But I am not sure what was. Well I haven't searched thoroughly, but I know one place that it's referenced is in descrintro[1]: Factoid: __new__ is a static method, not a class method. I initially thought it would have to be a class method, and that's why I added the classmethod primitive. Unfortunately, with class methods, upcalls don't work right in this case, so I had to make it a static method with an explicit class as its first argument. Ironically, there are now no known uses for class methods in the Python distribution (other than in the test suite). However, class methods are still useful in other places, for example, to program inheritable alternate constructors. Not sure if this is the only reason though, and even if it is, it might not be entirely applicable because while the itertools functions may be supplying alternate constructors, it's not clear why anyone would subclass iter[2], so the constructors aren't likely to be inherited. STeVe [1] http://www.python.org/2.2.3/descrintro.html#__new__ [2] That is, in the simple case, where iter is still basically a factory function, not a type wrapper. -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
On Sun, Mar 27, 2005 at 02:20:33PM -0700, Steven Bethard wrote: Michele Simionato wrote: I am surprised nobody suggested we put those two methods into a separate module (say dictutils or even UserDict) as functions: from dictutils import tally, listappend tally(mydict, key) listappend(mydict, key, value) Sorry to join the discussion so late (I've been away from my email for a week) but this was exactly my reaction too. In fact, I have a 'dicttools' module with similar methods in it: snipped I like this approach, it will give us a chance to test tweak the signature before hanging it off dict proper. It feels similar to the strings module to str transition, sets module to set builtin, and itertools module to iter transition. itertools to iter transition, huh? I slipped that one in, I mentioned it to Raymond at PyCon and he didn't flinch. It would be nice not to have to sprinkle 'import itertools as it' in code. iter could also become a type wrapper instead of a function, so an iter instance could be a wrapper that figures out whether to call .next or __getitem__ depending on it's argument. for item in iter(mylist).imap: print item or for item in iter.imap(mylist): print item I haven't digested that too much, just a thought. -jackdied -- http://mail.python.org/mailman/listinfo/python-list
itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
Jack Diederich wrote: itertools to iter transition, huh? I slipped that one in, I mentioned it to Raymond at PyCon and he didn't flinch. It would be nice not to have to sprinkle 'import itertools as it' in code. iter could also become a type wrapper instead of a function, so an iter instance could be a wrapper that figures out whether to call .next or __getitem__ depending on it's argument. for item in iter(mylist).imap: print item or for item in iter.imap(mylist): print item Very cool idea. I think the transition from itertools.XXX(iterable, *args, **kwargs) to iter.XXX(iterable, *args, **kwargs) ought to be pretty easy. The transition from here to iter(iterable).XXX(*args, **kwargs) seems like it might be more complicated though -- iter would have to return a proxy object instead of the object returned by __iter__[1]. I guess it already does that for objects that support only the __getitem__ protocol though, so maybe it's not so bad... STeVe [1] And you'd probably also want to special-case this so that if iter() was called on an object that's already an instance of iter, that the object itself was returned, not a proxy. -- http://mail.python.org/mailman/listinfo/python-list
Re: itertools to iter transition (WAS: Pre-PEP: Dictionary accumulator methods)
On Mon, Mar 28, 2005 at 10:28:29AM -0700, Steven Bethard wrote: Jack Diederich wrote: itertools to iter transition, huh? I slipped that one in, I mentioned it to Raymond at PyCon and he didn't flinch. It would be nice not to have to sprinkle 'import itertools as it' in code. iter could also become a type wrapper instead of a function, so an iter instance could be a wrapper that figures out whether to call .next or __getitem__ depending on it's argument. for item in iter(mylist).imap: print item or for item in iter.imap(mylist): print item Very cool idea. I think the transition from itertools.XXX(iterable, *args, **kwargs) to iter.XXX(iterable, *args, **kwargs) ought to be pretty easy. The transition from here to iter(iterable).XXX(*args, **kwargs) seems like it might be more complicated though -- iter would have to return a proxy object instead of the object returned by __iter__[1]. I guess it already does that for objects that support only the __getitem__ protocol though, so maybe it's not so bad... I only included making iter a type to make it more symmetric with str being a type. iter is currently a function, as a practical matter I wouldn't mind if it doubled as a namespace but that might make others flinch. [1] And you'd probably also want to special-case this so that if iter() was called on an object that's already an instance of iter, that the object itself was returned, not a proxy. -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Jack Diederich wrote: On Sun, Mar 27, 2005 at 02:20:33PM -0700, Steven Bethard wrote: Michele Simionato wrote: I am surprised nobody suggested we put those two methods into a separate module (say dictutils or even UserDict) as functions: from dictutils import tally, listappend tally(mydict, key) listappend(mydict, key, value) Sorry to join the discussion so late (I've been away from my email for a week) but this was exactly my reaction too. In fact, I have a 'dicttools' module with similar methods in it: snipped I like this approach, it will give us a chance to test tweak the signature before hanging it off dict proper. It feels similar to the strings module to str transition, sets module to set builtin, and itertools module to iter transition. itertools to iter transition, huh? I slipped that one in, I mentioned it to Raymond at PyCon and he didn't flinch. It would be nice not to have to sprinkle 'import itertools as it' in code. Not that that is such a pain, but accessing itertools functions from an outlying module seems somewhat incompatible with putting iterative approaches center stage. iter could also become a type wrapper instead of a function, so an iter instance could be a wrapper that figures out whether to call .next or __getitem__ depending on it's argument. for item in iter(mylist).imap: print item Also opening the door for iter to be subclassed. For example, could listiter become a specialization of iter - one that uses __getitem__ and which could allow reverse iteration? or for item in iter.imap(mylist): print item I haven't digested that too much, just a thought. -jackdied A very good thought IMO. Michael -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Michele Simionato wrote: I am surprised nobody suggested we put those two methods into a separate module (say dictutils or even UserDict) as functions: from dictutils import tally, listappend tally(mydict, key) listappend(mydict, key, value) Sorry to join the discussion so late (I've been away from my email for a week) but this was exactly my reaction too. In fact, I have a 'dicttools' module with similar methods in it: # like tally but without ability to set increment def counts(iterable, key=None): result = {} for item in iterable: # apply key function if necessary if key is None: k = item else: k = key(item) # increment key's count try: result[k] += 1 except KeyError: result[k] = 1 return result # like listappend but with the option to use key and value funcs def groupby(iterable, key=None, value=None): result = {} for item in iterable: # apply key function if necessary if key is None: k = item else: k = key(item) # apply value function if necessary if value is None: v = item else: v = value(item) # append value to key's list try: result[k].append(v) except KeyError: result[k] = [v] return result These two functions have covered all my use cases for tally and listappend -- I always want to perform the increments or list appends over a sequence of values, so having functions that operate on sequences covers all my needs. STeVe -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Michele Simionato wrote: FWIW, here is my take on the defaultdict approach: def defaultdict(defaultfactory, dictclass=dict): class defdict(dictclass): def __getitem__(self, key): try: return super(defdict, self).__getitem__(key) except KeyError: return self.setdefault(key, defaultfactory()) return defdict d = defaultdict(int)() d[x] += 1 d[x] += 1 d[y] += 1 print d d = defaultdict(list)() d[x].append(1) d[x].append(2) d[y].append(1) print d Michele Simionato Very pretty! =) It does mean, however, that if the defaultfactory function takes any arguments, you have to wrap the function to make this work. I'd probably prefer something like: py def defaultdict(*args, **kwargs): ... defaultfactory, args = args[0], args[1:] ... class defdict(dict): ... def __getitem__(self, key): ... try: ... return super(defdict, self).__getitem__(key) ... except KeyError: ... return self.setdefault(key, defaultfactory( ... *args, **kwargs)) ... return defdict ... py d = defaultdict(int)() py d['x'] += 1 py d['x'] += 1 py d['y'] += 1 py d {'y': 1, 'x': 2} py d = defaultdict(list, [0])() py d['x'].append(1) py d['x'].append(2) py d['y'].append(1) py d {'y': [0, 1], 'x': [0, 1, 2]} That said, I still think a dictools module is a better solution to this problem. STeVe -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Greg Ewing wrote: Michele Simionato wrote: def defaultdict(defaultfactory, dictclass=dict): class defdict(dictclass): def __getitem__(self, key): try: return super(defdict, self).__getitem__(key) except KeyError: return self.setdefault(key, defaultfactory()) return defdict That looks really nice! I'd prefer a more elegant name than 'defaultdict', though. How about 'table'? By obvious analogy with Icon (where the dictionary-like object was created with the option of a default value) this gets my +1. regards Steve -- Meet the Python developers and your c.l.py favorites March 23-25 Come to PyCon DC 2005 http://www.pycon.org/ Steve Holden http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
R.H.: The setdefault() method would continue to exist but would likely not make it into Py3.0. I agee to remove the setdefault. I like the new count method, but I don't like the appendlist method, because I think it's too much specilized. I too use sets a lot; recently I've suggested to add a couple of set methods to dicts (working on the keys): intersection() and difference(). Bearophile -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
On Sat, 19 Mar 2005 01:24:57 GMT, rumours say that Raymond Hettinger [EMAIL PROTECTED] might have written: I would like to get everyone's thoughts on two new dictionary methods: def count(self, value, qty=1): try: self[key] += qty except KeyError: self[key] = qty def appendlist(self, key, *values): try: self[key].extend(values) except KeyError: self[key] = list(values) Both are useful and often needed, so I am +1 on adding such functionality. However, I am -0 on adding methods to dict. I believe BJörn Lindqvist suggested a subtype of dict instead, which feels more right. I believe this is a type of 'bag' collection, and it could go to the collections module. The default argument 99% of the time is the same for all calls to setdefault of a specific instance. So I would suggest that the default argument should be an attribute of the bag instance, given at instance creation. And since unbound methods are going to stay, we can use the accumulator method as a default argument (ie int.__add__ or list.append) Based on the above, I would suggest something like the following implementation, waiting criticism on names, algorithm or applicability:) .class bag(dict): .def __init__(self, accumulator=int.__add__): .self.accumulator = accumulator . .# refinement needed for the following .self.accu_class = accumulator.__objclass__ . .# if there was an exception, probably the accumulator .# provided was not appropriate . .def accumulate(self, key, value): .try: .old_value = self[key] .except KeyError: .self[key] = old_value = self.accu_class() .new_value = self.accumulator(old_value, item) . .# and this needs refinement .if new_value is not None: # method of immutable object .self[key] = new_value This works ok for int.__add__ and list.append. PS I wrote these more than 36 hours ago, and before having read the so-far downloaded messages of the thread. I kept on reading and obviously others thought the same too (default argument at initialisation). What the heck, Bengt at least could like the class method idea :) -- TZOTZIOY, I speak England very best. Be strict when sending and tolerant when receiving. (from RFC1958) I really should keep that in mind when talking with people, actually... -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Raymond, I am +1 for both suggestions, tally and appendlist. Extended: Also, in all of my code base, I've not run across a single opportunity to use something like unionset(). This is surprising because I'm the set() author and frequently use set based algorithms.Your example was a good one and I can also image a graph represented as a dictionary of sets. Still, I don't mind writing out the plain Python for this one if it only comes up once in a blue moon. I am more than sure you are right about this. But, please keep in mind that you and we all have come very, very accustomed to using lists for everything and the kitchen sink in Python. Lists where there from the beginning of Python, and even before the birth of Python; very powerfull, well implemented and theoretically well founded datastructures - I heared there is a whole language based on list processing. *pun intended* sets on the other hand --- I know, in mathematics they have a deep, long history. But in programming? Yeah, in SQL and ABAP/4 basically you are doing set operations on every join. But its rather uncommon to call it set. With 2.3 Python grew a set module. And, in ONLY ONE revision it got promoted to a buildin type - a honour only those who read c.l.p.d. regualary can value correctly. And sets are SO NATURALLY for a lot of problems ... I never thought of replacing my list in dict constructs with sets before, BUT there are 90% of problem domains where order is not important, AND fast membership testing is a unique sales point. So please for best impressions: let us have a look at our code, where we use the dict.setdefault(key,[]).append() idiom, where it could be replaced to a better effectivity with dict.setdefault(key,set()).add() If it is less than 60%, forget it. If it is more Harald -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Raymond Hettinger wrote: I would like to get everyone's thoughts on two new dictionary methods: def count(self, value, qty=1): try: self[key] += qty except KeyError: self[key] = qty def appendlist(self, key, *values): try: self[key].extend(values) except KeyError: self[key] = list(values) -0.9 Not impressed, they feel too specific for being builtin dictionary methods and give the impression of just trying to save a few lines here and there. I don't feel the names convey the functionality of the methods either. I know there's the speed argument but I'd rather not have these on the dict at all. +0.1 I sort of feel a slight need for this. But where would you stop? What if people decrement lots? what if next there's a need for division? How would you determine how you add the item to the key if it already exists? In a general way: mydict.set(key, value=None, default=None, how=operator.setitem) This feels slightly better as it's not tied down to what sort of item you're setting. But: for word in words: mydict.set(word, 1, 0, operator.add) I dunno, feels a bit verbose maybe. The setdefault() method would continue to exist but would likely not make it into Py3.0. I agree that setdefault is wart though. And for dict.default = value: (Quoth RON): With a preset default mode, it then becomes possible to inadvertently create default values that will cause problems without knowing it. So then we have to remember to change the setdefault value to None or null to avoid problems. Ouch! Agreed, -1 there then. PROBLEMS BEING SOLVED - The readability issues with the existing constructs are: * They are awkward to teach, create, read, and review. * Their wording tends to hide the real meaning (accumulation). * The meaning of setdefault() 's method name is not self-evident. I feel this only really applies for setdefault (which I wouldn't be sorry to see the back of). And your examples: d[key] = d.get(key, 0) + qty d.setdefault(key, []).extend(values) Would better be written in a long-handed fashion anyway as per the implementations were suggested: try: d[key] += qty except KeyError: d[key] = 0 Yeah, yeah, I know, speed. But not like this. Sorry. -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
FWIW, here is my take on the defaultdict approach: def defaultdict(defaultfactory, dictclass=dict): class defdict(dictclass): def __getitem__(self, key): try: return super(defdict, self).__getitem__(key) except KeyError: return self.setdefault(key, defaultfactory()) return defdict d = defaultdict(int)() d[x] += 1 d[x] += 1 d[y] += 1 print d d = defaultdict(list)() d[x].append(1) d[x].append(2) d[y].append(1) print d Michele Simionato -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Michele Simionato [EMAIL PROTECTED] wrote: FWIW, here is my take on the defaultdict approach: def defaultdict(defaultfactory, dictclass=dict): class defdict(dictclass): def __getitem__(self, key): try: return super(defdict, self).__getitem__(key) except KeyError: return self.setdefault(key, defaultfactory()) return defdict d = defaultdict(int)() d[x] += 1 d[x] += 1 d[y] += 1 print d d = defaultdict(list)() d[x].append(1) d[x].append(2) d[y].append(1) print d Michele Simionato Best solution so far. If it wasn't for the really bad decision to add the dict(**kwargs) constructor, I'd love to see something like d = dict(valType=int) d[x] += 1 George -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Raymond Hettinger wrote: I would like to get everyone's thoughts on two new dictionary methods: def count(self, value, qty=1): def appendlist(self, key, *values): -1.0 When I need these, I just use subtype recipes. They seem way too special-purpose for the base dict type. class Counter(dict): def __iadd__(self, other): if other in self: self[other] += 1 else: self[other] = 1 return self c = Counter() for item in items: c += item class Collector(dict): def add(self, key, value): if key in self: self[key].append(value) else: self[key] = [value] c = Collector() for k,v in items: c.add(k, v) Cheers, Evan @ 4-am -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Michele Simionato wrote: def defaultdict(defaultfactory, dictclass=dict): class defdict(dictclass): def __getitem__(self, key): try: return super(defdict, self).__getitem__(key) except KeyError: return self.setdefault(key, defaultfactory()) return defdict That looks really nice! I'd prefer a more elegant name than 'defaultdict', though. How about 'table'? -- Greg Ewing, Computer Science Dept, University of Canterbury, Christchurch, New Zealand http://www.cosc.canterbury.ac.nz/~greg -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
I agree -- I find myself NEEDING sets more and more. I use them with this idiom quite often. Once they become more widely available (i.e. Python 2.3 is installed everywhere), I will use them almost as much as lists I bet. So any solution IMO needs to at least encompass sets. But preferably something like the Dict with Default approach which encompasses all possibilities. Roose I am more than sure you are right about this. But, please keep in mind that you and we all have come very, very accustomed to using lists for everything and the kitchen sink in Python. Lists where there from the beginning of Python, and even before the birth of Python; very powerfull, well implemented and theoretically well founded datastructures - I heared there is a whole language based on list processing. *pun intended* sets on the other hand --- I know, in mathematics they have a deep, long history. But in programming? Yeah, in SQL and ABAP/4 basically you are doing set operations on every join. But its rather uncommon to call it set. With 2.3 Python grew a set module. And, in ONLY ONE revision it got promoted to a buildin type - a honour only those who read c.l.p.d. regualary can value correctly. And sets are SO NATURALLY for a lot of problems ... I never thought of replacing my list in dict constructs with sets before, BUT there are 90% of problem domains where order is not important, AND fast membership testing is a unique sales point. So please for best impressions: let us have a look at our code, where we use the dict.setdefault(key,[]).append() idiom, where it could be replaced to a better effectivity with dict.setdefault(key,set()).add() If it is less than 60%, forget it. If it is more Harald -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Paul Rubin wrote: If the compiler can do some type inference, it can optimize out those multiple calls pretty straightforwardly. It can be tipped like that: di = dict(int) di.setdefault(0) di[key] += 1 dl = dict(list) dl.setdefault([]) dl.append(word) dl.extend(mylist) But the point is that if method not found in dict it delegated to container type specified in constructor. It solves dict specialization without bloating dict class and is generic. Mike -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Mike Rovner [EMAIL PROTECTED] writes: It can be tipped like that: di = dict(int) di.setdefault(0) di[key] += 1 ... But the point is that if method not found in dict it delegated to container type specified in constructor. It solves dict specialization without bloating dict class and is generic. Hey, I like that. I'd let the default be an optional extra arg to the constructor: di = dict(int, default=0) di[key] += 1 without the setdefault. I might even add optional type checking: di = dict(int, default=0, typecheck=True) di[key] = 'foo'# raises TypeError -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Raymond Hettinger wrote: The rationale is to replace the awkward and slow existing idioms for dictionary based accumulation: d[key] = d.get(key, 0) + qty d.setdefault(key, []).extend(values) How about the alternative approach of allowing the user to override the action to be taken when accessing a non-existent key? d.defaultValue(0) and the accumulation becomes: d[key] += 1 and: d.defaultValue(function=list) would allow a safe: d[key].extend(values) -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Raymond Hettinger wrote: I would like to get everyone's thoughts on two new dictionary methods: def count(self, value, qty=1): try: self[key] += qty except KeyError: self[key] = qty def appendlist(self, key, *values): try: self[key].extend(values) except KeyError: self[key] = list(values) They look as a special-case to me. They don't solve the problem for lists of sets or lists of deques for instance, not to mention other possible user-defined containers. defaultdicts look to me as a solution that is more elegant and solves more problems. What is the problem with them? -- Ciao, Matteo -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Mike Rovner wrote: Paul Rubin wrote: If the compiler can do some type inference, it can optimize out those multiple calls pretty straightforwardly. It can be tipped like that: di = dict(int) di.setdefault(0) di[key] += 1 Interesting, but why do you need to give the int type to the constructor? dl = dict(list) dl.setdefault([]) dl.append(word) dl.extend(mylist) I don't quite understand that. Which dict item are you extending? Don't you need something like dl[key].append(word) ? Anyway, using `setdefault' as the method name is quite confusing, although yours is IMHO a much better behavior given the name ;) So what about `setdefaultvalue'? Reinhold -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
John Machin wrote: Reinhold Birkenfeld wrote: John Machin wrote: Are you kidding? If you know what set and default means, you will be able to guess what setdefault means. Same goes for updateBy. No I'm not kidding -- people from some cultures have no difficulty at all in mentally splitting up words like setdefault or the German equivalent of Danubesteamnavigationcompany'sdirector'swife; others from other cultures where agglutinisation is not quite so rife will have extreme difficulty. Okay - as I'm German I might be preoccupied on this matter wink Reinhold -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
George Sakkis wrote: -1 form me. I'm not very glad with both of them ( not a naming issue ) because i think that the dict type should offer only methods that apply to each dict whatever it contains. count() specializes to dict values that are addable and appendlist to those that are extendable. Why not subtractable, dividable or right-shiftable? Because of majority approval? I'm mot a speed fetishist and destroying the clarity of a very fundamental data structure for speedup rather arbitrary accumulations seems to be a bad idea. I would move this stuff in a subclass. Regards Kay +1 on this. The new suggested operations are meaningful for a subset of all valid dicts, so they should not be part of the base dict API. If any version of this is approved, it will clearly be an application of the practicality beats purity zen rule, and the justification for applying it in this case instead of subclassing should better be pretty strong; so far I'm not convinced though. So, would the `setdefaultvalue' approach be more consistent in your eyes? Reinhold -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
How about the alternative approach of allowing the user to override the action to be taken when accessing a non-existent key? d.defaultValue(0) I like this a lot. It makes it more clear from the code what is going on, rather than having to figure out what the name appendlist, count, tally, whatever, is supposed to mean. When you see the value you'll know. It's more general, because you can support dictionaries and sets then as well. I think someone mentioned that it might be a problem to add another piece of state to all dicts though. I don't know enough about the internals to say anything about this. setdefault gets around this by having you pass in the value every time, so it doesn't have to store it. It's very similar, but somehow many times more awkward. -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods - typing initialising
Duncan Booth wrote: Raymond Hettinger wrote: The rationale is to replace the awkward and slow existing idioms for dictionary based accumulation: d[key] = d.get(key, 0) + qty d.setdefault(key, []).extend(values) How about the alternative approach of allowing the user to override the action to be taken when accessing a non-existent key? d.defaultValue(0) and the accumulation becomes: d[key] += 1 and: d.defaultValue(function=list) would allow a safe: d[key].extend(values) +0 The best suggestion up to now. But i find this premature because it addresses only a special aspect of typing issues which should be disussed together with Guidos type guard proposals in a broader context. Besides this the suggestion though feeling pythonic is still uneven. Why do You set d.defaultValue(0) d.defaultValue(function=list) but not d.defaultValue(0) d.defaultValue([]) ? And why not dict(type=int), dict(type=list) instead where default values are instantiated during object creation? A consistent pythonic handling of all types should be envisioned not some ad hoc solutions that go deprecated two Python releases later. Regards Kay -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Paul Rubin wrote: Reinhold Birkenfeld [EMAIL PROTECTED] writes: Any takers for tally()? Well, as a non-native speaker, I had to look up this one in my dictionary. That said, it may be bad luck on my side, but it may be that this word is relatively uncommon and there are many others who would be happier with increment. It is sort of an uncommon word. As a US English speaker I'd say it sounds a bit old-fashioned, except when used idiomatically (let's tally up the posts about accumulator messages) or in nonstandard dialect (Hey mister tally man, tally me banana is a song about working on plantations in Jamaica). It may be more common in UK English. There's an expression tally-ho! which had something to do with British fox hunts, but they don't have those any more. Has anyone _not_ heard Jeff Probst say, I'll go tally the votes?! :) --Max -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Max wrote: Has anyone _not_ heard Jeff Probst say, I'll go tally the votes?! :) Who is Jeff Probst? -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods - typing initialising
Kay Schluehr wrote: Why do You set d.defaultValue(0) d.defaultValue(function=list) but not d.defaultValue(0) d.defaultValue([]) ? I think that's because you have to instantiate a different object for each different key. Otherwise, you would instantiate just one list as a default value for *all* default values. In other words, given: class DefDict(dict): def __init__(self, default): self.default = default def __getitem__(self, item): try: return dict.__getitem__(self, item) except KeyError: return self.default you'll get In [12]: d = DefDict([]) In [13]: d[42].extend(['foo']) In [14]: d.default Out[14]: ['foo'] In [15]: d[10].extend(['bar']) In [16]: d.default Out[16]: ['foo', 'bar'] In [17]: d[10] Out[17]: ['foo', 'bar'] In [18]: d[10] is d.default Out[18]: True and this isn't what you really wanted. By the way, to really work, I think that Duncan's proposal should create new objects when you try to access them, and to me it seems a bit counterintuitive. Nevertheless, I'm +0 on it. And why not dict(type=int), dict(type=list) instead where default values are instantiated during object creation? A consistent pythonic handling of all types should be envisioned not some ad hoc solutions that go deprecated two Python releases later. I don't really understand you. What should 'type' return? A callable that returns a new default value? That's exactly what Duncan proposed with the function keyword argument. -- Ciao, Matteo -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
In article [EMAIL PROTECTED], Raymond Hettinger wrote: I would like to get everyone's thoughts on two new dictionary methods: def count(self, value, qty=1): try: self[key] += qty except KeyError: self[key] = qty Yes, yes, YES! *Man*, this would be useful. def appendlist(self, key, *values): try: self[key].extend(values) except KeyError: self[key] = list(values) Woohoo! *Just* as useful. I'd *definitely* use these. Hot 100% sure about the names, though. (add() and append() seem like more natural names -- but they may be confusing, considering their other uses...) +1 on both (possibly allowing for some naming discussion...) -- Magnus Lie Hetland Time flies like the wind. Fruit flies http://hetland.org like bananas. -- Groucho Marx -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
I like count() and appendlist() or whatever they will be named. But I have one question/idea: Why does the methods have to be put in dict? Can't their be a subtype of dict that includes those two methods? I.e.: .histogram = counting_dict() .for ch in text: .histogram.count(ch) Then maybe some more methods can be added tailor-mode for these two types of dicts?: .for ch in string.ascii_letters: .print Frequency of %s = %d. % (ch, histogram.freq(ch)) Or something, you get the idea. -- mvh Björn -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Roose wrote: I think someone mentioned that it might be a problem to add another piece of state to all dicts though. I don't know enough about the internals to say anything about this. setdefault gets around this by having you pass in the value every time, so it doesn't have to store it. It's very similar, but somehow many times more awkward. Another option with no storage overhead which goes part way to reducing the awkwardness would be to provide a decorator class accessible through dict. The decorator class would take a value or function to be used as the default, but apart from __getitem__ would simply forward all other methods through to the underlying dictionary. That gives you the ability to have not one default value for a dictionary, but many different ones: you just decorate the dictionary anywhere you need a default and use the underlying dictionary everywhere else. Some code which demonstrates the principle rather than the implementation. dictDefaultValue could be imagined as dict.defaultValue, dictDefaultValue(d, ...) could be d.defaultValue(...) although the actual name used needs work: class dictDefaultValue(object): def __init__(self, d, value=_marker, function=_marker): self.__d = d if value is _marker: if function is _marker: raise TypeError, expected either value or function argument self.__dv = function else: def defaultValue(): return value self.__dv = defaultValue def __getattr__(self, name): return getattr(self.__d, name) def __getitem__(self, name): try: return self.__d[name] except KeyError: value = self.__dv() self.__d[name] = value return value def __setitem__(self, name, value): self.__d[name] = value d = {} accumulator = dictDefaultValue(d, 0) accumulator['a'] += 1 aggregator = dictDefaultValue(d, function=list) aggregator['b'].append('hello') d {'a': 1, 'b': ['hello']} -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods - typing initialising
Matteo Dell'Amico wrote: Kay Schluehr wrote: Why do You set d.defaultValue(0) d.defaultValue(function=list) but not d.defaultValue(0) d.defaultValue([]) ? I think that's because you have to instantiate a different object for each different key. Otherwise, you would instantiate just one list as a default value for *all* default values. Or the default value will be copied, which is not very hard either or type(self._default)() will be called. This is all equivalent and it does not matter ( except for performance reasons ) which way to go as long only one is selected. [...] By the way, to really work, I think that Duncan's proposal should create new objects when you try to access them, and to me it seems a bit counterintuitive. If the dict has a fixed semantics by applying defaultValue() and it returns defaults instead of exceptions whenever a key is missing i.e. behavioural invariance the client of the dict has nothing to worry about, hasn't he? And why not dict(type=int), dict(type=list) instead where default values are instantiated during object creation? A consistent pythonic handling of all types should be envisioned not some ad hoc solutions that go deprecated two Python releases later. I don't really understand you. What should 'type' return? A callable that returns a new default value? That's exactly what Duncan proposed with the function keyword argument. I suspect the proposal really makes sense only if the dict-values are of the same type. Filling it with strings, custom objects and other stuff and receiving 0 or [] or '' if a key is missing would be a surprise - at least for me. Instantiating dict the way I proposed indicates type-guards! This is the reason why I want to delay this issue and discuss it in a broader context. But I'm also undecided. Guidos Python-3000 musings are in danger to become vaporware. Now is better then never... Therefore +0. Regards Kay -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Paul Rubin wrote: El Pitonero [EMAIL PROTECTED] writes: What about no name at all for the scalar case: a['hello'] += 1 a['bye'] -= 2 I like this despite the minor surprise that it works even when a['hello'] is uninitialized. +1 and if the value is a list: a['hello']= [1, 2, 3] a['hello']+= [4]# adding the brackets is a lot simpler than typing append or extend. Any user is free to add his/her own subclass to handle defaults. Colin W. -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Another option with no storage overhead which goes part way to reducing the awkwardness would be to provide a decorator class accessible through dict. The decorator class would take a value or function to be used as the default, but apart from __getitem__ would simply forward all other methods through to the underlying dictionary. I'm not sure I like the decorator -- I would never use that flexibility to have more than one default. I can't come up with any reason to ever use that. I think it works best as a simple subclass: class DefaultDict(dict): def __init__(self, default, *args, **kwargs): dict.__init__(self, *args, **kwargs) self.default = default def __getitem__(self, key): return self.setdefault(key, copy.copy(self.default)) d = DefaultDict(0) for x in [1, 3, 1, 2, 2, 3, 3, 3, 3]: d[x] += 1 pprint(d) d = DefaultDict([]) for i, x in enumerate([1, 3, 1, 2, 2, 3, 3, 3, 3]): d[x].append(i) pprint(d) Output: {1: 2, 2: 2, 3: 5} {1: [0, 2], 2: [3, 4], 3: [1, 5, 6, 7, 8]} -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
On Sat, 19 Mar 2005 20:07:40 -0800, Kay Schluehr wrote: It is bad OO design, George. I want to be a bit more become more specific on this and provide an example: Having thought about this for a bit, I agree it is a powerful counter-argument and in many other languages I'd consider this a total win. But this is Python, and frankly, I've overridden dict more than once and violated the Liskov substitution principle without thinking twice. (Once, oh yes, but not twice.) Of course, all the code was under my control then. I think the tipping point for me depends on how the interfaces in Python are going to be implemented, which I haven't dug into. If the dict class gets an interface definition, can I subclass from dict and cancel (or some other term) the interface I inherited? If so, then this might still be OK, although if interfaces aren't going to confuse newbies enough, wait 'till we try to explain that their code is blowing up because they forgot to cancel their interface, and they can't *really* pass their subclass in to something expecting a dict interface. If you *can't* cancel or downgrade the interface, then I'd say this argument is still good; dict should be kept minimal and this should go in a subclass. -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods - typing initialising
Kay Schluehr wrote: I think that's because you have to instantiate a different object for each different key. Otherwise, you would instantiate just one list as a default value for *all* default values. Or the default value will be copied, which is not very hard either or type(self._default)() will be called. This is all equivalent and it does not matter ( except for performance reasons ) which way to go as long only one is selected. I don't like it very much... it seems too implicit to be pythonic. Also, it won't work with non-copyable objects, and type(42)() = 0, and getting 0 when the default is 42 looks very strange. I prefer the explicit give me a callable approach. If the dict has a fixed semantics by applying defaultValue() and it returns defaults instead of exceptions whenever a key is missing i.e. behavioural invariance the client of the dict has nothing to worry about, hasn't he? For idioms like d[foo].append('blah') to work properly, you'd have to set the default value every time you access a variable. It can be really strange to fill up memory only by apparently accessing values. I suspect the proposal really makes sense only if the dict-values are of the same type. Filling it with strings, custom objects and other stuff and receiving 0 or [] or '' if a key is missing would be a surprise - at least for me. Instantiating dict the way I proposed indicates type-guards! This is the reason why I want to delay this issue and discuss it in a broader context. But I'm also undecided. Guidos Python-3000 musings are in danger to become vaporware. Now is better then never... Therefore +0. Having duck-typing, we can have things that have common interface but no common type. For instance, iterables. I can imagine a list of iterables of different types, and a default value of maybe [] or set([]). -- Ciao, Matteo -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Beni Cherniavsky [EMAIL PROTECTED] writes: The relatively recent improvement of the dict constructor signature (``dict(foo=bar,...)``) obviously makes it impossible to just extend the constructor to ``dict(default=...)`` (or anything else for that matter) which would seem much less ad hoc. But why not use a classmethod (e.g. ``d=dict.withdefault(0)``) then? You mean giving a dictionary a default value at creation time, right? Yes. But creating a defaultdict type with aliased content to the original dict would also be fine by me. Such a dictionary could be used very easily, as in gaspPerl:: foreach $word ( @words ) { $d{$word}++; # default of 0 assumed, simple code! } /gasp. You would like to write:: d = dict.withdefault(0) # or something for word in words: d[word] += 1 # again, simple code! I agree that it's a good idea but I'm not sure the default should be specified at creation time. The problem with that is that if you pass such a dictionary into an unsuspecting function, it will not behave like a normal dictionary. Have you got a specific example in mind? Code that needlessly relies on exceptions for normal operation is rather perverse IMO and I find it hard to think of other examples. Also, this will go awry if the default is a mutable object, like ``[]`` - you must create a new one at every access (or introduce a rule that the object is copied every time, which I dislike). I think copying should on by default for objects that are mutable (and explicitly selectable via ``.withdefault(bar,copy=False)``). Python of course doesn't have an interface to query whether something is mutable or not (maybe something that'll eventually be fixed), but hashable might be a first approximation. If that's too dodgy, most commonly the value will be a builtin type anyway, so copy by default with efficient implementation (i.e. doing nothing) for ints, tuples etc. ought to work fine in practice. And there are cases where in different points in the code operating on the same dictionary you need different default values. The main problem here is that the obvious .setdefault is already taken to misnome something else. Which I guess strengthens the point for some kind of proxy object. So perhaps specifying the default at every point of use by creating a proxy is cleaner:: d = {} for word in words: d.withdefault(0)[word] += 1 Of course, you can always create the proxy once and still pass it into an unsuspecting function when that is actually what you mean. Yup (I'd presumably prefer that second option for the above code). How should a dictionary with a default value behave (wheter inherently or a proxy)? - ``d.__getattr__(key)`` never raises KeyError for missing keys - instead it returns the default value and stores the value as `d.setdefult()` does. This is needed for make code like:: d.withdefault([])[key].append(foo) to work - there is no call of `d.__setattr__()`, so `d.__getattr__()` must have stored it. I'm confused -- are you refering to __getitem__/__setitem__? Even then I don't get what you mean: __getitem__ obviously works differently, but that would be the whole point. - `d.__setattr__()` and `d.__delattr__()` behave normally. s/attr/item/ and agreed. - Should ``key in d`` return True for all keys? No. See below. It is desiarable to have *some* way to know whether a key is really present. But if it returns False for missing keys, code that checks ``key in d`` will behave differently from normally equivallent code that uses try..except. If we use the proxy interface, we can always check on the original dictionary object, which removes the problem. - ``d.has_key(key)`` must do whatever we decide ``key in d`` does. - What should ``d.get(key, [default])`` and ``d.setdefault(key, default)`` do? There is a conflict between the default of `d` and the explicitly given default. I think consistency is better and these should pretend that `key` is always present. But either way, there is a subtle problem here. .setdefault ought to trump defaultdict's default. I feel that code that operated without raising an KeyError on normal dicts should also operate the same way on defaultdicts were possible. I'd also suspect that if you're effectively desiring to override .setdefault's default you're up to something dodgy. - Of course `iter(d)`, `d.items()` and the like should only see the keys that are really present (the alternative inventing an infinite amount of items out of the blue is clearly bogus). If the idea that the default should be specified in every operation (creating a proxy) is accepted, there is a simpler and more fool-proof solution: the ptoxy will not support anything except `__getitem__()` and `__setitem__()` at all. Use the original dictionary for everything else. This prevents subtle
Re: Pre-PEP: Dictionary accumulator methods
Hi, I really do not like it. So -1 for me. Your two methods are very specialized whereas the dict type is very generic. Usually, when I see something like this in code, I can smell it's a patch to overcome some shortcomings on a previous design, thereby making the economy of re-designing. Simply said, that's bad programming. After that patch to provide a solution for only two of the more common use-cases, you are nonetheless stucked with the old solution for all the other use-cases (what if the value type is another dictionary or some user-made class ?). Here's an alternate solution which I think answers all of the problems you mentionned while being generic. === BEGIN SNAP def update_or_another_great_name(self, key, createFunc, updtFunc): try: self[key] = updtFunc(self[key]) ## This is slow with Python = since the key has to be searched ## twice But the new built-in method just has to update the value the ## first time the key is found. Therefore speed should be ok. return True except KeyError: self[key] = createFunc() return false ## Now your two specialized methods can be easily written as : ## A built-in should be provided for this (if not already proposed) : def identical(val): return val def count(self, key, qty=1): self.update_or_another_great_name(key, identical, partial(operator.add, qty)) ## partial is coming from : http://www.python.org/peps/pep-0309.html ## Using only built-in function (assuming identical) as arguments makes it ## ok for speed (I guess). def appendlist(self, key, *values): self.update_or_another_great_name(key, partial(list, values), partial(ListType.extend, X = values)) ## The first partial usage here is an abuse just to make sure that the ## list is not actually constructed before needed. It should work. ## The second usage is more uncertain as we need to bind the arguments from ## the right. Therefore I have to use the name of the parameter and I am not ## sure if there's one. As this list is very prolific, someone might have an ## idea on how to improve this. === END SNAP By using only built-in constructs, this should be fast enough. Otherwise, optimizing these built-ins is a much more clean and sane way of thinking then messing the API with ad-hoc propositions. Reviewing the problems you mention : The readability issues with the existing constructs are: * They are awkward to teach, create, read, and review. The method update_or_another_great_name is easy to understand, I think. But it might not always be easy to use it efficiently with built-ins. But this is always the case. Recipees can be added to show how to efficiently use the method. * Their wording tends to hide the real meaning (accumulation). Solved. * The meaning of setdefault() 's method name is not self-evident. Solved. The performance issues with the existing constructs are: * They translate into many opcodes which slows them considerably. I really don't know what will be the outcome of the solution I propose. I certainly do not know anything about how my Python code translates into opcodes. * The get() idiom requires two dictionary lookups of the same key. Solved * The setdefault() idiom instantiates a new, empty list prior to every Solved call. * That new list is often not needed and is immediately discarded. Solved * The setdefault() idiom requires an attribute lookup for extend/append. Solved * The setdefault() idiom makes two function calls. Solved And perhaps, what you say here is also true for your two special use-cases : For other uses, plain Python code suffices in terms of speed, clarity, and avoiding unnecessary instantiation of empty containers: if key not in d: d.key = {subkey:value} else: d[key][subkey] = value Much better than adding special cases on a generic class. Special cases always demultiply and if we open the door Regards, Francis Girard Le samedi 19 Mars 2005 02:24, Raymond Hettinger a crit: I would like to get everyone's thoughts on two new dictionary methods: def count(self, value, qty=1): try: self[key] += qty except KeyError: self[key] = qty def appendlist(self, key, *values): try: self[key].extend(values) except KeyError: self[key] = list(values) The rationale is to replace the awkward and slow existing idioms for dictionary based accumulation: d[key] = d.get(key, 0) + qty d.setdefault(key, []).extend(values) In simplest form, those two statements would now be coded more readably as: d.count(key) d.appendlist(key, value) In their multi-value forms, they would now be coded as: d.count(key, qty) d.appendlist(key, *values) The error
Re: Pre-PEP: Dictionary accumulator methods
Reinhold Birkenfeld wrote: I don't quite understand that. Which dict item are you extending? Don't you need something like dl[key].append(word) Rigth. It was just a typo on my part. Thanks for fixing. Mike -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
In article [EMAIL PROTECTED], Michele Simionato [EMAIL PROTECTED] wrote: I am surprised nobody suggested we put those two methods into a separate module (say dictutils or even UserDict) as functions: from dictutils import tally, listappend tally(mydict, key) listappend(mydict, key, value) That seems like a reasonable compromise. -- Aahz ([EMAIL PROTECTED]) * http://www.pythoncraft.com/ The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death. --GvR -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
In article [EMAIL PROTECTED], [EMAIL PROTECTED] (Aahz) wrote: I am surprised nobody suggested we put those two methods into a separate module (say dictutils or even UserDict) as functions: from dictutils import tally, listappend tally(mydict, key) listappend(mydict, key, value) That seems like a reasonable compromise. The more messages I see on this thread, the more I think adding a different new method for each commonly used kind of update is the wrong solution. We already have methods that work pretty well and, I think, read better than the new methods: mydict[key] += 1 mydict[key].append(value) The problem is merely that they don't work when key is missing, so we need to resort to setdefault circumlocutions instead. A better solution seems to be the one I've seen suggested here several times, of changing the dict's behavior so that the setdefault is automatic whenever trying to access a missing key. If this would be in a separate module or separate subclass of dict, so much the better. -- David Eppstein Computer Science Dept., Univ. of California, Irvine http://www.ics.uci.edu/~eppstein/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
On Sun, 20 Mar 2005 15:14:22 -0800, David Eppstein [EMAIL PROTECTED] wrote: In article [EMAIL PROTECTED], [EMAIL PROTECTED] (Aahz) wrote: I am surprised nobody suggested we put those two methods into a separate module (say dictutils or even UserDict) as functions: from dictutils import tally, listappend tally(mydict, key) listappend(mydict, key, value) That seems like a reasonable compromise. The more messages I see on this thread, the more I think adding a different new method for each commonly used kind of update is the wrong solution. We already have methods that work pretty well and, I think, read better than the new methods: mydict[key] += 1 mydict[key].append(value) The problem is merely that they don't work when key is missing, so we need to resort to setdefault circumlocutions instead. A better solution seems to be the one I've seen suggested here several times, of changing the dict's behavior so that the setdefault is automatic whenever trying to access a missing key. If this would be in a separate module or separate subclass of dict, so much the better. I think that the setdefault behavior needs to be done on an per application basis because whose to say what default is best?. With a preset default mode, it then becomes possible to inadvertently create default values that will cause problems without knowing it. So then we have to remember to change the setdefault value to None or null to avoid problems. Ouch! Also pythons normal behavior for retrieving objects that are not defined is to give an error. So having dictionaries that auto defaults to a mode that doesn't behave that way is inconsistent with the rest of the language. Yet, I'm all for the creation of specialized containers in a standard module! :) Then we can have string dicts, and int dicts, and card dicts, account dicts, etc, as well as specialized lists. Call them 'smart containers'. But they should not be built into the base class. Ron -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Raymond Hettinger: Any takers for tally()? Dunno, to me tally reads counts the numbers of votes for a candidate in an election. We should avoid abbreviations like inc() or incr() that different people tend to abbreviate differently (for example, that is why the new partial() function has its keywords argument spelled-out). The only other issue I see with that name is that historically incrementing is more associated with +=1 than with +=n. Also, there are reasonable use cases for a negative n and it would be misleading to call it incrementing when decrementing is what is intended. I agree with Paul Rubin's argument on that issue, let's use increment() and do not worry about negative increments. appendlist seems a bit too specific (I do not use dictionaries of lists that often). I'm curious. When you do use setdefault, what is the typical second argument? Well, I have used setdefault *very few times* in years of heavy Python usage. His disappearence would not bother me that much. Grepping my source code I find that practically my main use case for setdefault is in a memoize recipe where the result of a function call is stored in a dictionary (if not already there) and returned. Then I have a second case with a list as second argument. The problem with setdefault is the name, not the functionality. Are you happy with the readability of the argument order? To me, the key and default value are not at all related. Do you prefer having the default value pre-instantiated on every call when the effort is likely to be wasted? Do you like the current design of returning an object and then making a further (second dot) method lookup and call for append or extend? When you first saw setdefault explained, was it immediately obvious or did it taking more learning effort than other dictionary methods? To me, it is the least explainable dictionary method. Even when given a good definition of setdefault(), it is not immediately obvious that it is meant to be futher combined with append() or some such. When showing code to newbies or non-pythonistas, do they find the meaning of the current idiom self-evident? That last question is not compelling, but it does contrast with other Python code which tends to be grokkable by non-pythonistas and clients. get_or_set would be a better name: we could use it as an alias for setdefault and then remove setdefault in Python 3000. While get_or_set would be a bit of an improvement, it is still obtuse. Eventhough a set operation only occurs conditionally, the get always occurs. The proposed name doesn't make it clear that the method alway returns an object. Honestly, I don't care about the performance arguments. However I care a lot about about readability and clarity. setdefault is terrible in this respect, since most of the time it does *not* set a default, it just get a value. So I am always confused and I have to read at the documentation to remind to myself what it is doing. The only right name would be get_and_possibly_set but it is a bit long to type. Even if a wording is found that better describes the both the get and set operation, it is still a distractor from the intent of the combined statement, the intent of building up a list. That is an intrinsic wording limitation that cannot be solved by a better name for setdefault. If any change is made at all, we ought to go the distance and provide a better designed tool rather than just a name change. Well, I never figured out that the intent of setdefault was to build up a list ;) Anyway, if I think at how many times I have used setdefault in my code (practically twice) and how much time I have spent trying to decipher it (any time I reread the code using it) I think I would have better served by NOT having the setdefault method available ;) About appendlist(): still it seems a bit special purpose to me. I mean, dictionaries already have lots of methods and I would think twice before adding new ones; expecially methods that may turn out not that useful in the long range, or easily replaceble by user code. Michele Simionato -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Reinhold Birkenfeld [EMAIL PROTECTED] writes: Any takers for tally()? Well, as a non-native speaker, I had to look up this one in my dictionary. That said, it may be bad luck on my side, but it may be that this word is relatively uncommon and there are many others who would be happier with increment. It is sort of an uncommon word. As a US English speaker I'd say it sounds a bit old-fashioned, except when used idiomatically (let's tally up the posts about accumulator messages) or in nonstandard dialect (Hey mister tally man, tally me banana is a song about working on plantations in Jamaica). It may be more common in UK English. There's an expression tally-ho! which had something to do with British fox hunts, but they don't have those any more. I'd say I prefer most of the suggested alternatives (count, add, incr/increment) to tally. -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
[Michele Simionato] Dunno, to me tally reads counts the numbers of votes for a candidate in an election. That isn't a pleasant image ;-) The only right name would be get_and_possibly_set but it is a bit long to type. Even if a wording is found that better describes the both the get and set operation, it is still a distractor from the intent of the combined statement, the intent of building up a list. That is an intrinsic wording limitation that cannot be solved by a better name for setdefault. If any change is made at all, we ought to go the distance and provide a better designed tool rather than just a name change. Well, I never figured out that the intent of setdefault was to build up a list ;) Right! What does have that intent is the full statement: d.setdefault(k, []).append(v). My thought is that setdefault() is rarely used by itself. Instead, it is typically part of a longer sentence whose intent and meaning is to accumulate or build-up. That meaning is not well expressed by the current idiom. Raymond Hettinger -- http://mail.python.org/mailman/listinfo/python-list
any() and all() Was: Pre-PEP: Dictionary accumulator methods
Py2.5 is already going to include any() and all() as builtins. The signature does not include a function, identity or otherwise. Instead, the caller can write a listcomp or genexp that evaluates to True or False: any(x = 42 for x in data) [Roose] Oh great, I just saw that. . . . But I wish it could be included in Python 2.4.x. If it is any consolation, the any() can already be expressed somewhat cleanly and efficiently in Py2.4 with genexps: True in (x = 42 for x in data) The translation for all() is a little less elegant: False not in (x = 42 for x in data) Raymond Hettinger -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Py2.5 is already going to include any() and all() as builtins. The signature does not include a function, identity or otherwise. Instead, the caller can write a listcomp or genexp that evaluates to True or False: Actually I was just looking at Python 2.5 docs since you mentioned this. http://www.python.org/dev/doc/devel/whatsnew/node3.html It says min() and max() will gain a key function parameter, and sort() gained one in Python 2.4 (news to me). And they do indeed default to the identity in all 3 cases, so this seems very inconsistent. If one of them has it, and sort gained the argument even in Python 2.4 with generator expressions, then they all should have it. any(x = 42 for x in data) Not to belabor the point, but in the example on that page, max(L, key=len) could be written max(len(x) for x in L). Now I know why Guido said he didn't want a PEP for this... such a trivial thing can produce a lot of opinions. : ) Roose -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Roose wrote: Not to belabor the point, but in the example on that page, max(L, key=len) could be written max(len(x) for x in L). No, it can't: Python 2.5a0 (#2, Mar 5 2005, 17:44:37) [GCC 3.3.3 (SuSE Linux)] on linux2 Type help, copyright, credits or license for more information. max([a, bbb, cc], key=len) 'bbb' Peter -- http://mail.python.org/mailman/listinfo/python-list
any() and all() Was: Pre-PEP: Dictionary accumulator methods
[Roose] Actually I was just looking at Python 2.5 docs since you mentioned this. http://www.python.org/dev/doc/devel/whatsnew/node3.html It says min() and max() will gain a key function parameter, and sort() gained one in Python 2.4 (news to me). It also appears in itertools.groupby() and, for Py2.5, in heapq.nsmallest() and heapq.nlargest(). And they do indeed default to the identity in all 3 cases, so this seems very inconsistent. If one of them has it, and sort gained the argument even in Python 2.4 with generator expressions, then they all should have it. any(x = 42 for x in data) Not to belabor the point, but in the example on that page, max(L, key=len) could be written max(len(x) for x in L). Think about it. A key= function is quite a different thing. It provides a *temporary* comparison key while retaining the original value. IOW, your re-write is incorrect: L = ['the', 'quick', 'brownish', 'toad'] max(L, key=len) 'brownish' max(len(x) for x in L) 8 Remain calm. Keep the faith. Guido's design works fine. No important use cases were left unserved by any() and all(). Raymond Hettinger -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
On Sat, 19 Mar 2005 01:24:57 GMT, Raymond Hettinger [EMAIL PROTECTED] wrote: I would like to get everyone's thoughts on two new dictionary methods: def count(self, value, qty=1): try: self[key] += qty except KeyError: self[key] = qty def appendlist(self, key, *values): try: self[key].extend(values) except KeyError: self[key] = list(values) Bengt Richter wrote: class xdict(dict): ... def valadd(self, key, incr=1): ... try: self[key] = self[key] + type(self[key])(incr) ... except KeyError: self[key] = incr What about: import copy class safedict(dict): def __init__(self, default=None): self.default = default def __getitem__(self, key): try: return dict.__getitem__(self, key) except KeyError: return copy.copy(self.default) x = safedict(0) x[3] += 1 y = safedict([]) y[5] += range(3) print x, y print x[123], y[234] -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Bengt Richter wrote: On Sat, 19 Mar 2005 01:24:57 GMT, Raymond Hettinger [EMAIL PROTECTED] wrote: I would like to get everyone's thoughts on two new dictionary methods: def count(self, value, qty=1): try: self[key] += qty except KeyError: self[key] = qty def appendlist(self, key, *values): try: self[key].extend(values) except KeyError: self[key] = list(values) How about an efficient duck-typing value-incrementer to replace both? E.g. functionally like: class xdict(dict): ... def valadd(self, key, incr=1): ... try: self[key] = self[key] + type(self[key])(incr) ... except KeyError: self[key] = incr A big problem with this is that there are reasonable use cases for both d.count(key, some integer) and d.appendlist(key, some integer) Word counting is an obvious use for the first. Consolidating a list of key, value pairs where the values are ints requires the second. Combining count() and appendlist() into one function eliminates the second possibility. Kent -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Raymond Hettinger wrote: I would like to get everyone's thoughts on two new dictionary methods: def count(self, value, qty=1): try: self[key] += qty except KeyError: self[key] = qty def appendlist(self, key, *values): try: self[key].extend(values) except KeyError: self[key] = list(values) Emphatic +1 I use both of these idioms all the time. (Kind of surprised to see people confused about the need for the latter; I do it regularly.) This is just the kind of thing experience shows cropping up enough that it makes sense to put it in the language. About the names: Seeing that these have specific uses, and do something that is hard to explain in one word, I would suggest that short names like count might betray the complexity of the operations. Therefore, I'd suggest: increment_value() (or add_to_value()) append_to_value() Although they don't explicitly communicate that a value would be created if it didn't exist, they do at least make it clear that it happens to the value, which kind of implies that it would be created. If we do have to use short names: I don't like increment (or inc or incr) at all because it has the air of a mutator method. Maybe it's just my previous experience with Java and C++, but to me, a.incr() looks like it's incrementing a, and a.incr(b) looks like it might be adding b to a. I don't like count because it's too vague; it's pretty obvious what it does as an iterator, but not as a method of dict. I could live with tally, though. As for a short name for the other one, maybe fileas or fileunder? -- CARL BANKS -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Brian van den Broek wrote: Raymond Hettinger said unto the world upon 2005-03-18 20:24: I would like to get everyone's thoughts on two new dictionary methods: def appendlist(self, key, *values): try: self[key].extend(values) except KeyError: self[key] = list(values) For appendlist, I would have expected def appendlist(self, key, sequence): try: self[key].extend(sequence) except KeyError: self[key] = list(sequence) The original proposal reads better at the point of call when values is a single item. In my experience this will be the typical usage: d.appendlist(key, 'some value') as opposed to your proposal which has to be written d.appendlist(key, ['some value']) The original allows values to be a sequence using d.appendlist(key, *value_list) Kent -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
On Sat, 19 Mar 2005 01:24:57 GMT, Raymond Hettinger [EMAIL PROTECTED] wrote: The proposed names could possibly be improved (perhaps tally() is more active and clear than count()). Curious that in this lengthy discussion, a method name of accumulate never came up. I'm not sure how to separate the two cases (accumulating scalars vs. accumulating a list), though. Regards, Dan -- Dan Sommers http://www.tombstonezero.net/dan/ c = 1 -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Ivan Van Laningham a écrit : Hi All-- Maybe I'm not getting it, but I'd think a better name for count would be add. As in d.add(key) d.add(key,-1) d.add(key,399) etc. [...] There is no existing add() method for dictionaries. Given the name change, I'd like to see it. Metta, Ivan I don't think add is a good name ... even if it doesn't exist in dictionnarie, it exists in sets and, IMHO, this would add confusion ... Pierre -- Ivan Van Laningham God N Locomotive Works http://www.pauahtun.org/ http://www.andi-holmes.com/ Army Signal Corps: Cu Chi, Class of '70 Author: Teach Yourself Python in 24 Hours -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
[Jeff Epler] Maybe something for sets like 'appendlist' ('unionset'?) On Sat, Mar 19, 2005 at 04:18:43AM +, Raymond Hettinger wrote: I do not follow. Can you provide a pure python equivalent? Here's what I had in mind: $ python /tmp/unionset.py Set(['set', 'self', 'since', 's', 'sys', 'source', 'S', 'Set', 'sets', 'starting']) # try: set except: from sets import Set as set def unionset(self, key, *values): try: self[key].update(values) except KeyError: self[key] = set(values) if __name__ == '__main__': import sys, re index = {} # We need a source of words. This file will do. corpus = open(sys.argv[0]).read() words = re.findall('\w+', corpus) # Create an index of the words according to the first letter. # repeated words are listed once since the values are sets for word in words: unionset(index, word[0].lower(), word) # Display the words starting with 'S' print index['s'] # Jeff pgpecFFOsSmDH.pgp Description: PGP signature -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Hi All-- Raymond Hettinger wrote: [Michele Simionato] +1 for inc instead of count. Any takers for tally()? Sure. Given the reasons for avoiding add(), tally()'s a much better choice than count(). What about d.tally(key,0) then? Deleting the key as was suggested by Michael Spencer seems non-intuitive to me. Just my 2 Eurocents, I raise you by a ruble and a pound ;-) hardly-anything-is-worth-less-than-vietnamese-dong-ly y'rs, Ivan -- Ivan Van Laningham God N Locomotive Works http://www.andi-holmes.com/ http://www.foretec.com/python/workshops/1998-11/proceedings.html Army Signal Corps: Cu Chi, Class of '70 Author: Teach Yourself Python in 24 Hours -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Michele Simionato wrote: +1 for inc instead of count. -1 for inc, increment, or anything that carries a connotation of *increasing* the value, so long as the proposal allows for negative numbers to be involved. Incrementing by -1 is a pretty silly picture. +1 for add and, given the above, I'm unsure there's a viable alternative (unless this is restricted to positive values, or perhaps even to +1 specifically). appendlist seems a bit too specific (I do not use dictionaries of lists that often). As Raymond does, I use this much more than the other. The problem with setdefault is the name, not the functionality. get_or_set would be a better name: we could use it as an alias for setdefault and then remove setdefault in Python 3000. Agreed... -Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Peter Hansen wrote: Michele Simionato wrote: +1 for inc instead of count. -1 for inc, increment, or anything that carries a connotation of *increasing* the value, so long as the proposal allows for negative numbers to be involved. Incrementing by -1 is a pretty silly picture. +1 for add and, given the above, I'm unsure there's a viable alternative (unless this is restricted to positive values, or perhaps even to +1 specifically). What about `addto()'? add() just has the connotation of adding something to the dict and not to an item in it. Reinhold -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Reinhold Birkenfeld wrote: Peter Hansen wrote: +1 for add and, given the above, I'm unsure there's a viable alternative (unless this is restricted to positive values, or perhaps even to +1 specifically). What about `addto()'? add() just has the connotation of adding something to the dict and not to an item in it. Hmm... better than add anyway. I take back my ill-considered +1 above, and apply instead a +0 to count. I don't actually like any of the alternatives at this point... needs more thought (for my part, anyway). To be honest, the only time I've ever seen this particular idiom is in tutorial code or examples of how you produce a histogram of word usage in a text document. Never in real code (not that it doesn't happen, just that I've never stumbled across it). The appending to a list idiom, on the other hand, I've seen and used quite often. I'm just going to stay out of the add/inc/count/addto debate and consider the other half of the thread now. :-) -Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
[Jeff Epler] Maybe something for sets like 'appendlist' ('unionset'?) While this could work and potentially be useful, I think it is better to keep the proposal focused on the two common use cases. Adding a third would reduce the chance of acceptance. Also, in all of my code base, I've not run across a single opportunity to use something like unionset(). This is surprising because I'm the set() author and frequently use set based algorithms.Your example was a good one and I can also image a graph represented as a dictionary of sets. Still, I don't mind writing out the plain Python for this one if it only comes up once in a blue moon. Raymond -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
[Dan Sommers] Curious that in this lengthy discussion, a method name of accumulate never came up. I'm not sure how to separate the two cases (accumulating scalars vs. accumulating a list), though. Separating the two cases is essential. Also, the wording should contain strong cues that remind you of addition and of building a list. For the first, how about addup(): d = {} for word in text.split(): d.addup(word) Raymond -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
On 18 Mar 2005 21:03:52 -0800 Michele Simionato wrote: MS +1 for inc instead of count. MS appendlist seems a bit too specific (I do not use dictionaries of MS lists that often). inc is too specific too. MS The problem with setdefault is the name, not the functionality. The problem with functionality: d.setdefault(k, v) can't be used as lvalue. If it could, we wouldn't need count/inc/add/tally method. MS get_or_set would be a better name: we could use it as an alias for MS setdefault and then remove setdefault in Python 3000. What about d.get(k, setdefault=v) alternative? Not sure whether it's good idea to overload get() method, just an idea. -- Denis S. Otkidach http://www.python.ru/ [ru] -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Hi All-- Raymond Hettinger wrote: Separating the two cases is essential. Also, the wording should contain strong cues that remind you of addition and of building a list. For the first, how about addup(): d = {} for word in text.split(): d.addup(word) I still prefer tally(), despite perceived political connotations. They're only connotations, after all, and tally() comprises both positive and negative incrementing, whereas add() and addup() will tease users into thinking they are only for incrementing. What about adding another method, setincrement()? d={} d.setincrement(-1) for word in text.split(): d.tally(word,1) if word.lower() in [a,an,the]: d.tally(word) Not that there's any real utility in that. Metta, Ivan -- Ivan Van Laningham God N Locomotive Works http://www.pauahtun.org/ http://www.andi-holmes.com/ Army Signal Corps: Cu Chi, Class of '70 Author: Teach Yourself Python in 24 Hours -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Dan Sommers wrote: On Sat, 19 Mar 2005 01:24:57 GMT, Raymond Hettinger [EMAIL PROTECTED] wrote: The proposed names could possibly be improved (perhaps tally() is more active and clear than count()). Curious that in this lengthy discussion, a method name of accumulate never came up. I'm not sure how to separate the two cases (accumulating scalars vs. accumulating a list), though. Is it even necessary to use a method name? import copy class safedict(dict): def __init__(self, default=None): self.default = default def __getitem__(self, key): try: return dict.__getitem__(self, key) except KeyError: return copy.copy(self.default) x = safedict(0) x[3] += 1 y = safedict([]) y[5] += range(3) print x, y print x[123], y[234] -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
[Ivan Van Laningham] What about adding another method, setincrement()? . . . Not that there's any real utility in that. That was a short lived suggestion ;-) Also, it would entail storing an extra value in the dictionary header. That alone would be a killer. Raymond -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
-1 on set increment. I think this makes your intent much clearer: .d={} .for word in text.split(): .d.tally(word) .if word.lower() in [a,an,the]: .d.tally(word,-1) or perhaps simplest: .d={} .for word in text.split(): .if word.lower() not in [a,an,the]: .d.tally(word) Personally, I'm +1 for tally(), and possibly tallyList() and tallySet() to complete the thought for the cumulative container cases. I think there is something to be gained if these methods get named in some similar manner. For those dead set against tally() and its ilk, how about accum(), accumList() and accumSet()? -- Paul -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
In article [EMAIL PROTECTED], Raymond Hettinger [EMAIL PROTECTED] wrote: The proposed names could possibly be improved (perhaps tally() is more active and clear than count()). +1 tally() -- Aahz ([EMAIL PROTECTED]) * http://www.pythoncraft.com/ The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death. --GvR -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Raymond Hettinger wrote: Separating the two cases is essential. Also, the wording should contain strong cues that remind you of addition and of building a list. For the first, how about addup(): d = {} for word in text.split(): d.addup(word) import copy class safedict(dict): def __init__(self, default=None): self.default = default def __getitem__(self, key): if not self.has_key(key): self[key] = copy.copy(self.default) return dict.__getitem__(self, key) text = 'a b c b a' words = text.split() counts = safedict(0) positions = safedict([]) for i, word in enumerate(words): counts[word] += 1 positions[word].append(i) print counts, positions -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
In article [EMAIL PROTECTED], Raymond Hettinger [EMAIL PROTECTED] wrote: How about countkey() or tabulate()? Those rank roughly equal to tally() for me, with a slight edge to these two for clarity and a slight edge to tally() for conciseness. -- Aahz ([EMAIL PROTECTED]) * http://www.pythoncraft.com/ The joy of coding Python should be in seeing short, concise, readable classes that express a lot of action in a small amount of clear code -- not in reams of trivial code that bores the reader to death. --GvR -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
[El Pitonero] Is it even necessary to use a method name? import copy class safedict(dict): def __init__(self, default=None): self.default = default def __getitem__(self, key): try: return dict.__getitem__(self, key) except KeyError: return copy.copy(self.default) x = safedict(0) x[3] += 1 y = safedict([]) y[5] += range(3) print x, y print x[123], y[234] safedict() and variants have been previously proposed with the name defaultdict or some such. For the most part, adding methods is much less disruptive than introducing a new type. As written out above, the += syntax works fine but does not work with append(). As written, the copy.copy() approach is dog slow but can be optimized for lists and ints while retaining its type flexibility. BTW, there is no need to make the same post three times. Raymond -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Kent Johnson said unto the world upon 2005-03-19 07:19: Brian van den Broek wrote: Raymond Hettinger said unto the world upon 2005-03-18 20:24: I would like to get everyone's thoughts on two new dictionary methods: def appendlist(self, key, *values): try: self[key].extend(values) except KeyError: self[key] = list(values) For appendlist, I would have expected def appendlist(self, key, sequence): try: self[key].extend(sequence) except KeyError: self[key] = list(sequence) The original proposal reads better at the point of call when values is a single item. In my experience this will be the typical usage: d.appendlist(key, 'some value') as opposed to your proposal which has to be written d.appendlist(key, ['some value']) The original allows values to be a sequence using d.appendlist(key, *value_list) Kent Right. I did try the alternatives out and get the issue you point to. But: 1) In my own code, cases where I'd use the proposed appendlist method are typically cases where I'd want to add multiple items that have already been collected in a sequence. But, since I've little coding under my belt, I concede that considerations drawn from my experience are not terribly probative. :-) 2) Much more important, IMHO, is that the method name `appendlist' really does suggest it's a list that will be appended. Hence my stated expectation. While it would make the method name longer, given the original interface Raymond posted, I would find appendtolist more transparent. out-of-my-depth-ly y'rs, Brian vdB -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Raymond Hettinger [EMAIL PROTECTED] wrote: [Jeff Epler] Maybe something for sets like 'appendlist' ('unionset'?) While this could work and potentially be useful, I think it is better to keep the proposal focused on the two common use cases. Adding a third would reduce the chance of acceptance. Also, in all of my code base, I've not run across a single opportunity to use something like unionset(). This is surprising because I'm the set() author and frequently use set based algorithms.Your example was a good one and I can also image a graph represented as a dictionary of sets. Still, I don't mind writing out the plain Python for this one if it only comes up once in a blue moon. Good example. I actually have a directed graph and multigraph module that uses dictionary of sets internally. It turns out I've used setdefault 8 times in this module alone ! George -- http://mail.python.org/mailman/listinfo/python-list
Re: any() and all() Was: Pre-PEP: Dictionary accumulator methods
Ah OK, I stand corrected. Whoops. I just read the web page and thought the wrong thing, that makes sense. Think about it. A key= function is quite a different thing. It provides a *temporary* comparison key while retaining the original value. IOW, your re-write is incorrect: L = ['the', 'quick', 'brownish', 'toad'] max(L, key=len) 'brownish' max(len(x) for x in L) 8 Remain calm. Keep the faith. Guido's design works fine. No important use cases were left unserved by any() and all(). Raymond Hettinger -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Hi if key not in d: d[key] = {subkey:value} else: d[key][subkey] = value and d[(key,subkey)] = value ? Michel Claveau -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Raymond Hettinger wrote: As written out above, the += syntax works fine but does not work with append(). ... BTW, there is no need to make the same post three times. The append() syntax works, if you use the other definition of safedict (*). There are more than one way of defining safedict, see the subtle differences between the two versions of safedict, and you'll be glad more than one version has been posted. At any rate, what has been presented is a general idea, nitpicking details is kind of out of place. Programmers know how to modify a general receipe to suit their actual needs, right? (*) In some cases, people do not want to create a dictionary entry when an inquiry is done on a missing item. In some case, they do. A general receipe cannot cater to the needs of everybody. -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Aahz [EMAIL PROTECTED] wrote: In article [EMAIL PROTECTED], Raymond Hettinger [EMAIL PROTECTED] wrote: The proposed names could possibly be improved (perhaps tally() is more active and clear than count()). +1 tally() -1 for count(): Implies an accessor, not a mutator. -1 for tally(): Unfriendly to non-native english speakers. +0.5 for add, increment. If incrementing a negative is unacceptable, how about update/updateby/updateBy ? +1 for accumulate. I don't think that separating the two cases -- adding to a scalar or appending to a list -- is that essential; a self-respecting program should make this obvious by the name of the parameter anyway (dictionary.accumulate('hello', words) vs a.accumulate('hello', b)). George -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
George Sakkis wrote: Aahz [EMAIL PROTECTED] wrote: In article [EMAIL PROTECTED], Raymond Hettinger [EMAIL PROTECTED] wrote: The proposed names could possibly be improved (perhaps tally() is more active and clear than count()). +1 tally() -1 for count(): Implies an accessor, not a mutator. -1 for tally(): Unfriendly to non-native english speakers. +0.5 for add, increment. If incrementing a negative is unacceptable, how about update/updateby/updateBy ? +1 for accumulate. I don't think that separating the two cases -- adding to a scalar or appending to a list -- is that essential; a self-respecting program should make this obvious by the name of the parameter anyway (dictionary.accumulate('hello', words) vs a.accumulate('hello', b)). What about no name at all for the scalar case: a['hello'] += 1 a['bye'] -= 2 and append() (or augmented assignment) for the list case: a['hello'].append(word) a['bye'] += [word] ? -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
On Sat, 19 Mar 2005 15:17:59 GMT, Raymond Hettinger [EMAIL PROTECTED] wrote: [Dan Sommers] Curious that in this lengthy discussion, a method name of accumulate never came up. I'm not sure how to separate the two cases (accumulating scalars vs. accumulating a list), though. Separating the two cases is essential. Also, the wording should contain strong cues that remind you of addition and of building a list. Agreed, with a slight hedge towards accumulation or tabulation rather than addition. I don't think summation gets us anywhere, either. Are the use cases for qty != 1 for weighted averages (that's the only one I can think of off the top of my head)? Is something like this: def accumulate( self, key, *values ): if values == ( ): values = 1 try: self[ key ] += values except KeyError: if type( key ) == int: self[ key ] = 1 else self[ key ] = *values possible? It's more klunky than I thought it would be before I started typing it out. Then we'd have these two use cases: histogram = { } for word in text.split( ): histogram.accumulate( word ) and org_chart = { } for employee in employees: org_chart.accumulate( employee.manager, employee.name ) Regards, Dan -- Dan Sommers http://www.tombstonezero.net/dan/ c = 1 -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Raymond Hettinger wrote: I would like to get everyone's thoughts on two new dictionary methods: def count(self, value, qty=1): try: self[key] += qty except KeyError: self[key] = qty def appendlist(self, key, *values): try: self[key].extend(values) except KeyError: self[key] = list(values) -1 form me. I'm not very glad with both of them ( not a naming issue ) because i think that the dict type should offer only methods that apply to each dict whatever it contains. count() specializes to dict values that are addable and appendlist to those that are extendable. Why not subtractable, dividable or right-shiftable? Because of majority approval? I'm mot a speed fetishist and destroying the clarity of a very fundamental data structure for speedup rather arbitrary accumulations seems to be a bad idea. I would move this stuff in a subclass. Regards Kay -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
El Pitonero [EMAIL PROTECTED] writes: What about no name at all for the scalar case: a['hello'] += 1 a['bye'] -= 2 I like this despite the minor surprise that it works even when a['hello'] is uninitialized. -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
On Sat, 19 Mar 2005 07:13:15 -0500, Kent Johnson [EMAIL PROTECTED] wrote: Bengt Richter wrote: On Sat, 19 Mar 2005 01:24:57 GMT, Raymond Hettinger [EMAIL PROTECTED] wrote: I would like to get everyone's thoughts on two new dictionary methods: def count(self, value, qty=1): try: self[key] += qty except KeyError: self[key] = qty def appendlist(self, key, *values): try: self[key].extend(values) except KeyError: self[key] = list(values) How about an efficient duck-typing value-incrementer to replace both? E.g. functionally like: class xdict(dict): ... def valadd(self, key, incr=1): ... try: self[key] = self[key] + type(self[key])(incr) ... except KeyError: self[key] = incr A big problem with this is that there are reasonable use cases for both d.count(key, some integer) and d.appendlist(key, some integer) Word counting is an obvious use for the first. Consolidating a list of key, value pairs where the values are ints requires the second. Combining count() and appendlist() into one function eliminates the second possibility. I don't see a big problem ;-) d.addval doesn't eliminate the functionalities if you want them. You just have to spell them d.addval(key, some integer) and d.addval(key, [some integer]) respectively. My example interactive stuff in a previous post shows both of these using xd['x'] and xd.['y']: xd = xdict() xd {} Default integer 1 arg creates initial int value xd.valadd('x') xd {'x': 1} Explicit list arg create initial list value xd.valadd('y', range(3)) xd {'y': [0, 1, 2], 'x': 1} Explicit int increment adds to existing int xd.valadd('x', 100) xd['x'] 101 Explicit list arg extends existing list with contents of increment list which you can of course optionally use with a length-1 list to achieve the .append effect xd.valadd('y', range(3,6)) xd['y'] [0, 1, 2, 3, 4, 5] Granted, d.appendlist will result in more efficient code, since the temp list arg [some integer] does not have to be created and the type of the existing dict value does not have to be tested as generally. OTOH, you can create and extend tuple or even custom object values using valadd, and extend with alternate sequences that get converted to the existing type if its contructor knows how to accept alternatives. So I think you are not prevented from doing anything. IMO Raymond's Zen concerns are the ones to think about first, and then efficiency, which was one of the motivators in the first place ;-) Regards, Bengt Richter -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Raymond Hettinger wrote: I would like to get everyone's thoughts on two new dictionary methods: +1 for each. PROBLEMS BEING SOLVED - The readability issues with the existing constructs are: * They are awkward to teach, create, read, and review. * Their wording tends to hide the real meaning (accumulation). * The meaning of setdefault() 's method name is not self-evident. +1 to EACH of the above 3 points. A question directed to the folk who had to look up tally in the dictionary: Which dictionary includes setdefault, updateBy, etc? The performance issues with the existing constructs are: [MANY] the performance improvement matches the readability improvement. Agreed. ISSUES -- The proposed names could possibly be improved (perhaps tally() is more active and clear than count()). +3 for tally !!! appendtolist is better than appendlist The appendlist() method is not as versatile as setdefault() which can be used with other object types (perhaps for creating dictionaries of dictionaries). However, most uses I've seen are with lists. My use cases for tally: (1) Yes the text-book word frequency gig applied in production data-matching etc applications (2) quick stats like from SQL group by e.g. customer.tally(state) customer_value.tally(state, dollar_value) # real or *DECIMAL* Use cases for appendlist: many; in general, how else do you implement a one-to-many relationship in memory?? -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
John Machin wrote: Raymond Hettinger wrote: I would like to get everyone's thoughts on two new dictionary methods: +1 for each. PROBLEMS BEING SOLVED - The readability issues with the existing constructs are: * They are awkward to teach, create, read, and review. * Their wording tends to hide the real meaning (accumulation). * The meaning of setdefault() 's method name is not self-evident. +1 to EACH of the above 3 points. A question directed to the folk who had to look up tally in the dictionary: Which dictionary includes setdefault, updateBy, etc? Are you kidding? If you know what set and default means, you will be able to guess what setdefault means. Same goes for updateBy. Of course, I had to look up setdefault in the Python docs the first time I ran across it, but in the case of tally, I would have to look up both in the Python docs and in my dictionary. Reinhold -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Raymond Hettinger [EMAIL PROTECTED] writes: I find the disassembly (presented in the first post) to be telling. The compiler has done a great job and there is no fluff -- all of those steps have been specified by the programmer and he/she must at some level be aware of every one of them (pre-instantiation, multiple method lookups and calls, multiple dictionary accesses, etc). If the compiler can do some type inference, it can optimize out those multiple calls pretty straightforwardly. -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
Reinhold Birkenfeld wrote: John Machin wrote: Are you kidding? If you know what set and default means, you will be able to guess what setdefault means. Same goes for updateBy. No I'm not kidding -- people from some cultures have no difficulty at all in mentally splitting up words like setdefault or the German equivalent of Danubesteamnavigationcompany'sdirector'swife; others from other cultures where agglutinisation is not quite so rife will have extreme difficulty. And Updateby sounds like a village somewhere in the Danelaw :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: Pre-PEP: Dictionary accumulator methods
[Bengt Richter] IMO Raymond's Zen concerns are the ones to think about first, and then efficiency, which was one of the motivators in the first place ;-) Well said. I find the disassembly (presented in the first post) to be telling. The compiler has done a great job and there is no fluff -- all of those steps have been specified by the programmer and he/she must at some level be aware of every one of them (pre-instantiation, multiple method lookups and calls, multiple dictionary accesses, etc). That is too bad because the intent could have been stated atomically: d.appendlist(k, v). Instead, the current idiom turns you away from what you want done and focuses the attention on how it is done: d.setdefault(k, []).append(v). That is too many steps for what should be an atomic operation (in the eyes of the programmer and code readers). Likewise, d.tally(k) is as atomic as this expression can get. Any other steps, added verbiage, new types, extra arguments, or whatnot are an unnecessary waste of brain cells. Raymond Hettinger -- http://mail.python.org/mailman/listinfo/python-list