Re: Pre-PEP: Dictionary accumulator methods

[Steven Bethard]

Greg Ewing wrote:
Steven Bethard wrote:
py> def defaultdict(*args, **kwargs):
... defaultfactory, args = args[0], args[1:]

which can be written more succinctly as
  def defaultdict(defaultfactory, *args, **kwargs):
...
Not if you want to allow the defaultfactory to be called with a keyword 
argument 'defaultfactory'.  Compare my code:

py> def defaultdict(*args, **kwargs):
... defaultfactory, args = args[0], args[1:]
... print defaultfactory, args, kwargs
...
py> defaultdict(dict, defaultfactory=True)
 () {'defaultfactory': True}
with the code you suggested:
py> def defaultdict(defaultfactory, *args, **kwargs):
... print defaultfactory, args, kwargs
...
py> defaultdict(dict, defaultfactory=True)
Traceback (most recent call last):
  File "", line 1, in ?
TypeError: defaultdict() got multiple values for keyword argument 
'defaultfactory'

Uncommon, sure, but I'd rather not rule it out if there's no need to.
STeVe
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Greg Ewing]

Steven Bethard wrote:
py> def defaultdict(*args, **kwargs):
... defaultfactory, args = args[0], args[1:]
which can be written more succinctly as
  def defaultdict(defaultfactory, *args, **kwargs):
...
--
Greg Ewing, Computer Science Dept,
University of Canterbury,   
Christchurch, New Zealand
http://www.cosc.canterbury.ac.nz/~greg
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Michael Spencer]

Jack Diederich wrote:
On Sun, Mar 27, 2005 at 02:20:33PM -0700, Steven Bethard wrote:
Michele Simionato wrote:
I am surprised nobody suggested we put those two methods into a
separate module (say dictutils or even UserDict) as functions:

from dictutils import tally, listappend

tally(mydict, key)
listappend(mydict, key, value)
Sorry to join the discussion so late (I've been away from my email for a 
week) but this was exactly my reaction too.  In fact, I have a 
'dicttools' module with similar methods in it:

I like this approach, it will give us a chance to test & tweak the signature
before hanging it off dict proper.  It feels similar to the strings module 
to str transition, sets module to set builtin, and itertools module to iter
transition.

itertools to iter transition, huh?  I slipped that one in, I mentioned it
to Raymond at PyCon and he didn't flinch.  It would be nice not to have to
sprinkle 'import itertools as it' in code.  
Not that that is such a pain, but accessing itertools functions from an 
"outlying" module seems somewhat incompatible with putting iterative approaches 
center stage.

iter could also become a type
wrapper instead of a function, so an iter instance could be a wrapper that
figures out whether to call .next or __getitem__ depending on it's argument.
for item in iter(mylist).imap:
  print item
Also opening the door for iter to be subclassed.  For example, could listiter 
become a specialization of iter -  one that uses __getitem__ and which could 
allow reverse iteration?

or
for item in iter.imap(mylist):
  print item
I haven't digested that too much, just a thought.
-jackdied
A very good thought IMO.
Michael
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Jack Diederich]

On Sun, Mar 27, 2005 at 02:20:33PM -0700, Steven Bethard wrote:
> Michele Simionato wrote:
> >I am surprised nobody suggested we put those two methods into a
> >separate module (say dictutils or even UserDict) as functions:
> >
> >from dictutils import tally, listappend
> >
> >tally(mydict, key)
> >listappend(mydict, key, value)
> 
> Sorry to join the discussion so late (I've been away from my email for a 
> week) but this was exactly my reaction too.  In fact, I have a 
> 'dicttools' module with similar methods in it:


I like this approach, it will give us a chance to test & tweak the signature
before hanging it off dict proper.  It feels similar to the strings module 
to str transition, sets module to set builtin, and itertools module to iter
transition.

itertools to iter transition, huh?  I slipped that one in, I mentioned it
to Raymond at PyCon and he didn't flinch.  It would be nice not to have to
sprinkle 'import itertools as it' in code.  iter could also become a type
wrapper instead of a function, so an iter instance could be a wrapper that
figures out whether to call .next or __getitem__ depending on it's argument.
for item in iter(mylist).imap:
  print item
or
for item in iter.imap(mylist):
  print item

I haven't digested that too much, just a thought.

-jackdied
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Steven Bethard]

Michele Simionato wrote:
FWIW, here is my take on the defaultdict approach:
def defaultdict(defaultfactory, dictclass=dict):
class defdict(dictclass):
def __getitem__(self, key):
try:
return super(defdict, self).__getitem__(key)
except KeyError:
return self.setdefault(key, defaultfactory())
return defdict
d = defaultdict(int)()
d["x"] += 1
d["x"] += 1
d["y"] += 1
print d
d = defaultdict(list)()
d["x"].append(1)
d["x"].append(2)
d["y"].append(1)
print d
  Michele Simionato
Very pretty! =)
It does mean, however, that if the defaultfactory function takes any 
arguments, you have to wrap the function to make this work.  I'd 
probably prefer something like:

py> def defaultdict(*args, **kwargs):
... defaultfactory, args = args[0], args[1:]
... class defdict(dict):
... def __getitem__(self, key):
... try:
... return super(defdict, self).__getitem__(key)
... except KeyError:
... return self.setdefault(key, defaultfactory(
... *args, **kwargs))
... return defdict
...
py> d = defaultdict(int)()
py> d['x'] += 1
py> d['x'] += 1
py> d['y'] += 1
py> d
{'y': 1, 'x': 2}
py> d = defaultdict(list, [0])()
py> d['x'].append(1)
py> d['x'].append(2)
py> d['y'].append(1)
py> d
{'y': [0, 1], 'x': [0, 1, 2]}
That said, I still think a dictools module is a better solution to this 
problem.

STeVe
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Steven Bethard]

Michele Simionato wrote:
I am surprised nobody suggested we put those two methods into a
separate module (say dictutils or even UserDict) as functions:
from dictutils import tally, listappend
tally(mydict, key)
listappend(mydict, key, value)
Sorry to join the discussion so late (I've been away from my email for a 
week) but this was exactly my reaction too.  In fact, I have a 
'dicttools' module with similar methods in it:

# like "tally" but without ability to set increment
def counts(iterable, key=None):
result = {}
for item in iterable:
# apply key function if necessary
if key is None:
k = item
else:
k = key(item)
# increment key's count
try:
result[k] += 1
except KeyError:
result[k] = 1
return result
# like "listappend" but with the option to use key and value funcs
def groupby(iterable, key=None, value=None):
result = {}
for item in iterable:
# apply key function if necessary
if key is None:
k = item
else:
k = key(item)
# apply value function if necessary
if value is None:
v = item
else:
v = value(item)
# append value to key's list
try:
result[k].append(v)
except KeyError:
result[k] = [v]
return result
These two functions have covered all my use cases for "tally" and 
"listappend" -- I always want to perform the increments or list appends 
over a sequence of values, so having functions that operate on sequences 
covers all my needs.

STeVe
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[TZOTZIOY]

On Sat, 19 Mar 2005 01:24:57 GMT, rumours say that "Raymond Hettinger"
<[EMAIL PROTECTED]> might have written:

>I would like to get everyone's thoughts on two new dictionary methods:
>
>def count(self, value, qty=1):
>try:
>self[key] += qty
>except KeyError:
>self[key] = qty
>
>def appendlist(self, key, *values):
>try:
>self[key].extend(values)
>except KeyError:
>self[key] = list(values)

Both are useful and often needed, so I am +1 on adding such
functionality.  However, I am -0 on adding methods to dict.

I believe BJörn Lindqvist suggested a subtype of dict instead, which
feels more right.  I believe this is a type of 'bag' collection, and it
could go to the collections module.

The default argument 99% of the time is the same for all calls to
setdefault of a specific instance.  So I would suggest that the default
argument should be an attribute of the bag instance, given at instance
creation.  And since unbound methods are going to stay, we can use the
accumulator method as a default argument (ie int.__add__ or list.append)

Based on the above, I would suggest something like the following
implementation, waiting criticism on names, algorithm or applicability:)

.class bag(dict):
.def __init__(self, accumulator=int.__add__):
.self.accumulator = accumulator
.
.# refinement needed for the following
.self.accu_class = accumulator.__objclass__
.
.# if there was an exception, probably the accumulator
.# provided was not appropriate
.
.def accumulate(self, key, value):
.try:
.old_value = self[key]
.except KeyError:
.self[key] = old_value = self.accu_class()
.new_value = self.accumulator(old_value, item)
.
.# and this needs refinement
.if new_value is not None: # method of immutable object
.self[key] = new_value

This works ok for int.__add__ and list.append.

PS I wrote these more than 36 hours ago, and before having read the
so-far downloaded messages of the thread.  I kept on reading and
obviously others thought the same too (default argument at
initialisation).

What the heck, Bengt at least could like the class method idea :)
-- 
TZOTZIOY, I speak England very best.
"Be strict when sending and tolerant when receiving." (from RFC1958)
I really should keep that in mind when talking with people, actually...
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[bearophileHUGS]

R.H.:
> The setdefault() method would continue to exist but
> would likely not make it into Py3.0.

I agee to remove the setdefault.

I like the new count method, but I don't like the appendlist method,
because I think it's too much specilized.

I too use sets a lot; recently I've suggested to add a couple of set
methods to dicts (working on the keys): intersection() and
difference().

Bearophile

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Steve Holden]

Greg Ewing wrote:
Michele Simionato wrote:
def defaultdict(defaultfactory, dictclass=dict):
class defdict(dictclass):
def __getitem__(self, key):
try:
return super(defdict, self).__getitem__(key)
except KeyError:
return self.setdefault(key, defaultfactory())
return defdict

That looks really nice!
I'd prefer a more elegant name than 'defaultdict', though.
How about 'table'?
By obvious analogy with Icon (where the dictionary-like object was 
created with the option of a default value) this gets my +1.

regards
 Steve
--
Meet the Python developers and your c.l.py favorites March 23-25
Come to PyCon DC 2005  http://www.pycon.org/
Steve Holden   http://www.holdenweb.com/
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Roose]

I agree -- I find myself NEEDING sets more and more.  I use them with this
idiom quite often.  Once they become more widely available (i.e. Python 2.3
is installed everywhere), I will use them almost as much as lists I bet.

So any solution IMO needs to at least encompass sets.  But preferably
something like the Dict with Default approach which encompasses all
possibilities.

Roose

> I am more than sure you are right about this. But, please keep in mind
> that you and we all have come very, very accustomed to using lists for
> everything and the kitchen sink in Python.
>
> Lists where there from the beginning of Python, and even before the
> birth of Python; very powerfull, well implemented and theoretically
> well founded datastructures - I heared there is a whole language based
> on list processing. *pun intended*
>
> sets on the other hand --- I know, in mathematics they have a deep,
> long history. But in programming? Yeah, in SQL and ABAP/4 basically
> you are doing set operations on every join. But its rather uncommon to
> call it set.
>
> With 2.3 Python grew a set module. And, in ONLY ONE revision it got
> promoted to a buildin type - a honour only those who read c.l.p.d.
> regualary can value correctly.
>
> And sets are SO NATURALLY for a lot of problems ... I never thought of
> replacing my  "list in dict" constructs with sets before, BUT 
> there are 90% of problem domains where order is not important, AND
> fast membership testing is a unique sales point.
>
> So please for best impressions: let us have a look at our code, where
> we use the
> dict.setdefault(key,[]).append() idiom, where it could be replaced to
> a better effectivity with dict.setdefault(key,set()).add()
>
> If it is less than 60%, forget it. If it is more
>
> Harald


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Greg Ewing]

Michele Simionato wrote:
def defaultdict(defaultfactory, dictclass=dict):
class defdict(dictclass):
def __getitem__(self, key):
try:
return super(defdict, self).__getitem__(key)
except KeyError:
return self.setdefault(key, defaultfactory())
return defdict
That looks really nice!
I'd prefer a more elegant name than 'defaultdict', though.
How about 'table'?
--
Greg Ewing, Computer Science Dept,
University of Canterbury,   
Christchurch, New Zealand
http://www.cosc.canterbury.ac.nz/~greg
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Evan Simpson]

Raymond Hettinger wrote:
I would like to get everyone's thoughts on two new dictionary methods:
def count(self, value, qty=1):
def appendlist(self, key, *values):
-1.0
When I need these, I just use subtype recipes.  They seem way too 
special-purpose for the base dict type.

class Counter(dict):
def __iadd__(self, other):
if other in self:
self[other] += 1
else:
self[other] = 1
return self
c = Counter()
for item in items:
c += item
class Collector(dict):
def add(self, key, value):
if key in self:
self[key].append(value)
else:
self[key] = [value]
c = Collector()
for k,v in items:
c.add(k, v)
Cheers,
Evan @ 4-am
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[George Sakkis]

"Michele Simionato" <[EMAIL PROTECTED]> wrote:

> FWIW, here is my take on the defaultdict approach:
>
> def defaultdict(defaultfactory, dictclass=dict):
> class defdict(dictclass):
> def __getitem__(self, key):
> try:
> return super(defdict, self).__getitem__(key)
> except KeyError:
> return self.setdefault(key, defaultfactory())
> return defdict
>
> d = defaultdict(int)()
> d["x"] += 1
> d["x"] += 1
> d["y"] += 1
> print d
>
> d = defaultdict(list)()
> d["x"].append(1)
> d["x"].append(2)
> d["y"].append(1)
> print d
>
>   Michele Simionato


Best solution so far. If it wasn't for the really bad decision to add the 
dict(**kwargs)
constructor, I'd love to see something like
d = dict(valType=int)
d["x"] += 1

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Michele Simionato]

FWIW, here is my take on the defaultdict approach:

def defaultdict(defaultfactory, dictclass=dict):
class defdict(dictclass):
def __getitem__(self, key):
try:
return super(defdict, self).__getitem__(key)
except KeyError:
return self.setdefault(key, defaultfactory())
return defdict

d = defaultdict(int)()
d["x"] += 1
d["x"] += 1
d["y"] += 1
print d

d = defaultdict(list)()
d["x"].append(1)
d["x"].append(2)
d["y"].append(1)
print d

  Michele Simionato

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[AndrewN]

Raymond Hettinger wrote:
> I would like to get everyone's thoughts on two new dictionary
methods:
>
> def count(self, value, qty=1):
> try:
> self[key] += qty
> except KeyError:
> self[key] = qty
>
> def appendlist(self, key, *values):
> try:
> self[key].extend(values)
> except KeyError:
> self[key] = list(values)
>

-0.9

Not impressed, they feel too specific for being builtin dictionary
methods and give the impression of just trying to save a few lines here
and there. I don't feel the names convey the functionality of the
methods either.

I know there's the speed argument but I'd rather not have these on the
dict at all.

+0.1

I sort of feel a slight need for this. But where would you stop? What
if people decrement lots? what if next there's a need for division? How
would you determine how you add the item to the key if it already
exists? In a general way:

mydict.set(key, value=None, default=None, how=operator.setitem)

This feels slightly better as it's not tied down to what sort of item
you're setting. But:

>>> for word in words:
>>> mydict.set(word, 1, 0, operator.add)

I dunno, feels a bit verbose maybe.

> The setdefault() method would continue to exist but would likely not
make it
> into Py3.0.

I agree that setdefault is wart though.

And for dict.default = value:

(Quoth RON):

"""With a preset default mode, it then becomes possible to
inadvertently
create default values that will cause problems without knowing it.  So
then we have to remember to change the setdefault value to None or
null to avoid problems.  Ouch!"""

Agreed, -1 there then.

> PROBLEMS BEING SOLVED
> -
>
> The readability issues with the existing constructs are:
>
> * They are awkward to teach, create, read, and review.
> * Their wording tends to hide the real meaning (accumulation).
> * The meaning of setdefault() 's method name is not self-evident.

I feel this only really applies for setdefault (which I wouldn't be
sorry to see the back of). And your examples:

d[key] = d.get(key, 0) + qty
d.setdefault(key, []).extend(values)

Would better be written in a long-handed fashion anyway as per the
implementations were suggested:

try:
d[key] += qty
except KeyError:
d[key] = 0

Yeah, yeah, I know, speed. But not like this. Sorry.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[haraldarminmassa]

Raymond,

I am +1 for both suggestions, tally and appendlist. 

Extended:
 
> Also, in all of my code base, I've not run across a single opportunity to use
> something like unionset().  This is surprising because I'm the set() author 
> and
> frequently use set based algorithms.Your example was a good one and I can
> also image a graph represented as a dictionary of sets.  Still, I don't mind
> writing out the plain Python for this one if it only comes up once in a blue
> moon.

I am more than sure you are right about this. But, please keep in mind
that you and we all have come very, very accustomed to using lists for
everything and the kitchen sink in Python.

Lists where there from the beginning of Python, and even before the
birth of Python; very powerfull, well implemented and theoretically
well founded datastructures - I heared there is a whole language based
on list processing. *pun intended*

sets on the other hand --- I know, in mathematics they have a deep,
long history. But in programming? Yeah, in SQL and ABAP/4 basically
you are doing set operations on every join. But its rather uncommon to
call it set.

With 2.3 Python grew a set module. And, in ONLY ONE revision it got
promoted to a buildin type - a honour only those who read c.l.p.d.
regualary can value correctly.

And sets are SO NATURALLY for a lot of problems ... I never thought of
replacing my  "list in dict" constructs with sets before, BUT 
there are 90% of problem domains where order is not important, AND
fast membership testing is a unique sales point.

So please for best impressions: let us have a look at our code, where
we use the
dict.setdefault(key,[]).append() idiom, where it could be replaced to
a better effectivity with dict.setdefault(key,set()).add()

If it is less than 60%, forget it. If it is more 

Harald
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Ron]

On Sun, 20 Mar 2005 15:14:22 -0800, David Eppstein
<[EMAIL PROTECTED]> wrote:

>In article <[EMAIL PROTECTED]>, [EMAIL PROTECTED] (Aahz) 
>wrote:
>
>> >I am surprised nobody suggested we put those two methods into a
>> >separate module (say dictutils or even UserDict) as functions:
>> >
>> >from dictutils import tally, listappend
>> >
>> >tally(mydict, key)
>> >listappend(mydict, key, value)
>> 
>> That seems like a reasonable compromise.
>
>The more messages I see on this thread, the more I think adding a 
>different new method for each commonly used kind of update is the wrong 
>solution.
>
>We already have methods that work pretty well and, I think, read better 
>than the new methods:
>  mydict[key] += 1
>  mydict[key].append(value)
>The problem is merely that they don't work when key is missing, so we 
>need to resort to setdefault circumlocutions instead.  A better solution 
>seems to be the one I've seen suggested here several times, of changing 
>the dict's behavior so that the setdefault is automatic whenever trying 
>to access a missing key.  If this would be in a separate module or 
>separate subclass of dict, so much the better.


I think that the setdefault behavior needs to be done on an per
application basis because whose to say what default is best?.

With a preset default mode, it then becomes possible to  inadvertently
create default values that will cause problems without knowing it.  So
then we have to remember to change the setdefault value to None or
null to avoid problems.  Ouch!

Also pythons normal behavior for retrieving objects that are not
defined is to give an error.  So having dictionaries that auto
defaults to a mode that doesn't behave that way is inconsistent with
the rest of the language.

Yet,  I'm all for the creation of specialized containers in a standard
module!  :)  Then we can have string dicts, and int dicts, and card
dicts, account dicts, etc, as well as specialized lists.  Call them
'smart containers'.  But they should not be built into the base class.

Ron

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[David Eppstein]

In article <[EMAIL PROTECTED]>, [EMAIL PROTECTED] (Aahz) 
wrote:

> >I am surprised nobody suggested we put those two methods into a
> >separate module (say dictutils or even UserDict) as functions:
> >
> >from dictutils import tally, listappend
> >
> >tally(mydict, key)
> >listappend(mydict, key, value)
> 
> That seems like a reasonable compromise.

The more messages I see on this thread, the more I think adding a 
different new method for each commonly used kind of update is the wrong 
solution.

We already have methods that work pretty well and, I think, read better 
than the new methods:
  mydict[key] += 1
  mydict[key].append(value)
The problem is merely that they don't work when key is missing, so we 
need to resort to setdefault circumlocutions instead.  A better solution 
seems to be the one I've seen suggested here several times, of changing 
the dict's behavior so that the setdefault is automatic whenever trying 
to access a missing key.  If this would be in a separate module or 
separate subclass of dict, so much the better.

-- 
David Eppstein
Computer Science Dept., Univ. of California, Irvine
http://www.ics.uci.edu/~eppstein/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Aahz]

In article <[EMAIL PROTECTED]>,
Michele Simionato <[EMAIL PROTECTED]> wrote:
>
>I am surprised nobody suggested we put those two methods into a
>separate module (say dictutils or even UserDict) as functions:
>
>from dictutils import tally, listappend
>
>tally(mydict, key)
>listappend(mydict, key, value)

That seems like a reasonable compromise.
-- 
Aahz ([EMAIL PROTECTED])   <*> http://www.pythoncraft.com/

"The joy of coding Python should be in seeing short, concise, readable
classes that express a lot of action in a small amount of clear code -- 
not in reams of trivial code that bores the reader to death."  --GvR
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Mike Rovner]

Reinhold Birkenfeld wrote:
I don't quite understand that. Which dict item are you extending? Don't
you need something like
dl[key].append("word")
Rigth. It was just a typo on my part. Thanks for fixing.
Mike
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Francis Girard]

Hi,

I really do not like it. So -1 for me. Your two methods are very specialized 
whereas the dict type is very generic. Usually, when I see something like 
this in code, I can smell it's a patch to overcome some shortcomings on a 
previous design, thereby making the economy of re-designing. Simply said, 
that's bad programming.

After that patch to provide a solution for only two of the more common 
use-cases, you are nonetheless stucked with the old solution for all the 
other use-cases (what if the value type is another dictionary or some 
user-made class ?).

Here's an alternate solution which I think answers all of the problems you 
mentionned while being generic.

=== BEGIN SNAP

def update_or_another_great_name(self, key, createFunc, updtFunc):
  try:
self[key] <<<= updtFunc(self[key])
  ## This is "slow" with Python "=" since the key has to be  searched 
  ## twice But the new built-in method just has to update the value the
  ## first time the key is found. Therefore speed should be ok.
return True
  except KeyError:
self[key] = createFunc()
return false

## Now your two specialized methods can be easily written as :

## A built-in should be provided for this (if not already proposed) :
def identical(val):
  return val

def count(self, key, qty=1):
  self.update_or_another_great_name(key, identical, 
partial(operator.add, qty))
  ## partial is coming from : http://www.python.org/peps/pep-0309.html
  ## Using only built-in function (assuming "identical") as arguments makes it
  ## ok for speed (I guess).
  
def appendlist(self, key, *values):
  self.update_or_another_great_name(key, 
partial(list, values),
partial(ListType.extend, X = values))
  ## The first "partial" usage here is an abuse just to make sure that the
  ## list is not actually constructed before needed. It should work.
  ## The second usage is more uncertain as we need to bind the arguments from
  ## the right. Therefore I have to use the name of the parameter and I am not
  ## sure if there's one. As this list is very prolific, someone might have an
  ## idea on how to improve this.
  
=== END SNAP

By using only built-in constructs, this should be fast enough. Otherwise, 
optimizing these built-ins is a much more clean and sane way of thinking then 
messing the API with ad-hoc propositions.

Reviewing the problems you mention :

> The readability issues with the existing constructs are:
>
> * They are awkward to teach, create, read, and review.

The method update_or_another_great_name is easy to understand, I think. But it 
might not always be easy to use it efficiently with built-ins. But this is 
always the case. "Recipees" can be added to show how to efficiently use the 
method.

> * Their wording tends to hide the real meaning (accumulation).

Solved.

> * The meaning of setdefault() 's method name is not self-evident.

Solved.

>
> The performance issues with the existing constructs are:
>
> * They translate into many opcodes which slows them considerably.

I really don't know what will be the outcome of the solution I propose. I 
certainly do not know anything about how my Python code translates into 
opcodes.

> * The get() idiom requires two dictionary lookups of the same key.

Solved

> * The setdefault() idiom instantiates a new, empty list prior to every

Solved

> call. * That new list is often not needed and is immediately discarded.

Solved

> * The setdefault() idiom requires an attribute lookup for extend/append.

Solved

> * The setdefault() idiom makes two function calls.

Solved

And perhaps, what you say here is also true for your two special use-cases :

> For other
> uses, plain Python code suffices in terms of speed, clarity, and avoiding
> unnecessary instantiation of empty containers:
>
> if key not in d:
> d.key = {subkey:value}
> else:
> d[key][subkey] = value


Much better than adding special cases on a generic class. Special cases always 
demultiply and if we open the door 

Regards,

Francis Girard


Le samedi 19 Mars 2005 02:24, Raymond Hettinger a ÃcritÂ:
> I would like to get everyone's thoughts on two new dictionary methods:
>
> def count(self, value, qty=1):
> try:
> self[key] += qty
> except KeyError:
> self[key] = qty
>
> def appendlist(self, key, *values):
> try:
> self[key].extend(values)
> except KeyError:
> self[key] = list(values)
>
> The rationale is to replace the awkward and slow existing idioms for
> dictionary based accumulation:
>
> d[key] = d.get(key, 0) + qty
> d.setdefault(key, []).extend(values)
>
> In simplest form, those two statements would now be coded more readably as:
>
>d.count(key)
>d.appendlist(key, value)
>
> In their multi-value forms, they would now be coded as:
>

Re: Pre-PEP: Dictionary accumulator methods

[Alexander Schmolck]

Beni Cherniavsky <[EMAIL PROTECTED]> writes:

>> The relatively recent "improvement" of the dict constructor signature
>> (``dict(foo=bar,...)``) obviously makes it impossible to just extend the
>> constructor to ``dict(default=...)`` (or anything else for that matter) which
>> would seem much less ad hoc. But why not use a classmethod (e.g.
>> ``d=dict.withdefault(0)``) then?
>>
> You mean giving a dictionary a default value at creation time, right?

Yes. But creating a defaultdict type with aliased content to the original
dict would also be fine by me.

> Such a dictionary could be used very easily, as in Perl::
>
>  foreach $word ( @words ) {
>  $d{$word}++; # default of 0 assumed, simple code!
>  }
>
> .  You would like to write::
>
>  d = dict.withdefault(0)  # or something
>  for word in words:
>  d[word] += 1 # again, simple code!
>
> I agree that it's a good idea but I'm not sure the default should be specified
> at creation time.  The problem with that is that if you pass such a dictionary
> into an unsuspecting function, it will not behave like a normal dictionary.

Have you got a specific example in mind? 

Code that needlessly relies on exceptions for "normal operation" is rather
perverse IMO and I find it hard to think of other examples.

> Also, this will go awry if the default is a mutable object, like ``[]`` - you
> must create a new one at every access (or introduce a rule that the object is
> copied every time, which I dislike).

I think copying should on by default for objects that are mutable (and
explicitly selectable via ``.withdefault(bar,copy=False)``).

Python of course doesn't have an interface to query whether something is
mutable or not (maybe something that'll eventually be fixed), but hashable
might be a first approximation. If that's too dodgy, most commonly the value
will be a builtin type anyway, so copy by default with "efficient
implementation" (i.e. doing nothing) for ints, tuples etc. ought to work fine
in practice.

> And there are cases where in different points in the code operating on the
> same dictionary you need different default values.

The main problem here is that the obvious .setdefault is already taken to
misnome something else. Which I guess strengthens the point for some kind of
proxy object.

> So perhaps specifying the default at every point of use by creating a proxy is
> cleaner::
>
>  d = {}
>  for word in words:
>  d.withdefault(0)[word] += 1
> Of course, you can always create the proxy once and still pass it into an
> unsuspecting function when that is actually what you mean.

Yup (I'd presumably prefer that second option for the above code).

>
> How should a dictionary with a default value behave (wheter inherently or a
> proxy)?
>
> - ``d.__getattr__(key)`` never raises KeyError for missing keys - instead it
>returns the default value and stores the value as `d.setdefult()` does.
>This is needed for make code like::
>
>d.withdefault([])[key].append(foo)
>
>to work - there is no call of `d.__setattr__()`, so `d.__getattr__()` must
>have stored it.

I'm confused -- are you refering to __getitem__/__setitem__? Even then I don't
get what you mean: __getitem__ obviously works differently, but that would be
the whole point.

>
>- `d.__setattr__()` and `d.__delattr__()` behave normally.

s/attr/item/ and agreed.

>
> - Should ``key in d`` return True for all keys?  

No. See below.

>It is desiarable to have *some* way to know whether a key is really
>present. But if it returns False for missing keys, code that checks ``key
>in d`` will behave differently from normally equivallent code that uses
>try..except. If we use the proxy interface, we can always check on the
>original dictionary object, which removes the problem.
>
>- ``d.has_key(key)`` must do whatever we decide ``key in d`` does.
>
>  - What should ``d.get(key, [default])`` and ``d.setdefault(key, default)``
>do?  There is a conflict between the default of `d` and the explicitly 
> given
>default.  I think consistency is better and these should pretend that `key`
>is always present.  But either way, there is a subtle problem here.

.setdefault ought to trump defaultdict's default. I feel that code that
operated without raising an KeyError on normal dicts should also operate the
same way on defaultdicts were possible. I'd also suspect that if you're
effectively desiring to override .setdefault's default you're up to something
dodgy.

> - Of course `iter(d)`, `d.items()` and the like should only see the keys
>that are really present (the alternative inventing an infinite amount of
>items out of the blue is clearly bogus).
>
> If the idea that the default should be specified in every operation (creating
> a proxy) is accepted, there is a simpler and more fool-proof solution: the
> ptoxy will not support anything except `__getitem__()` and `__setitem__()` at
> al

Re: Pre-PEP: Dictionary accumulator methods - typing & initialising

[Matteo Dell'Amico]

Kay Schluehr wrote:
I think that's because you have to instantiate a different object for
each different key. Otherwise, you would instantiate just one list as
a default value for *all* default values.
Or the default value will be copied, which is not very hard either or
type(self._default)() will be called. This is all equivalent and it
does not matter ( except for performance reasons ) which way to go as
long only one is selected.
I don't like it very much... it seems too implicit to be pythonic. Also, 
it won't work with non-copyable objects, and type(42)() = 0, and getting 
0 when the default is 42 looks very strange. I prefer the explicit "give 
me a callable" approach.

If the dict has a fixed semantics by applying defaultValue() and it
returns defaults instead of exceptions whenever a key is missing i.e.
behavioural invariance the client of the dict has nothing to worry
about, hasn't he?
For idioms like d[foo].append('blah') to work properly, you'd have to 
set the default value every time you access a variable. It can be really 
strange to fill up memory only by apparently accessing values.

I suspect the proposal really makes sense only if the dict-values are
of the same type. Filling it with strings, custom objects and other
stuff and receiving 0 or [] or '' if a key is missing would be a
surprise - at least for me. Instantiating dict the way I proposed
indicates type-guards! This is the reason why I want to delay this
issue and discuss it in a broader context. But I'm also undecided.
Guidos Python-3000 musings are in danger to become vaporware. "Now is
better then never"... Therefore +0.
Having duck-typing, we can have things that have common interface but no 
common type. For instance, iterables. I can imagine a list of iterables 
of different types, and a default value of maybe [] or set([]).

--
Ciao,
Matteo
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Jeremy Bowers]

On Sat, 19 Mar 2005 20:07:40 -0800, Kay Schluehr wrote:
> It is bad OO design, George. I want to be a bit more become more
> specific on this and provide an example:

Having thought about this for a bit, I agree it is a powerful
counter-argument and in many other languages I'd consider this a total win.

But this is Python, and frankly, I've overridden dict more than once and
violated the Liskov substitution principle without thinking twice. (Once,
oh yes, but not twice.) Of course, all the code was under my control then.

I think the tipping point for me depends on how the interfaces in Python
are going to be implemented, which I haven't dug into. If the dict class
gets an interface definition, can I subclass from dict and "cancel" (or
some other term) the interface I inherited? 

If so, then this might still be OK, although if interfaces aren't going to
confuse newbies enough, wait 'till we try to explain that their code is
blowing up because they forgot to "cancel" their interface, and they
can't *really* pass their subclass in to something expecting a dict
interface. If you *can't* cancel or downgrade the interface, then I'd say
this argument is still good; dict should be kept minimal and this should
go in a subclass.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Roose]

> Another option with no storage overhead which goes part way to reducing
> the awkwardness would be to provide a decorator class accessible through
> dict. The decorator class would take a value or function to be used as
> the default, but apart from __getitem__ would simply forward all other
> methods through to the underlying dictionary.

I'm not sure I like the decorator -- I would never use that flexibility to
have more than one default.  I can't come up with any reason to ever use
that.

I think it works best as a simple subclass:

class DefaultDict(dict):

  def __init__(self, default, *args, **kwargs):
dict.__init__(self, *args, **kwargs)
self.default = default

  def __getitem__(self, key):
return self.setdefault(key, copy.copy(self.default))

d = DefaultDict(0)
for x in [1, 3, 1, 2, 2, 3, 3, 3, 3]:
  d[x] += 1
pprint(d)

d = DefaultDict([])
for i, x in enumerate([1, 3, 1, 2, 2, 3, 3, 3, 3]):
  d[x].append(i)
pprint(d)

Output:

{1: 2, 2: 2, 3: 5}
{1: [0, 2], 2: [3, 4], 3: [1, 5, 6, 7, 8]}


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Colin J. Williams]

Paul Rubin wrote:
"El Pitonero" <[EMAIL PROTECTED]> writes:
What about no name at all for the scalar case:
a['hello'] += 1
a['bye'] -= 2

I like this despite the minor surprise that it works even when
a['hello'] is uninitialized.
+1
and if the value is a list:
a['hello']= [1, 2, 3]
a['hello']+= [4]# adding the brackets is a lot simpler than
  typing append or extend.
Any user is free to add his/her own subclass to handle defaults.
Colin W.
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods - typing & initialising

[Kay Schluehr]

Matteo Dell'Amico wrote:
> Kay Schluehr wrote:
>
> > Why do You set
> >
> > d.defaultValue(0)
> > d.defaultValue(function=list)
> >
> > but not
> >
> > d.defaultValue(0)
> > d.defaultValue([])
> >
> > ?
>
> I think that's because you have to instantiate a different object for

> each different key. Otherwise, you would instantiate just one list as
a
> default value for *all* default values.

Or the default value will be copied, which is not very hard either or
type(self._default)() will be called. This is all equivalent and it
does not matter ( except for performance reasons ) which way to go as
long only one is selected.

[...]

> By the way, to really work, I think that Duncan's proposal should
create
> new objects when you try to access them, and to me it seems a bit
> counterintuitive.

If the dict has a fixed semantics by applying defaultValue() and it
returns defaults instead of exceptions whenever a key is missing i.e.
behavioural invariance the client of the dict has nothing to worry
about, hasn't he?


> > And why not dict(type=int), dict(type=list) instead where default
> > values are instantiated during object creation? A consistent
pythonic
> > handling of all types should be envisioned not some ad hoc
solutions
> > that go deprecated two Python releases later.
>
> I don't really understand you. What should 'type' return?
> A callable
> that returns a new default value? That's exactly what Duncan proposed

> with the "function" keyword argument.

I suspect the proposal really makes sense only if the dict-values are
of the same type. Filling it with strings, custom objects and other
stuff and receiving 0 or [] or '' if a key is missing would be a
surprise - at least for me. Instantiating dict the way I proposed
indicates type-guards! This is the reason why I want to delay this
issue and discuss it in a broader context. But I'm also undecided.
Guidos Python-3000 musings are in danger to become vaporware. "Now is
better then never"... Therefore +0.

Regards Kay

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Duncan Booth]

Roose wrote:

> I think someone mentioned that it might be a problem to add another
> piece of state to all dicts though.  I don't know enough about the
> internals to say anything about this.
> 
> setdefault gets around this by having you pass in the value every
> time, so it doesn't have to store it.  It's very similar, but somehow
> many times more awkward.
> 

Another option with no storage overhead which goes part way to reducing
the awkwardness would be to provide a decorator class accessible through
dict. The decorator class would take a value or function to be used as
the default, but apart from __getitem__ would simply forward all other
methods through to the underlying dictionary. 

That gives you the ability to have not one default value for a
dictionary, but many different ones: you just decorate the dictionary
anywhere you need a default and use the underlying dictionary everywhere
else. 

Some code which demonstrates the principle rather than the
implementation. dictDefaultValue could be imagined as
dict.defaultValue, dictDefaultValue(d, ...) could be 
d.defaultValue(...) although the actual name used needs work:

>>> class dictDefaultValue(object):
def __init__(self, d, value=_marker, function=_marker):
self.__d = d
if value is _marker:
if function is _marker:
raise TypeError, "expected either value or function 
argument"
self.__dv = function
else:
def defaultValue():
return value
self.__dv = defaultValue

def __getattr__(self, name):
return getattr(self.__d, name)

def __getitem__(self, name):
try:
return self.__d[name]
except KeyError:
value = self.__dv()
self.__d[name] = value
return value

def __setitem__(self, name, value):
self.__d[name] = value


>>> d = {}
>>> accumulator = dictDefaultValue(d, 0)
>>> accumulator['a'] += 1
>>> aggregator = dictDefaultValue(d, function=list)
>>> aggregator['b'].append('hello')
>>> d
{'a': 1, 'b': ['hello']}
>>> 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[BJörn Lindqvist]

I like count() and appendlist() or whatever they will be named. But I
have one question/idea:

Why does the methods have to be put in dict? Can't their be a subtype
of dict that includes those two methods? I.e.:

.histogram = counting_dict()
.for ch in text:
.histogram.count(ch)

Then maybe some more methods can be added tailor-mode for these two
types of dicts?:

.for ch in string.ascii_letters:
.print "Frequency of %s = %d." % (ch, histogram.freq(ch))

Or something, you get the idea.

-- 
mvh Björn
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Magnus Lie Hetland]

In article <[EMAIL PROTECTED]>, Raymond Hettinger wrote:
>I would like to get everyone's thoughts on two new dictionary methods:
>
>def count(self, value, qty=1):
>try:
>self[key] += qty
>except KeyError:
>self[key] = qty

Yes, yes, YES!

*Man*, this would be useful.

>def appendlist(self, key, *values):
>try:
>self[key].extend(values)
>except KeyError:
>self[key] = list(values)

Woohoo! *Just* as useful.

I'd *definitely* use these.

Hot 100% sure about the names, though. (add() and append() seem like
more natural names -- but they may be confusing, considering their
other uses...)

+1 on both (possibly allowing for some naming discussion...)

-- 
Magnus Lie Hetland   Time flies like the wind. Fruit flies
http://hetland.org   like bananas. -- Groucho Marx
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods - typing & initialising

[Matteo Dell'Amico]

Kay Schluehr wrote:
Why do You set
d.defaultValue(0)
d.defaultValue(function=list)
but not
d.defaultValue(0)
d.defaultValue([])
?
I think that's because you have to instantiate a different object for 
each different key. Otherwise, you would instantiate just one list as a 
default value for *all* default values. In other words, given:

class DefDict(dict):
def __init__(self, default):
self.default = default
def __getitem__(self, item):
try:
return dict.__getitem__(self, item)
except KeyError:
return self.default
you'll get
In [12]: d = DefDict([])
In [13]: d[42].extend(['foo'])
In [14]: d.default
Out[14]: ['foo']
In [15]: d[10].extend(['bar'])
In [16]: d.default
Out[16]: ['foo', 'bar']
In [17]: d[10]
Out[17]: ['foo', 'bar']
In [18]: d[10] is d.default
Out[18]: True
and this isn't what you really wanted.
By the way, to really work, I think that Duncan's proposal should create 
new objects when you try to access them, and to me it seems a bit 
counterintuitive. Nevertheless, I'm +0 on it.

And why not dict(type=int), dict(type=list) instead where default
values are instantiated during object creation? A consistent pythonic
handling of all types should be envisioned not some ad hoc solutions
that go deprecated two Python releases later.
I don't really understand you. What should 'type' return? A callable 
that returns a new default value? That's exactly what Duncan proposed 
with the "function" keyword argument.

--
Ciao,
Matteo
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Peter Hansen]

Max wrote:
Has anyone _not_ heard Jeff Probst say, "I'll go tally the votes"?!
:)
Who is Jeff Probst?
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Max]

Paul Rubin wrote:
Reinhold Birkenfeld <[EMAIL PROTECTED]> writes:
Any takers for tally()?
Well, as a non-native speaker, I had to look up this one in my
dictionary. That said, it may be bad luck on my side, but it may be that
this word is relatively uncommon and there are many others who would be
happier with increment.

It is sort of an uncommon word.  As a US English speaker I'd say it
sounds a bit old-fashioned, except when used idiomatically ("let's
tally up the posts about accumulator messages") or in nonstandard
dialect ("Hey mister tally man, tally me banana" is a song about
working on plantations in Jamaica).  It may be more common in UK
English.  There's an expression "tally-ho!" which had something to do
with British fox hunts, but they don't have those any more.
Has anyone _not_ heard Jeff Probst say, "I'll go tally the votes"?!
:)
--Max
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods - typing & initialising

[Kay Schluehr]

Duncan Booth wrote:
> Raymond Hettinger wrote:
>
> > The rationale is to replace the awkward and slow existing idioms
for
> > dictionary based accumulation:
> >
> > d[key] = d.get(key, 0) + qty
> > d.setdefault(key, []).extend(values)
> >
>
> How about the alternative approach of allowing the user to override
the
> action to be taken when accessing a non-existent key?
>
>d.defaultValue(0)
>
> and the accumulation becomes:
>
>d[key] += 1
>
> and:
>
>d.defaultValue(function=list)
>
> would allow a safe:
>
>   d[key].extend(values)

+0

The best suggestion up to now. But i find this premature because it
addresses only a special aspect of typing issues which should be
disussed together with Guidos type guard proposals in a broader
context. Besides this the suggestion though feeling pythonic is still
uneven.

Why do You set

d.defaultValue(0)
d.defaultValue(function=list)

but not

d.defaultValue(0)
d.defaultValue([])

?

And why not dict(type=int), dict(type=list) instead where default
values are instantiated during object creation? A consistent pythonic
handling of all types should be envisioned not some ad hoc solutions
that go deprecated two Python releases later.

Regards Kay

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Roose]

> How about the alternative approach of allowing the user to override the
> action to be taken when accessing a non-existent key?
>
>d.defaultValue(0)

I like this a lot.  It makes it more clear from the code what is going on,
rather than having to figure out what the name appendlist, count, tally,
whatever, is supposed to mean.  When you see the value you'll know.

It's more general, because you can support dictionaries and sets then as
well.

I think someone mentioned that it might be a problem to add another piece of
state to all dicts though.  I don't know enough about the internals to say
anything about this.

setdefault gets around this by having you pass in the value every time, so
it doesn't have to store it.  It's very similar, but somehow many times more
awkward.



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Reinhold Birkenfeld]

George Sakkis wrote:
>> -1 form me.
>>
>> I'm not very glad with both of them ( not a naming issue ) because i
>> think that the dict type should offer only methods that apply to each
>> dict whatever it contains. count() specializes to dict values that are
>> addable and appendlist to those that are extendable. Why not
>> subtractable, dividable or right-shiftable? Because of majority
>> approval? I'm mot a speed fetishist and destroying the clarity of a
>> very fundamental data structure for speedup rather arbitrary
>> accumulations seems to be a bad idea. I would move this stuff in a
>> subclass.
>>
>> Regards Kay
> 
> +1 on this. The new suggested operations are meaningful for a subset of all 
> valid dicts, so they
> should not be part of the base dict API. If any version of this is approved, 
> it will clearly be an
> application of the "practicality beats purity" zen rule, and the 
> justification for applying it in
> this case instead of subclassing should better be pretty strong; so far I'm 
> not convinced though.

So, would the `setdefaultvalue' approach be more consistent in your eyes?

Reinhold
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Reinhold Birkenfeld]

John Machin wrote:
> Reinhold Birkenfeld wrote:
>> John Machin wrote:
>> Are you kidding? If you know what "set" and "default" means, you will
> be
>> able to guess what "setdefault" means. Same goes for updateBy.
>>
> 
> No I'm not kidding -- people from some cultures have no difficulty at
> all in mentally splitting up "words" like "setdefault" or the German
> equivalent of "Danubesteamnavigationcompany'sdirector'swife"; others
> from other cultures where agglutinisation is not quite so rife will
> have extreme difficulty.

Okay - as I'm German I might be preoccupied on this matter 

Reinhold
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Reinhold Birkenfeld]

Mike Rovner wrote:
> Paul Rubin wrote:
> 
>> If the compiler can do some type inference, it can optimize out those
>> multiple calls pretty straightforwardly.
> 
> It can be tipped like that:
> 
> di = dict(int)
> di.setdefault(0)
> di[key] += 1

Interesting, but why do you need to give the int type to the constructor?

> dl = dict(list)
> dl.setdefault([])
> dl.append("word")
> dl.extend(mylist)

I don't quite understand that. Which dict item are you extending? Don't
you need something like

dl[key].append("word")

?

Anyway, using `setdefault' as the method name is quite confusing,
although yours is IMHO a much better behavior given the name ;)

So what about `setdefaultvalue'?

Reinhold
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Matteo Dell'Amico]

Raymond Hettinger wrote:
I would like to get everyone's thoughts on two new dictionary methods:
def count(self, value, qty=1):
try:
self[key] += qty
except KeyError:
self[key] = qty
def appendlist(self, key, *values):
try:
self[key].extend(values)
except KeyError:
self[key] = list(values)
They look as a special-case to me. They don't solve the problem for 
lists of sets or lists of deques for instance, not to mention other 
possible user-defined containers.

defaultdicts look to me as a solution that is more elegant and solves 
more problems. What is the problem with them?

--
Ciao,
Matteo
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Duncan Booth]

Raymond Hettinger wrote:

> The rationale is to replace the awkward and slow existing idioms for
> dictionary based accumulation:
> 
> d[key] = d.get(key, 0) + qty
> d.setdefault(key, []).extend(values)
> 

How about the alternative approach of allowing the user to override the 
action to be taken when accessing a non-existent key?

   d.defaultValue(0)

and the accumulation becomes:

   d[key] += 1

and:

   d.defaultValue(function=list)

would allow a safe:

  d[key].extend(values)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Paul Rubin]

Mike Rovner <[EMAIL PROTECTED]> writes:
> It can be tipped like that:
> 
> di = dict(int)
> di.setdefault(0)
> di[key] += 1
...
> But the point is that if method not found in dict it delegated to
> container type specified in constructor.
> 
> It solves dict specialization without bloating dict class and is generic.

Hey, I like that.  I'd let the default be an optional extra arg to the
constructor:

   di = dict(int, default=0)
   di[key] += 1

without the setdefault.  I might even add optional type checking:

  di = dict(int, default=0, typecheck=True)
  di[key] = 'foo'# raises TypeError
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Mike Rovner]

Paul Rubin wrote:
If the compiler can do some type inference, it can optimize out those
multiple calls pretty straightforwardly.
It can be tipped like that:
di = dict(int)
di.setdefault(0)
di[key] += 1
dl = dict(list)
dl.setdefault([])
dl.append("word")
dl.extend(mylist)
But the point is that if method not found in dict it delegated to 
container type specified in constructor.

It solves dict specialization without bloating dict class and is generic.
Mike
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Kay Schluehr]

*pling* !

I'm sometimes a bit slow :)

Regards Kay

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[George Sakkis]

"Michael Spencer" <[EMAIL PROTECTED]> wrote:

> I could imagine a class: accumulator(mapping, default, incremetor) such that:
>   >>> my_tally = accumulator({}, 0, operator.add)
>   or
>   >>> my_dict_of_lists = accumulator({}, [], list.append)
> or
>   >>> my_dict_of_sets = accumulator({}, set(), set.add)
>
> then: .accumulate(key, value) "does the right thing" in each case.
>
> a bit cumbersome, because of having to specify the accumulation method, but
> avoids adding methods to, or sub-classing dict
>
> Michael

That's the equivalent of reduce() for mappings. Given the current trend of 
moving away from
traditional functional features (lambda,map,filter,etc.), I would guess it's 
not likely to become
mainstream.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Ron]

On Sat, 19 Mar 2005 01:24:57 GMT, "Raymond Hettinger"
<[EMAIL PROTECTED]> wrote:

>def count(self, value, qty=1):
>try:
>self[key] += qty
>except KeyError:
>self[key] = qty
>
>def appendlist(self, key, *values):
>try:
>self[key].extend(values)
>except KeyError:
>self[key] = list(values)


Why is it better than this? 

dict[key]+=n
dict[key]+=list  

Ron

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[George Sakkis]

> > +1 on this. The new suggested operations are meaningful for a subset
> of all
> > valid dicts, so they
> > should not be part of the base dict API. If any version of this is
> approved, > it will clearly be an
> > application of the "practicality beats purity" zen rule, and the
> > justification for applying it in
> > this case instead of subclassing should better be pretty strong; so
> far I'm
> > not convinced though.
> >
> > George
>
> It is bad OO design, George. I want to be a bit more become more
> specific on this and provide an example:

Kay, the '+1' was for your post, not the pre-PEP; I fully agree with you in 
that it's bad design. I
just tried to play devil's advocate by citing an argument that might be used to 
back the addition of
the proposed accumulating methods.

Regards,
George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Michele Simionato]

I am surprised nobody suggested we put those two methods into a
separate module (say dictutils or even UserDict) as functions:

from dictutils import tally, listappend

tally(mydict, key)
listappend(mydict, key, value)

I am -1 about a specific subclass of dict in the standard library, I
would not mind about a few new functions
in an utility module.

  Michele Simionato

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Michael Spencer]

Kay Schluehr wrote:
Maybe also the subclassing idea I introduced falls for short for the
same reasons. Adding an accumulator to a dict should be implemented as
a *decorator* pattern in the GoF meaning of the word i.e. adding an
interface to some object at runtime that provides special facilities.
Usage:

d = intdict(extend=MyAccumulator)
hasattr(d,"tally")
True
hasattr(d,"appendlist")
False
This could be generalized to other fundamental data structures as well.
Regards Kay
Or similarly, something like the 'reversed' view of a sequence:
I could imagine a class: accumulator(mapping, default, incremetor) such that:
 >>> my_tally = accumulator({}, 0, operator.add)
 or
 >>> my_dict_of_lists = accumulator({}, [], list.append)
or
 >>> my_dict_of_sets = accumulator({}, set(), set.add)
then: .accumulate(key, value) "does the right thing" in each case.
a bit cumbersome, because of having to specify the accumulation method, but 
avoids adding methods to, or sub-classing dict

Michael


--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Kay Schluehr]

George Sakkis wrote:
> > -1 form me.
> >
> > I'm not very glad with both of them ( not a naming issue ) because
i
> > think that the dict type should offer only methods that apply to
each
> > dict whatever it contains. count() specializes to dict values that
are
> > addable and appendlist to those that are extendable. Why not
> > subtractable, dividable or right-shiftable? Because of majority
> > approval? I'm mot a speed fetishist and destroying the clarity of a
> > very fundamental data structure for speedup rather arbitrary
> > accumulations seems to be a bad idea. I would move this stuff in a
> > subclass.
> >
> > Regards Kay

> +1 on this. The new suggested operations are meaningful for a subset
of all
> valid dicts, so they
> should not be part of the base dict API. If any version of this is
approved, > it will clearly be an
> application of the "practicality beats purity" zen rule, and the
> justification for applying it in
> this case instead of subclassing should better be pretty strong; so
far I'm
> not convinced though.
>
> George

It is bad OO design, George. I want to be a bit more become more
specific on this and provide an example:

Let be  a custom subclass of dict that stores only ints as
keys as well as values:

class intdict(dict):
def __setitem__(self, key, value):
assert isinstance(key, int) and isinstance(value, int)
dict.__setitem__(self,key,value)

or in Python3000 typeguard fashion:

class intdict(dict):
def __setitem__(self, key:int, value:int):
dict.__setitem__(self,key,value)

But  still overloads appendlist() i.e. a method that does not
work for any intdict is still part of it's public interface! This is
really bad design and should not be justified by a "practicality beats
purity" wisdom which should be cited with much care but not
carelesness.

Maybe also the subclassing idea I introduced falls for short for the
same reasons. Adding an accumulator to a dict should be implemented as
a *decorator* pattern in the GoF meaning of the word i.e. adding an
interface to some object at runtime that provides special facilities.

Usage:

>>> d = intdict(extend=MyAccumulator)
>>> hasattr(d,"tally")
True
>>> hasattr(d,"appendlist")
False

This could be generalized to other fundamental data structures as well.

Regards Kay

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[George Sakkis]

"John Machin" <[EMAIL PROTECTED]> wrote in message
news:[EMAIL PROTECTED]
>
> George Sakkis wrote:
> > +1 on this. The new suggested operations are meaningful for a subset
> of all valid dicts, so they
> > should not be part of the base dict API. If any version of this is
> approved, it will clearly be an
> > application of the "practicality beats purity" zen rule, and the
> justification for applying it in
> > this case instead of subclassing should better be pretty strong; so
> far I'm not convinced though.
>
> My background: I've been subclassing dict since this was possible, to
> provide not only a count/tally method but also a DIY intern method.
> Before that I just copied dictmodule.c (every release!) and diffed and
> hacked about till I had a mydict module.
>
> *any* version? Could we make you happy by having a subclass
> TotallySinfulDict provided as part of the core? You don't have to use
> it -- come to think of it, you don't have to use a sinful method in the
> base dict. You could even avert your gaze when reading the
> documentation.
>
> The justification for having it in the core: it's in C, not in Python,
> it gets implemented and maintained (a) ONCE (b) by folk like the timbot
> and Raymond instead of having (a) numerous versions lashed up (b) by
> you and me and a whole bunch of n00bz and b1ffz :-)

I believe it was pretty clear that I'm not against a new dict extension, in the 
core or in the
standard library; the proposed functionality is certainly useful and it would 
be most welcome. I
just don't find it appropriate for the existing base dict because it is not 
applicable to *every*
dictionary. As for the "you don't have to use feature X if you don't like it" 
argument, it's rarely
relevant in language design.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Beni Cherniavsky]

Alexander Schmolck wrote:
"Raymond Hettinger" <[EMAIL PROTECTED]> writes:
The rationale is to replace the awkward and slow existing idioms for dictionary
based accumulation:
   d[key] = d.get(key, 0) + qty
   d.setdefault(key, []).extend(values)
Indeed not too readable.  The try..except version is better but is too 
verbose.  There is a simple concept underneath of assuming a default value and 
we need "one obvious" way to write it.

In simplest form, those two statements would now be coded more readably as:
  d.count(key)
  d.appendlist(key, value)
Yuck.
-1 from me too on these two methods because they only add "duct tape" for the 
problem instead of solving it.  We need to improve upon `dict.setdefault()`, 
not specialize it.

The relatively recent "improvement" of the dict constructor signature
(``dict(foo=bar,...)``) obviously makes it impossible to just extend the
constructor to ``dict(default=...)`` (or anything else for that matter) which
would seem much less ad hoc. But why not use a classmethod (e.g.
``d=dict.withdefault(0)``) then?
You mean giving a dictionary a default value at creation time, right?
Such a dictionary could be used very easily, as in Perl::
foreach $word ( @words ) {
$d{$word}++; # default of 0 assumed, simple code!
}
.  You would like to write::
d = dict.withdefault(0)  # or something
for word in words:
d[word] += 1 # again, simple code!
I agree that it's a good idea but I'm not sure the default should be specified 
at creation time.  The problem with that is that if you pass such a dictionary 
into an unsuspecting function, it will not behave like a normal dictionary. 
Also, this will go awry if the default is a mutable object, like ``[]`` - you 
must create a new one at every access (or introduce a rule that the object is 
copied every time, which I dislike).  And there are cases where in different 
points in the code operating on the same dictionary you need different default 
values.

So perhaps specifying the default at every point of use by creating a proxy is 
cleaner::

d = {}
for word in words:
d.withdefault(0)[word] += 1
Of course, you can always create the proxy once and still pass it into an 
unsuspecting function when that is actually what you mean.

How should a dictionary with a default value behave (wheter inherently or a 
proxy)?

- ``d.__getattr__(key)`` never raises KeyError for missing keys - instead it
  returns the default value and stores the value as `d.setdefult()` does.
  This is needed for make code like::
  d.withdefault([])[key].append(foo)
  to work - there is no call of `d.__setattr__()`, so `d.__getattr__()` must
  have stored it.
  - `d.__setattr__()` and `d.__delattr__()` behave normally.
- Should ``key in d`` return True for all keys?  It is desiarable to have
  *some* way to know whether a key is really present.  But if it returns False
  for missing keys, code that checks ``key in d`` will behave differently from
  normally equivallent code that uses try..except.  If we use the proxy
  interface, we can always check on the original dictionary object, which
  removes the problem.
  - ``d.has_key(key)`` must do whatever we decide ``key in d`` does.
- What should ``d.get(key, [default])`` and ``d.setdefault(key, default)``
  do?  There is a conflict between the default of `d` and the explicitly given
  default.  I think consistency is better and these should pretend that `key`
  is always present.  But either way, there is a subtle problem here.
- Of course `iter(d)`, `d.items()` and the like should only see the keys
  that are really present (the alternative inventing an infinite amount of
  items out of the blue is clearly bogus).
If the idea that the default should be specified in every operation (creating 
a proxy) is accepted, there is a simpler and more fool-proof solution: the 
ptoxy will not support anything except `__getitem__()` and `__setitem__()` at 
all.  Use the original dictionary for everything else.  This prevents subtle 
ambiguities.

Or, for the first and most common case, just a bag type?
Too specialized IMHO.  You want a dictionary with any default anyway.  If you 
have that, what will be the benefit of a bag type?

--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[John Machin]

George Sakkis wrote:
> +1 on this. The new suggested operations are meaningful for a subset
of all valid dicts, so they
> should not be part of the base dict API. If any version of this is
approved, it will clearly be an
> application of the "practicality beats purity" zen rule, and the
justification for applying it in
> this case instead of subclassing should better be pretty strong; so
far I'm not convinced though.

My background: I've been subclassing dict since this was possible, to
provide not only a count/tally method but also a DIY intern method.
Before that I just copied dictmodule.c (every release!) and diffed and
hacked about till I had a mydict module.

*any* version? Could we make you happy by having a subclass
TotallySinfulDict provided as part of the core? You don't have to use
it -- come to think of it, you don't have to use a sinful method in the
base dict. You could even avert your gaze when reading the
documentation.

The justification for having it in the core: it's in C, not in Python,
it gets implemented and maintained (a) ONCE (b) by folk like the timbot
and Raymond instead of having (a) numerous versions lashed up (b) by
you and me and a whole bunch of n00bz and b1ffz :-)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Alexander Schmolck]

"Raymond Hettinger" <[EMAIL PROTECTED]> writes:

> The rationale is to replace the awkward and slow existing idioms for 
> dictionary
> based accumulation:
>
> d[key] = d.get(key, 0) + qty
> d.setdefault(key, []).extend(values)
>
> In simplest form, those two statements would now be coded more readably as:
>
>d.count(key)
>d.appendlist(key, value)

Yuck.

The relatively recent "improvement" of the dict constructor signature
(``dict(foo=bar,...)``) obviously makes it impossible to just extend the
constructor to ``dict(default=...)`` (or anything else for that matter) which
would seem much less ad hoc. But why not use a classmethod (e.g.
``d=dict.withdefault(0)``) then?

Or, for the first and most common case, just a bag type?


'as
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[George Sakkis]

Michael Spencer wrote:
> Raymond Hettinger wrote:
> > I would like to get everyone's thoughts on two new dictionary
methods:
> >
> > def count(self, value, qty=1):
> > try:
> > self[key] += qty
> > except KeyError:
> > self[key] = qty
> >
> > def appendlist(self, key, *values):
> > try:
> > self[key].extend(values)
> > except KeyError:
> > self[key] = list(values)
> >
>
> These undoubtedly address common cases, which are unsatisfactory when
spelled
> using setdefault.  However...
>
> Use of these methods implicitly specializes the dictionary.  The
methods are
> more-or-less mutually exclusive i.e., it would be at least strange to
use count
> and appendlist on the same dictionary.  Moreover, on many dictionary
instances,
> the methods would fail or produce meaningless results.
>
> This seems to be at odds with the other methods of built-in container
types
> which can be meaningfully applied, no matter what the types of the
contents.
> (There may be exceptions, but I can't think of any at the moment)
>
> Does anyone else think this is a problem?
>
> Michael


Yep, at least three more people in this thread:
- http://tinyurl.com/4bsdf
- http://tinyurl.com/3seqx
- http://tinyurl.com/6db27

George

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Michael Spencer]

Raymond Hettinger wrote:
I would like to get everyone's thoughts on two new dictionary methods:
def count(self, value, qty=1):
try:
self[key] += qty
except KeyError:
self[key] = qty
def appendlist(self, key, *values):
try:
self[key].extend(values)
except KeyError:
self[key] = list(values)
These undoubtedly address common cases, which are unsatisfactory when spelled 
using setdefault.  However...

Use of these methods implicitly specializes the dictionary.  The methods are 
more-or-less mutually exclusive i.e., it would be at least strange to use count 
and appendlist on the same dictionary.  Moreover, on many dictionary instances, 
the methods would fail or produce meaningless results.

This seems to be at odds with the other methods of built-in container types 
which can be meaningfully applied, no matter what the types of the contents. 
(There may be exceptions, but I can't think of any at the moment)

Does anyone else think this is a problem?
Michael

--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[George Sakkis]

> -1 form me.
>
> I'm not very glad with both of them ( not a naming issue ) because i
> think that the dict type should offer only methods that apply to each
> dict whatever it contains. count() specializes to dict values that are
> addable and appendlist to those that are extendable. Why not
> subtractable, dividable or right-shiftable? Because of majority
> approval? I'm mot a speed fetishist and destroying the clarity of a
> very fundamental data structure for speedup rather arbitrary
> accumulations seems to be a bad idea. I would move this stuff in a
> subclass.
>
> Regards Kay

+1 on this. The new suggested operations are meaningful for a subset of all 
valid dicts, so they
should not be part of the base dict API. If any version of this is approved, it 
will clearly be an
application of the "practicality beats purity" zen rule, and the justification 
for applying it in
this case instead of subclassing should better be pretty strong; so far I'm not 
convinced though.

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Ron]

On 19 Mar 2005 11:33:20 -0800, "Kay Schluehr" <[EMAIL PROTECTED]>
wrote:

>Raymond Hettinger wrote:
>> I would like to get everyone's thoughts on two new dictionary
>methods:
>>
>> def count(self, value, qty=1):
>> try:
>> self[key] += qty
>> except KeyError:
>> self[key] = qty
>>
>> def appendlist(self, key, *values):
>> try:
>> self[key].extend(values)
>> except KeyError:
>> self[key] = list(values)
>
>-1 form me.
>
>I'm not very glad with both of them ( not a naming issue ) because i
>think that the dict type should offer only methods that apply to each
>dict whatever it contains. count() specializes to dict values that are
>addable and appendlist to those that are extendable. Why not
>subtractable, dividable or right-shiftable? Because of majority
>approval? I'm mot a speed fetishist and destroying the clarity of a
>very fundamental data structure for speedup rather arbitrary
>accumulations seems to be a bad idea. I would move this stuff in a
>subclass.
>
>Regards Kay

-1 for me too.

I agree with this.  The other dictionary functions are all data
nuetral.  Adding special methods to handle data should be in a
durrived class and not a built in.  imho.

# Roll your own specialized dictionaries.

class NumDict( dict):
def count(self, key, qty=1):
try:
self[key] += qty
except KeyError:
self[key] = qty
def appendlist(self, key, *values):
try:
self[key].extend(values)
except KeyError:
self[key] = list(values)

mydict = NumDict()

n = 0
for k in list('abcdefg'):
n += 1
mydict[k] = n

print mydict
# {'a': 1, 'c': 3, 'b': 2, 'e': 5, 'd': 4, 'g': 7, 'f': 6}

mydict.count('c')
mydict['e'] = ['a']
mydict.appendlist('e', 'bcdef')
print mydict
# {'a': 1, 'c': 4, 'b': 2, 'e': ['a', 'bcdef'], 'd': 4, 'g': 7, 'f':
6}


This isn't that hard to do.  I don't think the extra methods should be
added to the base class.

Ron

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[David Eppstein]

In article <[EMAIL PROTECTED]>,
 "Raymond Hettinger" <[EMAIL PROTECTED]> wrote:

> Also, in all of my code base, I've not run across a single opportunity to use
> something like unionset(). 

In my code, this would be roughly tied with appendlist for frequency of 
use.

-- 
David Eppstein
Computer Science Dept., Univ. of California, Irvine
http://www.ics.uci.edu/~eppstein/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[David Eppstein]

In article <[EMAIL PROTECTED]>,
 "Raymond Hettinger" <[EMAIL PROTECTED]> wrote:

> The rationale is to replace the awkward and slow existing idioms for 
> dictionary
> based accumulation:
> 
> d[key] = d.get(key, 0) + qty
> d.setdefault(key, []).extend(values)
> 
> In simplest form, those two statements would now be coded more readably as:
> 
>d.count(key)
>d.appendlist(key, value)
> 
> In their multi-value forms, they would now be coded as:
> 
>   d.count(key, qty)
>   d.appendlist(key, *values)
> 
> The error messages returned by the new methods are the same as those returned 
> by
> the existing idioms.
> 
> The get() method would continue to exist because it is useful for 
> applications
> other than accumulation.
> 
> The setdefault() method would continue to exist but would likely not make it
> into Py3.0.

The other dictionary based accumulation I've used but don't see here is 
with sets as dictionary values, i.e. 
dictionary.setdefault(key,set()).add(value).

I suppose it would be possible to appendlist then make a set from the 
list, but if you were to take that minimalist philosophy to an extreme 
you wouldn't need count either, you could just appendlist then use len.

-- 
David Eppstein
Computer Science Dept., Univ. of California, Irvine
http://www.ics.uci.edu/~eppstein/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Raymond Hettinger]

[Bengt Richter]
> IMO Raymond's Zen concerns
> are the ones to think about first, and then efficiency, which was one of the
motivators
> in the first place ;-)

Well said.

I find the disassembly (presented in the first post) to be telling.  The
compiler has done a great job and there is no fluff -- all of those steps have
been specified by the programmer and he/she must at some level be aware of every
one of them (pre-instantiation, multiple method lookups and calls, multiple
dictionary accesses, etc).  That is too bad because the intent could have been
stated atomically: d.appendlist(k, v).  Instead, the current idiom turns you
away from what you want done and focuses the attention on how it is done:
d.setdefault(k, []).append(v).  That is too many steps for what should be an
atomic operation (in the eyes of the programmer and code readers).

Likewise, d.tally(k) is as atomic as this expression can get.  Any other steps,
added verbiage, new types, extra arguments, or whatnot are an unnecessary waste
of brain cells.


Raymond Hettinger


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[John Machin]

Reinhold Birkenfeld wrote:
> John Machin wrote:
> Are you kidding? If you know what "set" and "default" means, you will
be
> able to guess what "setdefault" means. Same goes for updateBy.
>

No I'm not kidding -- people from some cultures have no difficulty at
all in mentally splitting up "words" like "setdefault" or the German
equivalent of "Danubesteamnavigationcompany'sdirector'swife"; others
from other cultures where agglutinisation is not quite so rife will
have extreme difficulty.

And "Updateby" sounds like a village somewhere in the Danelaw :-)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Paul Rubin]

"Raymond Hettinger" <[EMAIL PROTECTED]> writes:
> I find the disassembly (presented in the first post) to be telling.
> The compiler has done a great job and there is no fluff -- all of
> those steps have been specified by the programmer and he/she must at
> some level be aware of every one of them (pre-instantiation,
> multiple method lookups and calls, multiple dictionary accesses, etc).

If the compiler can do some type inference, it can optimize out those
multiple calls pretty straightforwardly.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Reinhold Birkenfeld]

John Machin wrote:
> Raymond Hettinger wrote:
>> I would like to get everyone's thoughts on two new dictionary
> methods:
> 
> +1 for each.
> 
>> PROBLEMS BEING SOLVED
>> -
>>
>> The readability issues with the existing constructs are:
>>
>> * They are awkward to teach, create, read, and review.
>> * Their wording tends to hide the real meaning (accumulation).
>> * The meaning of setdefault() 's method name is not self-evident.
> 
> +1 to EACH of the above 3 points.
> 
> A question directed to the folk who had to look up "tally" in the
> dictionary: Which dictionary includes "setdefault", "updateBy", etc?

Are you kidding? If you know what "set" and "default" means, you will be
able to guess what "setdefault" means. Same goes for updateBy.

Of course, I had to look up setdefault in the Python docs the first time
I ran across it, but in the case of tally, I would have to look up both
in the Python docs and in my dictionary.

Reinhold
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[John Machin]

Raymond Hettinger wrote:
> I would like to get everyone's thoughts on two new dictionary
methods:

+1 for each.

> PROBLEMS BEING SOLVED
> -
>
> The readability issues with the existing constructs are:
>
> * They are awkward to teach, create, read, and review.
> * Their wording tends to hide the real meaning (accumulation).
> * The meaning of setdefault() 's method name is not self-evident.

+1 to EACH of the above 3 points.

A question directed to the folk who had to look up "tally" in the
dictionary: Which dictionary includes "setdefault", "updateBy", etc?

>
> The performance issues with the existing constructs are:

[MANY]

> the performance improvement matches the readability
> improvement.

Agreed.

>
>
> ISSUES
> --
>
> The proposed names could possibly be improved (perhaps tally() is
more active
> and clear than count()).

+3 for tally !!!

appendtolist is better than appendlist

>
> The appendlist() method is not as versatile as setdefault() which can
be used
> with other object types (perhaps for creating dictionaries of
dictionaries).
> However, most uses I've seen are with lists.

My use cases for tally:
(1) Yes the text-book "word" frequency gig applied in production
data-matching etc applications
(2) quick stats like from SQL "group by" e.g.
customer.tally(state)
customer_value.tally(state, dollar_value) # real or *DECIMAL*

Use cases for appendlist: many; in general, how else do you implement a
one-to-many relationship in memory??

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Bengt Richter]

On Sat, 19 Mar 2005 07:13:15 -0500, Kent Johnson <[EMAIL PROTECTED]> wrote:

>Bengt Richter wrote:
>> On Sat, 19 Mar 2005 01:24:57 GMT, "Raymond Hettinger" <[EMAIL PROTECTED]> 
>> wrote:
>> 
>>>I would like to get everyone's thoughts on two new dictionary methods:
>>>
>>>   def count(self, value, qty=1):
>>>   try:
>>>   self[key] += qty
>>>   except KeyError:
>>>   self[key] = qty
>>>
>>>   def appendlist(self, key, *values):
>>>   try:
>>>   self[key].extend(values)
>>>   except KeyError:
>>>   self[key] = list(values)
>>>
>> How about an efficient duck-typing value-incrementer to replace both? E.g. 
>> functionally like:
>> 
>>  >>> class xdict(dict):
>>  ... def valadd(self, key, incr=1):
>>  ... try: self[key] = self[key] + type(self[key])(incr)
>>  ... except KeyError: self[key] = incr
>
>A big problem with this is that there are reasonable use cases for both
>   d.count(key, )
>and
>   d.appendlist(key, )
>
>Word counting is an obvious use for the first. Consolidating a list of key, 
>value pairs where the 
>values are ints requires the second.
>
>Combining count() and appendlist() into one function eliminates the second 
>possibility.
I don't see a "big problem" ;-)

d.addval doesn't eliminate the functionalities if you want them. You just have 
to spell them
d.addval(key, )
and
d.addval(key, [])
respectively.

My example interactive stuff in a previous post shows both of these using 
xd['x'] and xd.['y']:
"""
 >>> xd = xdict()
 >>> xd
 {}

Default integer 1 arg creates initial int value
 >>> xd.valadd('x')
 >>> xd
 {'x': 1}

Explicit list arg create initial list value
 >>> xd.valadd('y', range(3))
 >>> xd
 {'y': [0, 1, 2], 'x': 1}

Explicit int increment adds to existing int
 >>> xd.valadd('x', 100)
 >>> xd['x']
 101

Explicit list arg extends existing list with contents of increment list
which you can of course optionally use with a length-1 list to achieve the 
.append effect
 >>> xd.valadd('y', range(3,6))
 >>> xd['y']
 [0, 1, 2, 3, 4, 5]
"""

Granted, d.appendlist will result in more efficient code, since the temp list 
arg
[] does not have to be created and the type of the existing dict 
value
does not have to be tested as generally. OTOH, you can create and extend tuple
or even custom object values using valadd, and extend with alternate sequences 
that
get converted to the existing type if its contructor knows how to accept 
alternatives.

So I think you are not prevented from doing anything. IMO Raymond's Zen concerns
are the ones to think about first, and then efficiency, which was one of the 
motivators
in the first place ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Paul Rubin]

"El Pitonero" <[EMAIL PROTECTED]> writes:
> What about no name at all for the scalar case:
> 
> a['hello'] += 1
> a['bye'] -= 2

I like this despite the minor surprise that it works even when
a['hello'] is uninitialized.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Kay Schluehr]

Raymond Hettinger wrote:
> I would like to get everyone's thoughts on two new dictionary
methods:
>
> def count(self, value, qty=1):
> try:
> self[key] += qty
> except KeyError:
> self[key] = qty
>
> def appendlist(self, key, *values):
> try:
> self[key].extend(values)
> except KeyError:
> self[key] = list(values)

-1 form me.

I'm not very glad with both of them ( not a naming issue ) because i
think that the dict type should offer only methods that apply to each
dict whatever it contains. count() specializes to dict values that are
addable and appendlist to those that are extendable. Why not
subtractable, dividable or right-shiftable? Because of majority
approval? I'm mot a speed fetishist and destroying the clarity of a
very fundamental data structure for speedup rather arbitrary
accumulations seems to be a bad idea. I would move this stuff in a
subclass.

Regards Kay

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Dan Sommers]

On Sat, 19 Mar 2005 15:17:59 GMT,
"Raymond Hettinger" <[EMAIL PROTECTED]> wrote:

> [Dan Sommers]

>> Curious that in this lengthy discussion, a method name of
>> "accumulate" never came up.  I'm not sure how to separate the two
>> cases (accumulating scalars vs. accumulating a list), though.

> Separating the two cases is essential.  Also, the wording should
> contain strong cues that remind you of addition and of building a
> list.

Agreed, with a slight hedge towards accumulation or tabulation rather
than addition.  I don't think "summation" gets us anywhere, either.

Are the use cases for qty != 1 for weighted averages (that's the only
one I can think of off the top of my head)?  Is something like this:

def accumulate( self, key, *values ):
if values == ( ):
values = 1
try:
self[ key ] += values
except KeyError:
if type( key ) == int:
self[ key ] = 1
else
self[ key ] = *values

possible?  It's more "klunky" than I thought it would be before I
started typing it out.

Then we'd have these two use cases:

histogram = { }
for word in text.split( ):
histogram.accumulate( word )

and

org_chart = { }
for employee in employees:
org_chart.accumulate( employee.manager, employee.name )

Regards,
Dan

-- 
Dan Sommers

Îâ à Îâ à c = 1
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[El Pitonero]

George Sakkis wrote:
> "Aahz" <[EMAIL PROTECTED]> wrote:
> > In article <[EMAIL PROTECTED]>,
> > Raymond Hettinger <[EMAIL PROTECTED]> wrote:
> > >
> > >The proposed names could possibly be improved (perhaps tally() is
more active
> > >and clear than count()).
> >
> > +1 tally()
>
> -1 for count(): Implies an accessor, not a mutator.
> -1 for tally(): Unfriendly to non-native english speakers.
> +0.5 for add, increment. If incrementing a negative is unacceptable,
how about
> update/updateby/updateBy ?
> +1 for accumulate. I don't think that separating the two cases --
adding to a scalar or appending to
> a list -- is that essential; a self-respecting program should make
this obvious by the name of the
> parameter anyway ("dictionary.accumulate('hello', words)" vs
"a.accumulate('hello', b)").

What about no name at all for the scalar case:

a['hello'] += 1
a['bye'] -= 2

and append() (or augmented assignment) for the list case:

a['hello'].append(word)
a['bye'] += [word]

?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[George Sakkis]

"Aahz" <[EMAIL PROTECTED]> wrote:
> In article <[EMAIL PROTECTED]>,
> Raymond Hettinger <[EMAIL PROTECTED]> wrote:
> >
> >The proposed names could possibly be improved (perhaps tally() is more active
> >and clear than count()).
>
> +1 tally()

-1 for count(): Implies an accessor, not a mutator.
-1 for tally(): Unfriendly to non-native english speakers.
+0.5 for add, increment. If incrementing a negative is unacceptable, how about
update/updateby/updateBy ?
+1 for accumulate. I don't think that separating the two cases -- adding to a 
scalar or appending to
a list -- is that essential; a self-respecting program should make this obvious 
by the name of the
parameter anyway ("dictionary.accumulate('hello', words)" vs 
"a.accumulate('hello', b)").

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[El Pitonero]

Raymond Hettinger wrote:
>
> As written out above, the += syntax works fine but does not work with
append().
> ...
> BTW, there is no need to make the same post three times.

The append() syntax works, if you use the other definition of safedict
(*). There are more than one way of defining safedict, see the subtle
differences between the two versions of safedict, and you'll be glad
more than one version has been posted. At any rate, what has been
presented is a general idea, nitpicking details is kind of out of
place. Programmers know how to modify a general receipe to suit their
actual needs, right?

(*) In some cases, people do not want to create a dictionary entry when
an inquiry is done on a missing item. In some case, they do. A general
receipe cannot cater to the needs of everybody.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Do Re Mi chel La Si Do]

Hi


if key not in d:
d[key] = {subkey:value}
else:
d[key][subkey] = value


and

d[(key,subkey)] = value

?


Michel Claveau


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[George Sakkis]

"Raymond Hettinger" <[EMAIL PROTECTED]> wrote:

> [Jeff Epler]
> > Maybe something for sets like 'appendlist' ('unionset'?)
>
> While this could work and potentially be useful, I think it is better to keep
> the proposal focused on the two common use cases.  Adding a third would reduce
> the chance of acceptance.
>
> Also, in all of my code base, I've not run across a single opportunity to use
> something like unionset().  This is surprising because I'm the set() author 
> and
> frequently use set based algorithms.Your example was a good one and I can
> also image a graph represented as a dictionary of sets.  Still, I don't mind
> writing out the plain Python for this one if it only comes up once in a blue
> moon.

Good example. I actually have a directed graph and multigraph module that uses 
dictionary of sets
internally. It turns out I've used setdefault 8 times in this module alone !

George


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Brian van den Broek]

Kent Johnson said unto the world upon 2005-03-19 07:19:
Brian van den Broek wrote:
Raymond Hettinger said unto the world upon 2005-03-18 20:24:
I would like to get everyone's thoughts on two new dictionary methods:
def appendlist(self, key, *values):
try:
self[key].extend(values)
except KeyError:
self[key] = list(values)
For appendlist, I would have expected
def appendlist(self, key, sequence):
try:
self[key].extend(sequence)
except KeyError:
self[key] = list(sequence)

The original proposal reads better at the point of call when values is a 
single item. In my experience this will be the typical usage:
  d.appendlist(key, 'some value')

as opposed to your proposal which has to be written
  d.appendlist(key, ['some value'])
The original allows values to be a sequence using
  d.appendlist(key, *value_list)
Kent
Right. I did try the alternatives out and get the issue you point to.
But:
1) In my own code, cases where I'd use the proposed appendlist method 
are typically cases where I'd want to add multiple items that have 
already been collected in a sequence. But, since I've little coding 
under my belt, I concede that considerations drawn from my experience 
are not terribly probative. :-)

2) Much more important, IMHO, is that the method name `appendlist' 
really does suggest it's a list that will be appended. Hence my stated 
expectation. While it would make the method name longer, given the 
original interface Raymond posted, I would find appendtolist more 
transparent.

out-of-my-depth-ly y'rs,
Brian vdB
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Raymond Hettinger]

[El Pitonero]
> Is it even necessary to use a method name?
>
> import copy
> class safedict(dict):
> def __init__(self, default=None):
> self.default = default
> def __getitem__(self, key):
> try:
> return dict.__getitem__(self, key)
> except KeyError:
> return copy.copy(self.default)
>
>
> x = safedict(0)
> x[3] += 1
> y = safedict([])
> y[5] += range(3)
> print x, y
> print x[123], y[234]



safedict() and variants have been previously proposed with the name defaultdict
or some such.

For the most part, adding methods is much less disruptive than introducing a new
type.

As written out above, the += syntax works fine but does not work with append().

As written, the copy.copy() approach is dog slow but can be optimized for lists
and ints while retaining its type flexibility.

BTW, there is no need to make the same post three times.


Raymond




-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Aahz]

In article <[EMAIL PROTECTED]>,
Raymond Hettinger <[EMAIL PROTECTED]> wrote:
>
>How about countkey() or tabulate()?

Those rank roughly equal to tally() for me, with a slight edge to these
two for clarity and a slight edge to tally() for conciseness.
-- 
Aahz ([EMAIL PROTECTED])   <*> http://www.pythoncraft.com/

"The joy of coding Python should be in seeing short, concise, readable
classes that express a lot of action in a small amount of clear code -- 
not in reams of trivial code that bores the reader to death."  --GvR
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[El Pitonero]

Raymond Hettinger wrote:
> Separating the two cases is essential.  Also, the wording should
contain strong
> cues that remind you of addition and of building a list.
>
> For the first, how about addup():
>
> d = {}
> for word in text.split():
>  d.addup(word)

import copy
class safedict(dict):
def __init__(self, default=None):
self.default = default
def __getitem__(self, key):
if not self.has_key(key):
self[key] = copy.copy(self.default)
return dict.__getitem__(self, key)

text = 'a b c b a'
words = text.split()
counts = safedict(0)
positions = safedict([])
for i, word  in enumerate(words):
counts[word] += 1
positions[word].append(i)

print counts, positions

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Aahz]

In article <[EMAIL PROTECTED]>,
Raymond Hettinger <[EMAIL PROTECTED]> wrote:
>
>The proposed names could possibly be improved (perhaps tally() is more active
>and clear than count()).

+1 tally()
-- 
Aahz ([EMAIL PROTECTED])   <*> http://www.pythoncraft.com/

"The joy of coding Python should be in seeing short, concise, readable
classes that express a lot of action in a small amount of clear code -- 
not in reams of trivial code that bores the reader to death."  --GvR
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Paul McGuire]

-1 on set increment.
I think this makes your intent much clearer:

.d={}
.for word in text.split():
.d.tally(word)
.if word.lower() in ["a","an","the"]:
.d.tally(word,-1)

or perhaps simplest:

.d={}
.for word in text.split():
.if word.lower() not in ["a","an","the"]:
.d.tally(word)

Personally, I'm +1 for tally(), and possibly tallyList() and tallySet()
to complete the thought for the cumulative container cases.  I think
there is something to be gained if these methods get named in some
similar manner.

For those dead set against tally() and its ilk, how about accum(),
accumList() and accumSet()?

-- Paul

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Raymond Hettinger]

[Ivan Van Laningham]

> What about adding another method, "setincrement()"?
 . . .

> Not that there's any real utility in that.

That was a short lived suggestion ;-)

Also, it would entail storing an extra value in the dictionary header.  That
alone would be a killer.


Raymond


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[El Pitonero]

Dan Sommers wrote:
> On Sat, 19 Mar 2005 01:24:57 GMT,
> "Raymond Hettinger" <[EMAIL PROTECTED]> wrote:
>
> > The proposed names could possibly be improved (perhaps tally() is
more
> > active and clear than count()).
>
> Curious that in this lengthy discussion, a method name of
"accumulate"
> never came up.  I'm not sure how to separate the two cases
(accumulating
> scalars vs. accumulating a list), though.

Is it even necessary to use a method name?

import copy
class safedict(dict):
def __init__(self, default=None):
self.default = default
def __getitem__(self, key):
try:
return dict.__getitem__(self, key)
except KeyError:
return copy.copy(self.default)


x = safedict(0)
x[3] += 1
y = safedict([]) 
y[5] += range(3) 
print x, y 
print x[123], y[234]

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Ivan Van Laningham]

Hi All--

Raymond Hettinger wrote:
> 
> Separating the two cases is essential.  Also, the wording should contain 
> strong
> cues that remind you of addition and of building a list.
> 
> For the first, how about addup():
> 
> d = {}
> for word in text.split():
>  d.addup(word)
> 

I still prefer tally(), despite perceived political connotations. 
They're only connotations, after all, and tally() comprises both
positive and negative incrementing, whereas add() and addup() will tease
users into thinking they are only for incrementing.

What about adding another method, "setincrement()"?

d={}
d.setincrement(-1)
for word in text.split():
d.tally(word,1)
if word.lower() in ["a","an","the"]:
d.tally(word)

Not that there's any real utility in that.

Metta,
Ivan
--
Ivan Van Laningham
God N Locomotive Works
http://www.pauahtun.org/
http://www.andi-holmes.com/
Army Signal Corps:  Cu Chi, Class of '70
Author:  Teach Yourself Python in 24 Hours
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Denis S. Otkidach]

On 18 Mar 2005 21:03:52 -0800 Michele Simionato wrote:

MS> +1 for inc instead of count.
MS> appendlist seems a bit too specific (I do not use dictionaries of
MS> lists that often).

inc is too specific too.

MS> The problem with setdefault is the name, not the functionality.

The problem with functionality: d.setdefault(k, v) can't be used as
lvalue.  If it could, we wouldn't need count/inc/add/tally method.

MS> get_or_set would be a better name: we could use it as an alias for
MS> setdefault and then remove setdefault in Python 3000.

What about d.get(k, setdefault=v) alternative?  Not sure whether it's
good idea to overload get() method, just an idea.

-- 
Denis S. Otkidach
http://www.python.ru/  [ru]
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Raymond Hettinger]

[Dan Sommers]
> Curious that in this lengthy discussion, a method name of "accumulate"
> never came up.  I'm not sure how to separate the two cases (accumulating
> scalars vs. accumulating a list), though.

Separating the two cases is essential.  Also, the wording should contain strong
cues that remind you of addition and of building a list.

For the first, how about addup():

d = {}
for word in text.split():
 d.addup(word)


Raymond


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Raymond Hettinger]

[Jeff Epler]
> Maybe something for sets like 'appendlist' ('unionset'?)

While this could work and potentially be useful, I think it is better to keep
the proposal focused on the two common use cases.  Adding a third would reduce
the chance of acceptance.

Also, in all of my code base, I've not run across a single opportunity to use
something like unionset().  This is surprising because I'm the set() author and
frequently use set based algorithms.Your example was a good one and I can
also image a graph represented as a dictionary of sets.  Still, I don't mind
writing out the plain Python for this one if it only comes up once in a blue
moon.


Raymond


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Peter Hansen]

Reinhold Birkenfeld wrote:
Peter Hansen wrote:
+1 for add and, given the above, I'm unsure there's
a viable alternative (unless this is restricted to
positive values, or perhaps even to "+1" specifically).
What about `addto()'? add() just has the connotation of adding something
to the dict and not to an item in it.
Hmm... better than add anyway.  I take back my ill-considered
+1 above, and apply instead a +0 to "count".  I don't actually
like any of the alternatives at this point... needs more thought
(for my part, anyway).
To be honest, the only time I've ever seen this particular
idiom is in tutorial code or examples of how you produce
a histogram of word usage in a text document.  Never in real
code (not that it doesn't happen, just that I've never
stumbled across it).  The "appending to a list" idiom, on
the other hand, I've seen and used quite often.
I'm just going to stay out of the "add/inc/count/addto"
debate and consider the other half of the thread now. :-)
-Peter
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Reinhold Birkenfeld]

Peter Hansen wrote:
> Michele Simionato wrote:
>> +1 for inc instead of count.
> 
> -1 for inc, increment, or anything that carries a
> connotation of *increasing* the value, so long as
> the proposal allows for negative numbers to be
> involved.  "Incrementing by -1" is a pretty silly
> picture.
> 
> +1 for add and, given the above, I'm unsure there's
> a viable alternative (unless this is restricted to
> positive values, or perhaps even to "+1" specifically).

What about `addto()'? add() just has the connotation of adding something
to the dict and not to an item in it.

Reinhold
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Peter Hansen]

Michele Simionato wrote:
+1 for inc instead of count.
-1 for inc, increment, or anything that carries a
connotation of *increasing* the value, so long as
the proposal allows for negative numbers to be
involved.  "Incrementing by -1" is a pretty silly
picture.
+1 for add and, given the above, I'm unsure there's
a viable alternative (unless this is restricted to
positive values, or perhaps even to "+1" specifically).
appendlist seems a bit too specific (I do not use dictionaries of lists
that often).
As Raymond does, I use this much more than the other.
The problem with setdefault is the name, not the functionality.
get_or_set would be a better name: we could use it as an alias for
setdefault and then remove setdefault in Python 3000.
Agreed...
-Peter
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Ivan Van Laningham]

Hi All--

Raymond Hettinger wrote:
> 
> [Michele Simionato]
> > +1 for inc instead of count.
> 
> Any takers for tally()?
> 

Sure.  Given the reasons for avoiding add(), tally()'s a much better
choice than count().

What about d.tally(key,0) then?  Deleting the key as was suggested by
Michael Spencer seems non-intuitive to me.

> > Just my 2 Eurocents,
> 
> I raise you by a ruble and a pound ;-)
> 

-ly y'rs,
Ivan
--
Ivan Van Laningham
God N Locomotive Works
http://www.andi-holmes.com/
http://www.foretec.com/python/workshops/1998-11/proceedings.html
Army Signal Corps:  Cu Chi, Class of '70
Author:  Teach Yourself Python in 24 Hours
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Jeff Epler]

> [Jeff Epler]
> > Maybe something for sets like 'appendlist' ('unionset'?)
> 
On Sat, Mar 19, 2005 at 04:18:43AM +, Raymond Hettinger wrote:
> I do not follow.  Can you provide a pure python equivalent?

Here's what I had in mind:

$ python /tmp/unionset.py
Set(['set', 'self', 'since', 's', 'sys', 'source', 'S', 'Set', 'sets', 
'starting'])

#
try:
set
except:
from sets import Set as set

def unionset(self, key, *values):
try:
self[key].update(values)
except KeyError:
self[key] = set(values)

if __name__ == '__main__':
import sys, re
index = {}

# We need a source of words.  This file will do.
corpus = open(sys.argv[0]).read()
words = re.findall('\w+', corpus)

# Create an index of the words according to the first letter.
# repeated words are listed once since the values are sets
for word in words:
unionset(index, word[0].lower(), word)

# Display the words starting with 'S'
print index['s']
#

Jeff


pgpecFFOsSmDH.pgp
Description: PGP signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Dan Sommers]

On Sat, 19 Mar 2005 01:24:57 GMT,
"Raymond Hettinger" <[EMAIL PROTECTED]> wrote:

> The proposed names could possibly be improved (perhaps tally() is more
> active and clear than count()).

Curious that in this lengthy discussion, a method name of "accumulate"
never came up.  I'm not sure how to separate the two cases (accumulating
scalars vs. accumulating a list), though.

Regards,
Dan

-- 
Dan Sommers

Îâ à Îâ à c = 1
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Pierre Barbier de Reuille]

Ivan Van Laningham a écrit :
Hi All--
Maybe I'm not getting it, but I'd think a better name for count would be
add.  As in
d.add(key)
d.add(key,-1)
d.add(key,399)
etc.
>
[...]
There is no existing add() method for dictionaries.  Given the name
change, I'd like to see it.
Metta,
Ivan
I don't think "add" is a good name ... even if it doesn't exist in 
dictionnarie, it exists in sets and, IMHO, this would add confusion ...

Pierre

--
Ivan Van Laningham
God N Locomotive Works
http://www.pauahtun.org/
http://www.andi-holmes.com/
Army Signal Corps:  Cu Chi, Class of '70
Author:  Teach Yourself Python in 24 Hours
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Kent Johnson]

Brian van den Broek wrote:
Raymond Hettinger said unto the world upon 2005-03-18 20:24:
I would like to get everyone's thoughts on two new dictionary methods:
def appendlist(self, key, *values):
try:
self[key].extend(values)
except KeyError:
self[key] = list(values)
For appendlist, I would have expected
def appendlist(self, key, sequence):
try:
self[key].extend(sequence)
except KeyError:
self[key] = list(sequence)
The original proposal reads better at the point of call when values is a single item. In my 
experience this will be the typical usage:
  d.appendlist(key, 'some value')

as opposed to your proposal which has to be written
  d.appendlist(key, ['some value'])
The original allows values to be a sequence using
  d.appendlist(key, *value_list)
Kent
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Carl Banks]

Raymond Hettinger wrote:
> I would like to get everyone's thoughts on two new dictionary
methods:
>
> def count(self, value, qty=1):
> try:
> self[key] += qty
> except KeyError:
> self[key] = qty
>
> def appendlist(self, key, *values):
> try:
> self[key].extend(values)
> except KeyError:
> self[key] = list(values)


Emphatic +1

I use both of these idioms all the time.  (Kind of surprised to see
people confused about the need for the latter; I do it regularly.)
This is just the kind of thing experience shows cropping up enough that
it makes sense to put it in the language.

About the names: Seeing that these have specific uses, and do something
that is hard to explain in one word, I would suggest that short names
like count might betray the complexity of the operations.  Therefore,
I'd suggest:

increment_value() (or add_to_value())
append_to_value()

Although they don't explicitly communicate that a value would be
created if it didn't exist, they do at least make it clear that it
happens to the value, which kind of implies that it would be created.

If we do have to use short names:

I don't like increment (or inc or incr) at all because it has the air
of a mutator method.  Maybe it's just my previous experience with Java
and C++, but to me, a.incr() looks like it's incrementing a, and
a.incr(b) looks like it might be adding b to a.  I don't like count
because it's too vague; it's pretty obvious what it does as an
iterator, but not as a method of dict.  I could live with tally,
though.  As for a short name for the other one, maybe fileas or
fileunder?


-- 
CARL BANKS

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Kent Johnson]

Bengt Richter wrote:
On Sat, 19 Mar 2005 01:24:57 GMT, "Raymond Hettinger" <[EMAIL PROTECTED]> wrote:
I would like to get everyone's thoughts on two new dictionary methods:
  def count(self, value, qty=1):
  try:
  self[key] += qty
  except KeyError:
  self[key] = qty
  def appendlist(self, key, *values):
  try:
  self[key].extend(values)
  except KeyError:
  self[key] = list(values)
How about an efficient duck-typing value-incrementer to replace both? E.g. 
functionally like:
 >>> class xdict(dict):
 ... def valadd(self, key, incr=1):
 ... try: self[key] = self[key] + type(self[key])(incr)
 ... except KeyError: self[key] = incr
A big problem with this is that there are reasonable use cases for both
  d.count(key, )
and
  d.appendlist(key, )
Word counting is an obvious use for the first. Consolidating a list of key, value pairs where the 
values are ints requires the second.

Combining count() and appendlist() into one function eliminates the second 
possibility.
Kent
--
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[El Pitonero]

On Sat, 19 Mar 2005 01:24:57 GMT, "Raymond Hettinger"
<[EMAIL PROTECTED]> wrote:
>I would like to get everyone's thoughts on two new dictionary methods:
>
>def count(self, value, qty=1):
>try:
>self[key] += qty
>except KeyError:
>self[key] = qty
>
>def appendlist(self, key, *values):
>try:
>self[key].extend(values)
>except KeyError:
>self[key] = list(values)

Bengt Richter wrote:
>  >>> class xdict(dict):
>  ... def valadd(self, key, incr=1):
>  ... try: self[key] = self[key] + type(self[key])(incr)
>  ... except KeyError: self[key] = incr

What about:

import copy
class safedict(dict):
def __init__(self, default=None):
self.default = default
def __getitem__(self, key):
try:
return dict.__getitem__(self, key)
except KeyError:
return copy.copy(self.default)

x = safedict(0)
x[3] += 1
y = safedict([])
y[5] += range(3)
print x, y
print x[123], y[234]

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Peter Otten]

Roose wrote:

> Not to belabor the point, but in the example on that page, max(L, key=len)
> could be written max(len(x) for x in L).
 
No, it can't:

Python 2.5a0 (#2, Mar  5 2005, 17:44:37)
[GCC 3.3.3 (SuSE Linux)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> max(["a", "bbb", "cc"], key=len)
'bbb'

Peter

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Roose]

> Py2.5 is already going to include any() and all() as builtins.  The
signature
> does not include a function, identity or otherwise.  Instead, the caller
can
> write a listcomp or genexp that evaluates to True or False:

Actually I was just looking at Python 2.5 docs since you mentioned this.

http://www.python.org/dev/doc/devel/whatsnew/node3.html

It says min() and max() will gain a key function parameter, and sort()
gained one in Python 2.4 (news to me).

And they do indeed default to the identity in all 3 cases, so this seems
very inconsistent.  If one of them has it, and sort gained the argument even
in Python 2.4 with generator expressions, then they all should have it.

> any(x >= 42 for x in data)

Not to belabor the point, but in the example on that page, max(L, key=len)
could be written max(len(x) for x in L).

Now I know why Guido said he didn't want a PEP for this... such a trivial
thing can produce a lot of opinions.  : )

Roose


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Raymond Hettinger]

[Michele Simionato]
> Dunno, to me "tally" reads "counts the numbers of votes for a candidate
> in an election".

That isn't a pleasant image ;-)



>The
> only right name would be "get_and_possibly_set" but it is a bit long to
> type.
>
>> Even if a wording is found that better describes the both the get and
>> set operation, it is still a distractor from the intent of the combined
>> statement, the intent of building up a list.  That is an intrinsic wording
>> limitation that cannot be solved by a better name for setdefault.
>> If any change is made at all, we ought to go the distance and provide a
>> better designed tool rather than just a name change.
>
> Well, I never figured out that the intent of setdefault was to build up
> a list ;)

Right!  What does have that intent is the full statement: d.setdefault(k,
[]).append(v).

My thought is that setdefault() is rarely used by itself.  Instead, it is
typically part of a longer sentence whose intent and meaning is to accumulate or
build-up.  That meaning is not well expressed by the current idiom.



Raymond Hettinger


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Pre-PEP: Dictionary accumulator methods

[Roose]

> Py2.5 is already going to include any() and all() as builtins.  The
signature
> does not include a function, identity or otherwise.  Instead, the caller
can
> write a listcomp or genexp that evaluates to True or False:
>
> any(x >= 42 for x in data)
>
> If you wanted an identify function, that simplifies to just:
>
> any(data)

Oh great, I just saw that.  I was referring to this, which didn't get much
discussion:

http://mail.python.org/pipermail/python-dev/2005-February/051556.html

but it looks like it went much further, to builtins!  I'm surprised.

But I wish it could be included in Python 2.4.x.  I really hope it won't
have any bugs in it.  :)  At my job we are probably going to upgrade to 2.4,
and that takes a long time, so it'll probably be a year or 18 months after
that happens (which itself might be months from now) that we would consider
upgrading again.  Oh well...



-- 
http://mail.python.org/mailman/listinfo/python-list