Re: Counting number of objects
Thank you everyone for your very helpful comments and suggestions. I have interacted in other newsgroups, but this is the most helpful of them all. As per the comments, I have now decided to go with the weakref mechanism - as per Andreas suggestion, functionally it looks correct that the person should not know the total number of people. So, in a higher level class, have a weakref list which contains a reference to each person. Total count will be len(list) at any time. Now, I couldnt find a weakref list - so I am using WeakKeyDictionary with the value as None - since len(dict) also should give me the data any time. I have another question here. In the documentation, it is mentioned that - Note: Caution: Because a WeakKeyDictionary is built on top of a Python dictionary, it must not change size when iterating over it. This can be difficult to ensure for a WeakKeyDictionary because actions performed by the program during iteration may cause items in the dictionary to vanish by magic (as a side effect of garbage collection). Now, the only two operations that I am doing are - __init__: d = weakref.WeakKeyDictionary() method y: x = aa() d[x] = None method z: total = len(d) I believe that all the commands which I perform on WeakKeyDictionary here - (adding a new element) (len(d)) - are atomic - or atleast nothing that can cause any worry as per the Note given above. Can anyone let me know whether my assumption is correct or not? Reason: My code has many many number of threads which interact with each other in myraid ways - so I do want to decrease the number of locks as much as possible. Especially I do not want to block other threads just for getting the count. -- http://mail.python.org/mailman/listinfo/python-list
Re: Counting number of objects
Kottiyath wrote: So, in a higher level class, have a weakref list which contains a reference to each person. Total count will be len(list) at any time. Now, I couldnt find a weakref list - so I am using WeakKeyDictionary with the value as None - since len(dict) also should give me the data I typically use a WeakValueDictionary, with the key id(obj). ... Now, the only two operations that I am doing are - __init__: d = weakref.WeakKeyDictionary() method y: x = aa() d[x] = None method z: total = len(d) At least at one point, the WeakValueDictionary was more stable. I don't remember the details, but I remember my work-around. If you do have troubles, try using almost the same code you now use, but substitute for the obvious lines above: d = weakref.WeakValueDictionary() and d[id(x)] = x --Scott David Daniels scott.dani...@acm.org -- http://mail.python.org/mailman/listinfo/python-list
Re: Counting number of objects
On Mon, 26 Jan 2009 02:37:37 + Mark Wooding m...@distorted.org.uk wrote: This looks OK, although I'd suggest using cls.counter += 1 instead of a.counter += 1 in the __new__() method. Just seems clearer to me, esp. when you think about subclassing. I'm not sure about clarity, but that would be semantically different. The code as written counts all instances of a and its subclasses. Your suggestion would count instances of subclasses independently. I don't know which behaviour the OP would prefer, but I don't think choosing between them is a matter of clarity. Oh shoot. I should have known that. I hope I will from now on, at least. :/ /W -- My real email address is constructed by swapping the domain with the recipient (local part). -- http://mail.python.org/mailman/listinfo/python-list
Re: Counting number of objects
Mark Wooding wrote: Andreas Waldenburger geekm...@usenot.de writes: On Sun, 25 Jan 2009 09:23:35 -0800 (PST) Kottiyath n.kottiy...@gmail.com wrote: class a(object): counter = 0 def __new__(cls, *args, **kwargs): a.counter += 1 return object.__new__(cls, *args, **kwargs) Hmm. Exceptions raised during object creation make this rather precarious. In your code, if object.__new__ raises an exception, the counter will end up too high (the __del__ method won't get called in this case). If object.__new__ raises an exception you have bigger problems to worry about than an object count being wrong. One might try to rewrite it: def __new__(cls, *args, **kw): thing = object.__new__(cls, *args, **kw) a.counter += 1 return thing Now this won't work in subclasses if they also have a __new__ method: again, if the subclass's __new__ raises an exception then __del__ won't be called and the counter will be too high. To make this technique work, I think you need to do the counter increment in __init__ rather than __new__, and to set an attribute so that __del__ knows whether to do the decrement. (If a subclass's __init__ raises an exception before yours gets called, you don't want to do the decrement because that'll leave the counter too low.) Yes, rather better to do it that way and decouple it from __new__(). Of course super() might help here, though not everyone approves of it. This looks OK, although I'd suggest using cls.counter += 1 instead of a.counter += 1 in the __new__() method. Just seems clearer to me, esp. when you think about subclassing. I'm not sure about clarity, but that would be semantically different. The code as written counts all instances of a and its subclasses. Your suggestion would count instances of subclasses independently. I don't know which behaviour the OP would prefer, but I don't think choosing between them is a matter of clarity. Correct, but pointing out the differences has highlighted that is a real design decision to be made in the event that subclassing is used. Another way to go would be to use the weakref module and create a weakref-set (or list) as the counter. That way you would only need to add the objects in the __new__() method and not worry about removing them. I will admit that this is overengineering the problem a bit, but might be a good exercise. This is a better approach, because it avoids the problems with exceptions during object construction that I described above. Another question - unrelated to the major topic: How much time does it take to be proficient in Python? Don't concern yourself with that question at all, is my advice. Indeed. Besides, it varies an awful lot. [...] - or am I just dumb? You're writing programs and you're communicating with like-minded people about your problems (in a socially appropriate way). Not what dumb people do, in my book. Absolutely! Besides, you're asking sensible questions about subtle parts of the language -- I wouldn't say that __new__ was beginner territory, for example. So, no, you certainly don't seem dumb to me. Hear, hear! regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Counting number of objects
Kottiyath wrote: Hi, I am creating a class called people - subclasses men, women, children etc. I want to count the number of people at any time. So, I created code like the following: class a(object): counter = 0 def __new__(cls, *args, **kwargs): a.counter += 1 return object.__new__(cls, *args, **kwargs) def __del__(self): a.counter -= 1 class aa(a): pass Now, the code works Ok. I have the following questions: 1. Is this code Ok? Is there any straightforward mechanism other than this to get the number of objects? 2. I read in Python Documentation that inside __del__ we should the minimum of interaction with external parameters. So, I am a little worried in subclassing __del__ to check the counter. Is whatever I have done Ok? Yes. Just be aware that if instances become involved in cyclic data structures (or in implementations other than CPython, where reference counting isn't used) __del__ might not be called until garbage collection kicks in, so you may want a more explicit way to stop an instance from being in the count. Another question - unrelated to the major topic: How much time does it take to be proficient in Python? I have been working exclusively in Python for close to 3 months now, and even now I get inferiority complex when I read the answers sent by many of you. I have been programming for close to 7 years now (earlier in a language similar to COBOL). Does it take quite a bit of time to be proficient - as many of you guys - or am I just dumb? By your code above you seem to be doing OK. Python is like an iceberg - only an eighth of what goes on is above the surface. That eighth will suffice for many people's total programming needs. I've been using Python ten years, and I am still learning. Just go at your own pace, and carry on asking for help when you need it. regards Steve -- Steve Holden+1 571 484 6266 +1 800 494 3119 Holden Web LLC http://www.holdenweb.com/ -- http://mail.python.org/mailman/listinfo/python-list
Re: Counting number of objects
On Sun, 25 Jan 2009 09:23:35 -0800 (PST) Kottiyath n.kottiy...@gmail.com wrote: Hi, I am creating a class called people - subclasses men, women, children etc. I want to count the number of people at any time. So, I created code like the following: class a(object): counter = 0 def __new__(cls, *args, **kwargs): a.counter += 1 return object.__new__(cls, *args, **kwargs) def __del__(self): a.counter -= 1 class aa(a): pass This looks OK, although I'd suggest using cls.counter += 1 instead of a.counter += 1 in the __new__() method. Just seems clearer to me, esp. when you think about subclassing. This would create an asymmetry with __del__() then. Oh well. So maybe use self.__class__.counter -= 1 there, even if it is a bit ugly-ish. Another way to go would be to use the weakref module and create a weakref-set (or list) as the counter. That way you would only need to add the objects in the __new__() method and not worry about removing them. I will admit that this is overengineering the problem a bit, but might be a good exercise. A third option could be to remove the counting functions from the class altogether. From an OO-Design point of view this would seem appropriate, because neither any individual nor mankind as a whole would know the exact amount of people in the world of hand. An entity that would actually count all the people in the world, however, would know and it makes sense to implement it separately (as a subclass of list with birth() and death() methods, for instance). I'm just saying this to give you something to think about, I'm not saying that it is necessarily better or worse for your example. Now, the code works Ok. I have the following questions: 1. Is this code Ok? Is there any straightforward mechanism other than this to get the number of objects? I would say that you found the most obvious implementation there. Depending on your definition of straightforward you could do it as I outlined in my last example above, using explicit calls for births and deaths. This would remove the behind the scenes magic a bit, which may be a plus. 2. I read in Python Documentation that inside __del__ we should the minimum of interaction with external parameters. So, I am a little worried in subclassing __del__ to check the counter. Is whatever I have done Ok? Again, seems good to me, unless you do some other trickery that may lead to __del__() not being called directly. In that case, do look at the weakref module. Another question - unrelated to the major topic: How much time does it take to be proficient in Python? Don't concern yourself with that question at all, is my advice. Most people can learn to write programs in under a week, and many still don't write *good* programs 20 years later. You'll get better over time, as long as you keep doing it. Thinking about that you're not good enough will just consume mental resources that would be better invested in your programs. I have been working exclusively in Python for close to 3 months now, and even now I get inferiority complex when I read the answers sent by many of you. Hello?! :) Three months is *nothing* compared to the time those many of you folks have invested. Don't worry. Does it matter if you don't understand some stuff people write? As long as you pick out the stuff you do understand you're still gaining stuff from this group. I have been programming for close to 7 years now (earlier in a language similar to COBOL). Well, you have quite a background then. Why all the worries? Does it take quite a bit of time to be proficient - as many of you guys YES! Of course it does. - or am I just dumb? You're writing programs and you're communicating with like-minded people about your problems (in a socially appropriate way). Not what dumb people do, in my book. cheers, /W -- My real email address is constructed by swapping the domain with the recipient (local part). -- http://mail.python.org/mailman/listinfo/python-list
Re: Counting number of objects
En Sun, 25 Jan 2009 16:06:47 -0200, Andreas Waldenburger geekm...@usenot.de escribió: On Sun, 25 Jan 2009 09:23:35 -0800 (PST) Kottiyath n.kottiy...@gmail.com wrote: I am creating a class called people - subclasses men, women, children etc. I want to count the number of people at any time. So, I created code like the following: class a(object): counter = 0 def __new__(cls, *args, **kwargs): a.counter += 1 return object.__new__(cls, *args, **kwargs) def __del__(self): a.counter -= 1 class aa(a): pass This looks OK, although I'd suggest using cls.counter += 1 instead of a.counter += 1 in the __new__() method. Just seems clearer to me, esp. when you think about subclassing. This would create an asymmetry with __del__() then. Oh well. So maybe use self.__class__.counter -= 1 there, even if it is a bit ugly-ish. Using self.__class__ is safer, from a technical point of view. When __del__ is executed at interpreter shutdown, a may not be available -- in general, __del__ methods should not rely on any globals (one can inject names into the local namespace using default arguments). See http://bugs.python.org/issue1717900 -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Counting number of objects
En Sun, 25 Jan 2009 16:06:47 -0200, Andreas Waldenburger geekm...@usenot.de escribió: On Sun, 25 Jan 2009 09:23:35 -0800 (PST) Kottiyath n.kottiy...@gmail.com wrote: I am creating a class called people - subclasses men, women, children etc. I want to count the number of people at any time. So, I created code like the following: class a(object): counter = 0 def __new__(cls, *args, **kwargs): a.counter += 1 return object.__new__(cls, *args, **kwargs) def __del__(self): a.counter -= 1 class aa(a): pass This looks OK, although I'd suggest using cls.counter += 1 instead of a.counter += 1 in the __new__() method. Just seems clearer to me, esp. when you think about subclassing. This would create an asymmetry with __del__() then. Oh well. So maybe use self.__class__.counter -= 1 there, even if it is a bit ugly-ish. Using self.__class__ is safer, from a technical point of view. When __del__ is executed at interpreter shutdown, a may not be available -- in general, __del__ methods should not rely on any globals (one can inject names into the local namespace using default arguments). See http://bugs.python.org/issue1717900 -- Gabriel Genellina -- http://mail.python.org/mailman/listinfo/python-list
Re: Counting number of objects
Andreas Waldenburger geekm...@usenot.de writes: On Sun, 25 Jan 2009 09:23:35 -0800 (PST) Kottiyath n.kottiy...@gmail.com wrote: class a(object): counter = 0 def __new__(cls, *args, **kwargs): a.counter += 1 return object.__new__(cls, *args, **kwargs) Hmm. Exceptions raised during object creation make this rather precarious. In your code, if object.__new__ raises an exception, the counter will end up too high (the __del__ method won't get called in this case). One might try to rewrite it: def __new__(cls, *args, **kw): thing = object.__new__(cls, *args, **kw) a.counter += 1 return thing Now this won't work in subclasses if they also have a __new__ method: again, if the subclass's __new__ raises an exception then __del__ won't be called and the counter will be too high. To make this technique work, I think you need to do the counter increment in __init__ rather than __new__, and to set an attribute so that __del__ knows whether to do the decrement. (If a subclass's __init__ raises an exception before yours gets called, you don't want to do the decrement because that'll leave the counter too low.) This looks OK, although I'd suggest using cls.counter += 1 instead of a.counter += 1 in the __new__() method. Just seems clearer to me, esp. when you think about subclassing. I'm not sure about clarity, but that would be semantically different. The code as written counts all instances of a and its subclasses. Your suggestion would count instances of subclasses independently. I don't know which behaviour the OP would prefer, but I don't think choosing between them is a matter of clarity. Another way to go would be to use the weakref module and create a weakref-set (or list) as the counter. That way you would only need to add the objects in the __new__() method and not worry about removing them. I will admit that this is overengineering the problem a bit, but might be a good exercise. This is a better approach, because it avoids the problems with exceptions during object construction that I described above. Another question - unrelated to the major topic: How much time does it take to be proficient in Python? Don't concern yourself with that question at all, is my advice. Indeed. Besides, it varies an awful lot. I can't tell you from my personal experience, because I tend to learn programming languages by osmosis. I sit and read language manuals and newsgroups out of curiosity, because I find programming languages intrinsically interesting. One day, maybe a few years later, I start writing programs in some language. I'm usually up to half-decent language-lawyer standards within a few days of this point -- but I've had a /lot/ of practice at this game. My approach is almost certainly extremely atypical. But one of the best things about Python is the library. It's very big and wide-ranging, but not overcomplex, and I'm still discovering cool stuff, even though it's been there for years. And the advice to keep the Library Reference under your pillow is excellent. Expect to be flipping through it constantly. - or am I just dumb? You're writing programs and you're communicating with like-minded people about your problems (in a socially appropriate way). Not what dumb people do, in my book. Absolutely! Besides, you're asking sensible questions about subtle parts of the language -- I wouldn't say that __new__ was beginner territory, for example. So, no, you certainly don't seem dumb to me. -- [mdw] -- http://mail.python.org/mailman/listinfo/python-list
