Re: Searching equivalent to C++ RAII or deterministic destructors
On Jul 3, 5:34 am, Jack Diederich wrote: > On Thu, Jul 2, 2009 at 2:36 PM, Carl Banks wrote: > > Warning: objects with a __del__ attribute prevent reference cycle > > detection, which can potentially lead to memory (and resource) leaks. > > So you must be careful to avoid creating reference loops with that > > object. > > WARNING-er: As Carl points out, adding a __del__ method actually makes > it /less/ likely that your object will be cleaned up. No I wasn't pointing that out, and I don't really agree with it. (You can't really say it is "more likely" or "less likely" without considering the skill and patience of the programmer, and there are many programmers who are up to the task.) __del__ can succeed with care, and if you are willing to give up a certain amount of flexibility and portability I don't recommend going that way. For one thing RAII isn't quite so wonderful in Python's dynamic environment as it is in C++'s static universe, and people seem to expect more from it than they get. Second, relying on implementation-dependent behavior is not a good thing and can set you up for problems later, something few people expect to happen to them but it does more often than they think. (Search comp.lang.python for a thread started by Warren DeLano for an example of someone who got bit hard by such a change he should have seen coming.) But still I leave that for them to risk for themselves. [snip] > If you care enough about a resource to write a __del__ method > you care enough to clean it up explicitly. 'with' blocks are very > nice for that. The OP already said with blocks won't suffice since the resources are long-lived objects that aren't limited to a single scope. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Searching equivalent to C++ RAII or deterministic destructors
In article , Tim Roberts wrote: >Dave Angel wrote: >> >>You're right of course. What I was trying to say was it deletes the >>reference to the object. Unlike obj = None, del obj removes the >>reference (attribute) entirely. Although I don't know what it should be >>called if it's a local variable. Perhaps it "unbinds" the name. > >Yes. As far as the object is concerned, "obj = None" and "del obj" are >exactly identical. In both cases, there is one less binding to the name. > >The difference between the two is only whether the name lives on in the >namespace. > >A local variable is (usually) just a name in the local() namespace. OTOH, Python's ``del`` applies to targets generally, not just names: >>> L = [1,2,3] >>> del L[1] >>> L [1, 3] -- Aahz (a...@pythoncraft.com) <*> http://www.pythoncraft.com/ "as long as we like the same operating system, things are cool." --piranha -- http://mail.python.org/mailman/listinfo/python-list
Re: Searching equivalent to C++ RAII or deterministic destructors
Dave Angel wrote: > >You're right of course. What I was trying to say was it deletes the >reference to the object. Unlike obj = None, del obj removes the >reference (attribute) entirely. Although I don't know what it should be >called if it's a local variable. Perhaps it "unbinds" the name. Yes. As far as the object is concerned, "obj = None" and "del obj" are exactly identical. In both cases, there is one less binding to the name. The difference between the two is only whether the name lives on in the namespace. A local variable is (usually) just a name in the local() namespace. -- Tim Roberts, t...@probo.com Providenza & Boekelheide, Inc. -- http://mail.python.org/mailman/listinfo/python-list
Re: Searching equivalent to C++ RAII or deterministic destructors
On Thu, Jul 2, 2009 at 2:36 PM, Carl Banks wrote: > On Jul 2, 3:12 am, Ulrich Eckhardt wrote: >> Bearophile wrote: >> > Ulrich Eckhardt: >> >> a way to automatically release the resource, something >> >> which I would do in the destructor in C++. >> >> > Is this helpful? >> >http://effbot.org/pyref/with.htm >> >> Yes, it aims in the same direction. However, I'm not sure this applies to my >> case. The point is that the resource handle is not just used locally in a >> restricted scope but it is allocated and stored. The 'with' is something >> that makes sense in the context of mutex locking, where you have a >> well-defined critical section. What I need is something similar to open(), >> which returs a file. When the last reference to that object goes out of >> scope, the underlying file object is closed. > > On CPython you can do it with a __del__ attribute. > > Warning: objects with a __del__ attribute prevent reference cycle > detection, which can potentially lead to memory (and resource) leaks. > So you must be careful to avoid creating reference loops with that > object. WARNING-er: As Carl points out, adding a __del__ method actually makes it /less/ likely that your object will be cleaned up. __del__ is only useful if you have a complicated tear-down procedure. Since you come from C++ RAII land (my old haunt) your objects probably only allocate one resource each, or worse: a bunch of objects that allocate one resource each. In that case __del__ will always hurt you. The C++ temptation is to match every __init__ with a __del__. A better rule of thumb is to only add a __del__ method after confirming with someone else that it would be useful. Better still is to ask them by postal courier. For best results that someone should be your grandmother. > Note that file objects have a close method; you can explicitly close > it at any time. Your object should follow that example, and define a > close (or release, or whatever) method. I'd recommend making an > effort to call it and to rely on __del__ as little as possible. This. If you care enough about a resource to write a __del__ method you care enough to clean it up explicitly. 'with' blocks are very nice for that. -jack -- http://mail.python.org/mailman/listinfo/python-list
Re: Searching equivalent to C++ RAII or deterministic destructors
Thanks to all that answered, in particular I wasn't aware of the existence of the __del__ function. For completeness' sake, I think I have found another way to not really solve but at least circumvent the problem: weak references. If I understand correctly, those would allow me to pass out handles to the resources and, if some code decides it is time, release the resources and render all the weak references invalid. At least I don't suffer resource leaks but rather get meaningful errors that way, which is enough for my case. cheers! Uli -- Sator Laser GmbH Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932 -- http://mail.python.org/mailman/listinfo/python-list
Re: Searching equivalent to C++ RAII or deterministic destructors
> You can go ahead and implement a __del__() method. It will often work in > CPython, but you get no guarantees, especially when you have reference > cycles and with other Python implementations that don't use refcounting. And for resources whose lifetime is greater than your process (e.g. a file), you can also use the atexit module to help ensure they are cleaned/reclaimed at shutdown. One way to do this is to maintain a weak dictionary (from the weakref module) to your objects and install a handler which iterates this. This is useful for not only dealing with reference cycles, but for objects which may exist at the time the interpreter exits. These may not be deleted. http://docs.python.org/reference/datamodel.html#object.__del__ -- http://mail.python.org/mailman/listinfo/python-list
Re: Searching equivalent to C++ RAII or deterministic destructors
Peter Otten schreef: > Ulrich Eckhardt wrote: > >> Bearophile wrote: >>> Ulrich Eckhardt: a way to automatically release the resource, something which I would do in the destructor in C++. >>> Is this helpful? >>> http://effbot.org/pyref/with.htm >> Yes, it aims in the same direction. However, I'm not sure this applies to >> my case. The point is that the resource handle is not just used locally in >> a restricted scope but it is allocated and stored. The 'with' is something >> that makes sense in the context of mutex locking, where you have a >> well-defined critical section. > > Isn't that exactly what RAII does? RAII also works if the resource handle is stored, for example, in a data member of an object. If that object is destroyed (because it goes out of scope, or because it is deleted), the resource is automatically destroyed too. The way RAII works is actually the one thing from C++ that I miss in Python. -- The saddest aspect of life right now is that science gathers knowledge faster than society gathers wisdom. -- Isaac Asimov Roel Schroeven -- http://mail.python.org/mailman/listinfo/python-list
Re: Searching equivalent to C++ RAII or deterministic destructors
On Jul 2, 3:12 am, Ulrich Eckhardt wrote: > Bearophile wrote: > > Ulrich Eckhardt: > >> a way to automatically release the resource, something > >> which I would do in the destructor in C++. > > > Is this helpful? > >http://effbot.org/pyref/with.htm > > Yes, it aims in the same direction. However, I'm not sure this applies to my > case. The point is that the resource handle is not just used locally in a > restricted scope but it is allocated and stored. The 'with' is something > that makes sense in the context of mutex locking, where you have a > well-defined critical section. What I need is something similar to open(), > which returs a file. When the last reference to that object goes out of > scope, the underlying file object is closed. On CPython you can do it with a __del__ attribute. Warning: objects with a __del__ attribute prevent reference cycle detection, which can potentially lead to memory (and resource) leaks. So you must be careful to avoid creating reference loops with that object. Note that file objects have a close method; you can explicitly close it at any time. Your object should follow that example, and define a close (or release, or whatever) method. I'd recommend making an effort to call it and to rely on __del__ as little as possible. Carl Banks -- http://mail.python.org/mailman/listinfo/python-list
Re: Re: Searching equivalent to C++ RAII or deterministic destructors
Christian Heimes wrote: Dave Angel wrote: Look also at 'del' a command in the language which explicitly deletes an object. No, you are either explaining it the wrong way or you have been fallen for a common misinterpretation of the del statement. The del statement only removes the object from the current scope. This means it decreases the reference count by one. It does *not* remove the object. Christian You're right of course. What I was trying to say was it deletes the reference to the object. Unlike obj = None, del obj removes the reference (attribute) entirely. Although I don't know what it should be called if it's a local variable. Perhaps it "unbinds" the name. DaveA -- http://mail.python.org/mailman/listinfo/python-list
Re: Searching equivalent to C++ RAII or deterministic destructors
Dave Angel wrote:
> But I'm guessing you want something that automatically deletes objects
> whenever the last reference disappears. That's an implementation
> detail, not a language guarantee. In particular CPython does what you
> want, by using reference counting. That's the only Python I've used, so
> it's only hearsay when I say that other implementations, (maybe Cython
> or Jython) do not all work the same way.
Here are some examples from Kubuntu 9.04's zoo of python implementations:
$ cat del.py
import sys
print sys.version
class A(object):
def __init__(self, x):
self.x = x
def __del__(self):
print "releasing A(%r)" % self.x
def f():
a = A("local in function")
f()
a = A("global (one ref)")
c = A("global (cycle)")
c.a = c
del c
b = A("global (no refs)")
del b
print "about to quit"
$ python del.py
2.6.2 (release26-maint, Apr 19 2009, 01:58:18)
[GCC 4.3.3]
releasing A('local in function')
releasing A('global (no refs)')
about to quit
releasing A('global (one ref)')
$ jython del.py
2.2.1
about to quit
$ ipy del.py
2.4.0 (IronPython 1.1.1 (1.1.1) on .NET 2.0.50727.42)
about to quit
releasing A('global (no refs)')
releasing A('global (cycle)')
releasing A('local in function')
Unhandled Exception: System.ArgumentException: I/O operation on closed file
at IronPython.Runtime.PythonFile.ThrowIfClosed () [0x0]
at IronPython.Runtime.PythonFile.Write (System.String s) [0x0]
$
IronPython sometimes segfaulted.
Peter
--
http://mail.python.org/mailman/listinfo/python-list
Re: Searching equivalent to C++ RAII or deterministic destructors
Dave Angel wrote: > Look also at 'del' a command in the language which explicitly deletes an > object. No, you are either explaining it the wrong way or you have been fallen for a common misinterpretation of the del statement. The del statement only removes the object from the current scope. This means it decreases the reference count by one. It does *not* remove the object. Christian -- http://mail.python.org/mailman/listinfo/python-list
Re: Searching equivalent to C++ RAII or deterministic destructors
Ulrich Eckhardt wrote: Hi! I'm currently converting my bioware to handle Python code and I have stumbled across a problem... Simple scenario: I have a handle to a resource. This handle allows me to manipulate the resource in various ways and it also represents ownership. Now, when I put this into a class, instances to that class can be shared, using Python's reference counting. What I'm missing is a way to automatically release the resource, something which I would do in the destructor in C++. Any ideas how to solve this? Uli As someone else pointed out, 'with' is the first line of defense. It makes the stuff you could already do with try/except/finally much easier to get right. Look also at 'del' a command in the language which explicitly deletes an object. But I'm guessing you want something that automatically deletes objects whenever the last reference disappears. That's an implementation detail, not a language guarantee. In particular CPython does what you want, by using reference counting. That's the only Python I've used, so it's only hearsay when I say that other implementations, (maybe Cython or Jython) do not all work the same way. In CPython, any object whose *last* reference goes away, will get immediately deleted. So if you avoid circular references, it should do just what you want. Orphaned circular references are caught by the garbage collector, which runs periodically, but is non-deterministic. Two more caveats. Exceptions tend to hang onto stuff in their near vicinity, and I can't tell you the algorithm for what happens. But an explicit del in the except clause can probably handle that. Finally, closures can hang onto stuff. So if you have nested functions, or lambda functions, it's possible they're going to keep things longer than you realize. -- http://mail.python.org/mailman/listinfo/python-list
Re: Searching equivalent to C++ RAII or deterministic destructors
Ulrich Eckhardt wrote: > Bearophile wrote: >> Ulrich Eckhardt: >>> a way to automatically release the resource, something >>> which I would do in the destructor in C++. >> >> Is this helpful? >> http://effbot.org/pyref/with.htm > > Yes, it aims in the same direction. However, I'm not sure this applies to > my case. The point is that the resource handle is not just used locally in > a restricted scope but it is allocated and stored. The 'with' is something > that makes sense in the context of mutex locking, where you have a > well-defined critical section. Isn't that exactly what RAII does? > What I need is something similar to open(), > which returs a file. When the last reference to that object goes out of > scope, the underlying file object is closed. You can go ahead and implement a __del__() method. It will often work in CPython, but you get no guarantees, especially when you have reference cycles and with other Python implementations that don't use refcounting. Peter -- http://mail.python.org/mailman/listinfo/python-list
Re: Searching equivalent to C++ RAII or deterministic destructors
Bearophile wrote: > Ulrich Eckhardt: >> a way to automatically release the resource, something >> which I would do in the destructor in C++. > > Is this helpful? > http://effbot.org/pyref/with.htm Yes, it aims in the same direction. However, I'm not sure this applies to my case. The point is that the resource handle is not just used locally in a restricted scope but it is allocated and stored. The 'with' is something that makes sense in the context of mutex locking, where you have a well-defined critical section. What I need is something similar to open(), which returs a file. When the last reference to that object goes out of scope, the underlying file object is closed. Uli -- Sator Laser GmbH Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932 -- http://mail.python.org/mailman/listinfo/python-list
Re: Searching equivalent to C++ RAII or deterministic destructors
Ulrich Eckhardt: > a way to automatically release the resource, something > which I would do in the destructor in C++. Is this helpful? http://effbot.org/pyref/with.htm Bye, bearophile -- http://mail.python.org/mailman/listinfo/python-list
