[issue10576] Add a progress callback to gcmodule

Changes by Kristján Valur Jónsson : -- resolution: -> fixed status: open -> closed ___ Python tracker ___ ___ Python-bugs-list mailin

[issue10576] Add a progress callback to gcmodule

Roundup Robot added the comment: New changeset 88f8ef5785d7 by Kristján Valur Jónsson in branch 'default': Issue #10576: Add a progress callback to gcmodule http://hg.python.org/cpython/rev/88f8ef5785d7 -- nosy: +python-dev ___ Python tracker

[issue10576] Add a progress callback to gcmodule

Kristján Valur Jónsson added the comment: A new patch, taking Antoine's review and comments into account. -- Added file: http://bugs.python.org/file25158/gccallback.patch ___ Python tracker ___

[issue10576] Add a progress callback to gcmodule

Antoine Pitrou added the comment: Uploaded another review. I also notice you didn't really address my point, since self.visit is still initialized too early. IMO it should be initialized after the first gc.collect() at the beginning of each test (not in setUp()). -- _

[issue10576] Add a progress callback to gcmodule

Kristján Valur Jónsson added the comment: Here is an updated patch, taking Jim's and Antoine's comments into account. Jim, I´d like to comment that I think the reason __del__ objects are uncollectable is more subtle than there being no defined order of calling the __del__ functions. More sig

[issue10576] Add a progress callback to gcmodule

Kristján Valur Jónsson added the comment: You are right, I was thinking more of pyobject attributes. I'll fix this then. -- ___ Python tracker ___ _

[issue10576] Add a progress callback to gcmodule

Antoine Pitrou added the comment: > - gc.callbacks is a simple module attribute. Anyone can set it to > anything else, e.g. gc.callbacks=None. We have to accomodate this > possibility or else introduce annoying api functions to edit the list. > I thought it best to do things similarly to sys.i

[issue10576] Add a progress callback to gcmodule

Kristján Valur Jónsson added the comment: Thank you Antoine. Your points: - Yes, I can robustify this. - I think I worried that the actual contents might be too complex to test for it. I'll see if I can't just simplify it as you suggest. - right, thanks. - gc.callbacks is a simple module attrib

[issue10576] Add a progress callback to gcmodule

Antoine Pitrou added the comment: Comments: - the tests look fragile. How can you know a garbage collection will only collect your own objects? So you should call gc.collect() first at the beginning of each test and then initialize the self.visit list. We don't want weird failures because of

[issue10576] Add a progress callback to gcmodule

Kristján Valur Jónsson added the comment: Thanks, Jim. Unless anyone objects, I'll commit this then. -- ___ Python tracker ___ ___ Py

[issue10576] Add a progress callback to gcmodule

Jim Jewett added the comment: gccallback4a.patch is a few changes to gccallback4.patch. Most are just spelling or grammar in comments, but I did modify the test case so that the Uncollectable loop had *multiple* objects with __del__ methods. -- nosy: +Jim.Jewett Added file: http://bu

[issue10576] Add a progress callback to gcmodule

Kristján Valur Jónsson added the comment: Hm, actually it wasn't Michael, but Martin. No matter! Here is a proposed patch, as promised without all the bells and whistles, in particular, there is no feature to cancel the garbage collection. Can we get this into 3.3? -- Added file: ht

[issue10576] Add a progress callback to gcmodule

Kristján Valur Jónsson added the comment: Hi! Michael Foord reminded me of this little gem. I'm getting this started again, hopefully for inclusion in 3.3 -- nosy: +michael.foord ___ Python tracker _

[issue10576] Add a progress callback to gcmodule

Lukas Lueg added the comment: Agreed, let's have the simple callback first. To solve 2) later on, we could have the callback proposed here be the 'execution'-callback. It neither has nor will have the capability to prevent garbage-collection. We can introduce another 'prepare'-callback later

[issue10576] Add a progress callback to gcmodule

Kristján Valur Jónsson added the comment: 1) what I mean is that if a callback rejects GC, the GC algorithm may find its condition for invoking GC in the first place to be still valid immediately afterwards, so doing a GC will be immediately retried. I have to check, but it could mean that m

[issue10576] Add a progress callback to gcmodule

Lukas Lueg added the comment: Collection may re-occur at any time, there is no promise to the callback code. However, the callback can disable the gc, preventing further collection. I don't think we need the other callbacks to be informed. As the callbacks are worked down in the order they re

[issue10576] Add a progress callback to gcmodule

Kristján Valur Jónsson added the comment: Well, the idea is good and it did cross my mind. Particularly it could be useful for performance sensitive applications. However it complicates things. 1) If a GC is rejected, when do we make the next attempt? 2) If a callback cancels GC, then what abo

[issue10576] Add a progress callback to gcmodule

Lukas Lueg added the comment: Why not make the start-callback be able to return a boolean value to the gcmodule that indicates if garbage collection should take place or not. For example, any value returned from the callback that evaluates to False (like null) will cause the module to evaluat

[issue10576] Add a progress callback to gcmodule

Kristján Valur Jónsson added the comment: Uh oh. I forgot about this and there now we have passed beta 2. Didn't anyone want to review the patch? -- ___ Python tracker ___

[issue10576] Add a progress callback to gcmodule

Changes by Antoine Pitrou : -- stage: -> patch review versions: +Python 3.3 -Python 3.2 ___ Python tracker ___ ___ Python-bugs-list m

[issue10576] Add a progress callback to gcmodule

Kristján Valur Jónsson added the comment: Here is a third patch. The callback now gets two argument, phase and info. I've added documentation and unittests. -- Added file: http://bugs.python.org/file19946/gccallback3.patch ___ Python tracker

[issue10576] Add a progress callback to gcmodule

Changes by Yury Selivanov : -- nosy: +Yury.Selivanov ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mai

[issue10576] Add a progress callback to gcmodule

Jim Jewett added the comment: Does anyone think that it is simpler to register two different callbacks than one? Moderately, yes. Functions that actually help with cleanup should normally be run only in one phase; it is just stats-gathering and logging functions that might run both times,

[issue10576] Add a progress callback to gcmodule

Kristján Valur Jónsson added the comment: 1) I'm not sure what you are asking. Does anyone think that it is simpler to register two different callbacks than one? IMHO it complicates the interface and creates so many oppertunities to do things incorrectly. 2)No, it is an oversight, let me ve

[issue10576] Add a progress callback to gcmodule

Robert Lehmann added the comment: A few issues I'd like to raise: (1) Multiple callback chains. Is there any code in your existing use case of GC callbacks where you don't check for the phase argument and follow different code paths depending on it? If not, having two callback chains should

[issue10576] Add a progress callback to gcmodule

Daniel Stutzbach added the comment: > How about a string and a dict? the string can be "start" and "stop" > and we can add interesting information to the dict as you suggest. I like where this is headed. How about putting the string in the dict, too? d['phase'] = 'start' -- ___

[issue10576] Add a progress callback to gcmodule

Antoine Pitrou added the comment: > You are right, Antoine. > How about a string and a dict? the string can be "start" and "stop" > and we can add interesting information to the dict as you suggest. Looks good to me. -- ___ Python tracker

[issue10576] Add a progress callback to gcmodule

Kristján Valur Jónsson added the comment: You are right, Antoine. How about a string and a dict? the string can be "start" and "stop" and we can add interesting information to the dict as you suggest. -- ___ Python tracker

[issue10576] Add a progress callback to gcmodule

Antoine Pitrou added the comment: > Let's start bikeshedding the calling signature. I like having a > single callback, since multiple callables are a nuisance to manage. IMO the callback should have a second argument as a dict containing various statistics that we can expand over time. The gen

[issue10576] Add a progress callback to gcmodule

Kristján Valur Jónsson added the comment: Hi, as I stated, the original patch was simply our original implementation. Here is a new patch. It is simpler: 1) it exposes a gc.callbacks list where users can register themselves, in the spirit of sys.meta_path 2) One can have multiple callbacks 3)

[issue10576] Add a progress callback to gcmodule

Jim Jewett added the comment: I like the idea, but I do quibble with the signature. As nearly as I can tell, you're assuming (a) Only one callback. I would prefer a sequence of callbacks, to make cooperation easier. (This does mean you would need a callback removal, instead of just setti

[issue10576] Add a progress callback to gcmodule

Antoine Pitrou added the comment: Well, as you point out, it would make more sense to have two separate callbacks. Also, PyErr_WriteUnraisable() is better than PyErr_Clear(). Finally, you accidentally recoded the file; it should be kept utf-8, not latin-whatever. -- nosy: +pitrou __

[issue10576] Add a progress callback to gcmodule

Daniel Stutzbach added the comment: These functions will be very useful for any long-running program. Thank you for the patch. Would you be willing to write tests and documentation? Would it make more sense for the callback to take a boolean instead of an integer as the first argument? ---

[issue10576] Add a progress callback to gcmodule

Changes by Andrew Svetlov : -- nosy: +asvetlov ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe: http://mail.pyth

[issue10576] Add a progress callback to gcmodule

New submission from Kristján Valur Jónsson : As discussed here: http://mail.python.org/pipermail/python-ideas/2010-November/008813.html: Adding the ability to register callbacks to be invoked before and after garbage collection runs. This can be used to gather run-time statistics such as timi