Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Armin Rigo
Hi Brett, hi Floris, On Sat, Nov 19, 2005 at 04:12:28PM -0800, Brett Cannon wrote: Just for everyone's FYI while we are talking about profilers, Floris Bruynooghe (who I am cc'ing on this so he can contribute to the conversation), for Google's Summer of Code, wrote a replacement for

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Barry Warsaw
On Mon, 2005-11-21 at 12:14 +0100, Armin Rigo wrote: Still, people generally agree that profile.py, while taking a longer time overall, gives more meaningful results than hotshot. Now Brett's student, Floris, extended hotshot to allow custom timers. This is essential, because it enables

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Armin Rigo
Hi Barry, On Mon, Nov 21, 2005 at 11:40:37AM -0500, Barry Warsaw wrote: Hi Armin. Actually it was SF #900092 that I was referring to. Ah, we're talking about different things then. The patch in SF #900092 is not related to hotshot, it's just ceval.c not producing enough events to allow a

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Brett Cannon
On 11/21/05, Jeremy Hylton [EMAIL PROTECTED] wrote: Here's another attempt to disentagle some issues: - Should lsprof be added to the standard distribution? - Should hotshot be removed from the standard distribution? These two aren't at all related, unless you believe that two is the maximum

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Armin Rigo
Hi Floris, On Mon, Nov 21, 2005 at 04:41:04PM +, Floris Bruynooghe wrote: Now Brett's student, Floris, extended hotshot to allow custom timers. This is essential, because it enables testing. The timing parts of hotshot were not tested previously. Don't be too enthousiastic here.

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Armin Rigo
Hi Floris, On Mon, Nov 21, 2005 at 04:45:03PM +, Floris Bruynooghe wrote: Afaik I did test recursive calls etc. It seems to show up in any test case I try, e.g. import hprofile def wait(m): if m 0: wait(m-1) def f(n): wait(n) if n 1:

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Martin v. Löwis
Armin Rigo wrote: I see no incremental way of fixing some of the downsides of hotshot, like its huge log file size and loading time. I haven't looked into the details myself, but it appears that some google-summer-of-code contributor has found some way of fixing it. I doubt people often find

[Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Jim Jewett
Jeremy Hylton jeremy at alum.mit.edu Should lsprof be added to the standard distribution? Should hotshot be removed from the standard distribution? These two aren't at all related, unless you believe that two is the maximum number of profiles allowed per Python distribution. One is a

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Martin v. Löwis
Brett Cannon wrote: But this worry, in my mind, is alleviated since I believe both Michael and Armin are willing to maintain the code. With them both willing to make sure it stays working (which is a pretty damn good commitment since we have two core developers willing to keep this going and

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Nick Coghlan
Jim Jewett wrote: Jeremy Hylton jeremy at alum.mit.edu Should lsprof be added to the standard distribution? Should hotshot be removed from the standard distribution? These two aren't at all related, unless you believe that two is the maximum number of profiles allowed per Python

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread skip
Brett My question is whether anyone is willing to maintain it in the Brett stdlib? My answer is: I'm not sure it matters at this point. There are so many profiling possibilities, it doesn't seem like we yet know which options are the best. There is some tacit crowning of best of breed

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Armin Rigo
Hi Martin, On Mon, Nov 21, 2005 at 10:29:55PM +0100, Martin v. L?wis wrote: I see no incremental way of fixing some of the downsides of hotshot, like its huge log file size and loading time. I haven't looked into the details myself, but it appears that some google-summer-of-code

Re: [Python-Dev] s/hotshot/lsprof

2005-11-21 Thread Brett Cannon
On 11/21/05, Armin Rigo [EMAIL PROTECTED] wrote: Hi Martin, On Mon, Nov 21, 2005 at 10:29:55PM +0100, Martin v. L?wis wrote: I see no incremental way of fixing some of the downsides of hotshot, like its huge log file size and loading time. I haven't looked into the details myself, but

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread Michael Hudson
Martin v. Löwis [EMAIL PROTECTED] writes: Armin Rigo wrote: If anyone feels like this is a bad idea, please speak up. As stated, it certainly is a bad idea. This is a bit extreme... To make it a good idea, there should also be some commitment to maintain this library for a number of

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread Martin v. Löwis
Michael Hudson wrote: As stated, it certainly is a bad idea. This is a bit extreme... Yes, my apologies :-( To make it a good idea, there should also be some commitment to maintain this library for a number of years. So who would be maintaining it, and what are their plans for doing so?

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread Fredrik Lundh
Martin v. Löwis wrote: The same could be said about hotshot, which was originally contributed by Fred Drake, and hacked by Tim Peters, yourself, and others. Yet, now people want to remove it again. I'm really concerned that the same fate will happen to any new profiling library: anybody but

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread skip
Fredrik (on the other hand, I'm not sure we need a profiler as part of Fredrik the standard library either, but that's me...) Painful though hotshot can be at times, I occasionally find it extremely useful to zoom in on trouble spots. I haven't used profile in awhile and haven't tried

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread Tim Peters
[Armin Rigo] ... ... 'hotshot', new from 2.2, is quite faster (reportedly, only 30% added overhead). The log file is then loaded and turned into an instance of the same 'pstats.Stats'. This loading takes ages. The reason is that the log file only records events, and loading is done by

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread Tim Peters
[Martin v. Löwis] I'm really concerned that the same fate will happen to any new profiling library: anybody but the original author will hate it, write his own, and then suggest to replace the existing one. [Fredrik Lundh] is this some intrinsic property of profilers? if the existing tool

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread Steve Holden
Tim Peters wrote: [Martin v. Löwis] I'm really concerned that the same fate will happen to any new profiling library: anybody but the original author will hate it, write his own, and then suggest to replace the existing one. [Fredrik Lundh] is this some intrinsic property of profilers?

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread A.M. Kuchling
On Sun, Nov 20, 2005 at 11:33:42PM +0100, Fredrik Lundh wrote: do we really need CADT-based development in the standard library? I didn't recognize the acronym, but Google told me CADT = Cascade of Attention-Deficit Teenagers; see http://www.jwz.org/doc/cadt.html for a rant. --amk

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread Neil Schemenauer
Tim Peters [EMAIL PROTECTED] wrote: We should note that hotshot didn't intend to reduce total time overhead. What it's aiming at here is to be less disruptive (than profile.py) to the code being profiled _while_ that code is running. A statistical profiler (e.g.

Re: [Python-Dev] s/hotshot/lsprof

2005-11-20 Thread Martin v. Löwis
Tim Peters wrote: Center for Alcohol Drug Treatment Besides Jamie Zawinski's definition, Google also told me it stands for Computer Aided Drafting Technology where to draft turns out to have two different meanings :-) Regards, Martin

[Python-Dev] s/hotshot/lsprof

2005-11-19 Thread Armin Rigo
Hi! The current Python profilers situation is a mess. 'profile.Profile' is the ages-old pure Python profiler. At the end of a run, it builds a dict that is inspected by 'pstats.Stats'. It has some recent support for profiling C calls, which however make it crash in some cases [1]. And of

Re: [Python-Dev] s/hotshot/lsprof

2005-11-19 Thread Aahz
On Sat, Nov 19, 2005, Armin Rigo wrote: If anyone feels like this is a bad idea, please speak up. This sounds like a good idea, and your presentation already looks almost like a PEP. How about going ahead and making it a formal PEP, which will make it easier to push through the dev process? --

Re: [Python-Dev] s/hotshot/lsprof

2005-11-19 Thread Martin v. Löwis
Armin Rigo wrote: If anyone feels like this is a bad idea, please speak up. As stated, it certainly is a bad idea. To make it a good idea, there should also be some commitment to maintain this library for a number of years. So who would be maintaining it, and what are their plans for doing so?