Re: [Python-ideas] Efficient debug logging

2017-02-18 Thread Stephen J. Turnbull
Barry Warsaw writes: > On Feb 16, 2017, at 03:20 PM, M.-A. Lemburg wrote: > > >I know some people will disagree, but IMO using "assert" is the wrong > >approach in such situations - it's meant for development and testing > >only, not as short-cut to avoid having to write a proper error > >ha

Re: [Python-ideas] Efficient debug logging

2017-02-17 Thread Barry
> On 16 Feb 2017, at 21:03, Abe Dillon wrote: > > I personally don't see why you can't use floats for log levels, but I'm > interested to know how people are using logs such that they need dozens of > levels. That, however; is tangential to the discussion about conditional > execution of an e

Re: [Python-ideas] Efficient debug logging

2017-02-16 Thread Abe Dillon
> > You are right that Python logging infrastructure can deal with logs at a > finer grain, but the code required to do it is more verbose than logical > switches The default behavior of filters seems pretty useful to me (filtering by hierarchy), but, then again, so do the default log levels. I h

Re: [Python-ideas] Efficient debug logging

2017-02-16 Thread Barry Warsaw
On Feb 16, 2017, at 03:20 PM, M.-A. Lemburg wrote: >I know some people will disagree, but IMO using "assert" is the wrong >approach in such situations - it's meant for development and testing >only, not as short-cut to avoid having to write a proper error >handler :-) I use assertions for "things

Re: [Python-ideas] Efficient debug logging

2017-02-16 Thread Kyle Lahnakoski
On 2017-02-15 16:06, Abe Dillon wrote: > On 15.02.2017, 20:39 Kyle Lahnakoski wrote: > > Log "levels" never made sense to me; how can a single dimension be > useful substitute for a number of binary switches? With log > "levels", you either don't have enough logging, or you drown in >

Re: [Python-ideas] Efficient debug logging

2017-02-16 Thread Giampaolo Rodola'
On Thu, Feb 16, 2017 at 1:55 PM, M.-A. Lemburg wrote: > Some comments: > > 1. you don't need a preprocessor for this: simply put your > logging code into an "if __debug__:" block: > > https://docs.python.org/3.6/reference/simple_stmts.html? > the-assert-statement#grammar-token-assert_stmt >

Re: [Python-ideas] Efficient debug logging

2017-02-16 Thread M.-A. Lemburg
On 16.02.2017 14:23, Victor Stinner wrote: > 2017-02-16 13:55 GMT+01:00 M.-A. Lemburg : >> 1. you don't need a preprocessor for this: simply put your >> logging code into an "if __debug__:" block: > > The problem with -O is that it also disables assertions, whereas you > may want to keep them

Re: [Python-ideas] Efficient debug logging

2017-02-16 Thread Victor Stinner
2017-02-16 13:55 GMT+01:00 M.-A. Lemburg : > 1. you don't need a preprocessor for this: simply put your > logging code into an "if __debug__:" block: The problem with -O is that it also disables assertions, whereas you may want to keep them at runtime on production for good reasons. Victor _

Re: [Python-ideas] Efficient debug logging

2017-02-16 Thread M.-A. Lemburg
Some comments: 1. you don't need a preprocessor for this: simply put your logging code into an "if __debug__:" block: https://docs.python.org/3.6/reference/simple_stmts.html?the-assert-statement#grammar-token-assert_stmt and then run your production code with "python -O" (the trick h

Re: [Python-ideas] Efficient debug logging

2017-02-16 Thread Victor Stinner
Yeah, I had a similar issue in a previous company. A colleague wrote a script using a regex to remove these debug logs in the .py code. IHMO the clean design for that would be to support officially preprocessors in Python. My PEP opens the gate for that: https://www.python.org/dev/peps/pep-0511/

Re: [Python-ideas] Efficient debug logging

2017-02-16 Thread Barry Scott
> On 15 Feb 2017, at 21:06, Abe Dillon wrote: > > On 15.02.2017, 20:39 Kyle Lahnakoski wrote: > Log "levels" never made sense to me; how can a single dimension be useful > substitute for a number of binary switches? With log "levels", you either > don't have enough logging, or you drown in to

Re: [Python-ideas] Efficient debug logging

2017-02-16 Thread Barry Scott
So if python converted: debugLog( <# ‘format string %r’ % (expensive(),) #> ) Into: def __tmp__(): yield ‘format string %r’ % (expensive(),) debugLog( __tmp__ ) Then debugLog can detect the generator and call __next__ only if logging is enabled. I gues

Re: [Python-ideas] Efficient debug logging

2017-02-15 Thread Abe Dillon
On 15.02.2017, 20:39 Kyle Lahnakoski wrote: > Log "levels" never made sense to me; how can a single dimension be useful > substitute for a number of binary switches? With log "levels", you either > don't have enough logging, or you drown in too much logging (or you manage > a number of loggers, w

Re: [Python-ideas] Efficient debug logging

2017-02-15 Thread Sven R. Kunze
On 15.02.2017 12:55, Barry Scott wrote: The lazy eval sound like a generator. Exactly my thought. Sven ___ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/ps

Re: [Python-ideas] Efficient debug logging

2017-02-15 Thread Barry Scott
On Wednesday, 15 February 2017 22:18:31 GMT Steven D'Aprano wrote: > On Tue, Feb 14, 2017 at 03:51:05PM +, Barry Scott wrote: > > A common pattern I use is to have logging calls for debug and information > > with my applications. The logging calls can be separately enabled and > > disabled. >

Re: [Python-ideas] Efficient debug logging

2017-02-15 Thread Chris Angelico
On Wed, Feb 15, 2017 at 10:47 PM, Barry Scott wrote: > __repr__ is interesting however. > > Typically I describe in a string why the value is being logged. That means > that the object is always a string. I cannot recall using debugLog( obj ) > in production. > > dlog('This is the state of obj at

Re: [Python-ideas] Efficient debug logging

2017-02-15 Thread Barry Scott
On Wednesday, 15 February 2017 22:33:42 GMT Chris Angelico wrote: > On Wed, Feb 15, 2017 at 10:18 PM, Steven D'Aprano wrote: > > Python doesn't have thunks, but there is a relatively heavyweight > > solution for delayed evaluation: wrap the code in a function. > > > > debugLog( ‘info is %r’, lam

Re: [Python-ideas] Efficient debug logging

2017-02-15 Thread Chris Angelico
On Wed, Feb 15, 2017 at 10:18 PM, Steven D'Aprano wrote: > Python doesn't have thunks, but there is a relatively heavyweight > solution for delayed evaluation: wrap the code in a function. > > debugLog( ‘info is %r’, lambda: expensiveFunction() ) > > > and then adjust debugLog so that if the argum

Re: [Python-ideas] Efficient debug logging

2017-02-15 Thread Barry Scott
On Tuesday, 14 February 2017 22:00:31 GMT Barry wrote: > > On 14 Feb 2017, at 17:39, Kyle Lahnakoski wrote: > > > > > > Can you wrap the expensive functions in lambdas? And have your logger > > evaluate it, only if required? > > > >> debugLog( ‘info is %r’ % (lambda: expensiveFunction(),) ) >

Re: [Python-ideas] Efficient debug logging

2017-02-15 Thread Steven D'Aprano
On Tue, Feb 14, 2017 at 03:51:05PM +, Barry Scott wrote: > A common pattern I use is to have logging calls for debug and information > with my applications. > The logging calls can be separately enabled and disabled. [...] > What would be nice is to be able to avoid evaluation the tuple of arg

Re: [Python-ideas] Efficient debug logging

2017-02-15 Thread Barry Scott
> On 15 Feb 2017, at 02:39, Kyle Lahnakoski wrote: > > > On 2017-02-14 19:51, Abe Dillon wrote: >> The point is that the cost of creating the msg argument can be very high. >> >> At the point that logging decides to skip output it is to late to save >> the cost of creating the arg

Re: [Python-ideas] Efficient debug logging

2017-02-14 Thread Kyle Lahnakoski
On 2017-02-14 19:51, Abe Dillon wrote: >The point is that the cost of creating the msg argument can be very > high. > > At the point that logging decides to skip output it is to late to > save the cost of creating the arg tuple. > > This sounds like an optimization that's sufficiently

Re: [Python-ideas] Efficient debug logging

2017-02-14 Thread Abe Dillon
The point is that the cost of creating the msg argument can be very high. At the point that logging decides to skip output it is to late to save the cost of creating the arg tuple. This sounds like an optimization that's sufficiently rare and complex to warrant a custom fix or a 3rd p

Re: [Python-ideas] Efficient debug logging

2017-02-14 Thread Barry
> On 14 Feb 2017, at 17:03, Cory Benfield wrote: > > >> On 14 Feb 2017, at 15:51, Barry Scott wrote: >> And have the evaluation of the argument skipped unless its dbg_log is >> enabled. >> >> I cannot see how to do this with python as it stands. >> >> Something would have to be added to all

Re: [Python-ideas] Efficient debug logging

2017-02-14 Thread Barry
> On 14 Feb 2017, at 17:39, Kyle Lahnakoski wrote: > > > Can you wrap the expensive functions in lambdas? And have your logger > evaluate it, only if required? > >> debugLog( ‘info is %r’ % (lambda: expensiveFunction(),) ) Interesting idea. I will bench mark and see what the cost of the lamba

Re: [Python-ideas] Efficient debug logging

2017-02-14 Thread Abe Dillon
There are several mechanisms in the logging module to handle this use-case. First, note that logging functions can take multiple arguments ( https://docs.python.org/3/library/logging.html?highlight=logging#logging.Logger.debug ): >>> import logging >>> msg = "hello, %s! %s to %s you!" >>> args = (

Re: [Python-ideas] Efficient debug logging

2017-02-14 Thread MRAB
On 2017-02-14 15:51, Barry Scott wrote: A common pattern I use is to have logging calls for debug and information with my applications. The logging calls can be separately enabled and disabled. For example: debug_log_enabled = False def debugLog( msg ): If debug_log_enabled:

Re: [Python-ideas] Efficient debug logging

2017-02-14 Thread Stephan Houben
Seems slightly simpler to just make debugLog accept a callable as an alternative to a string. debugLog(lambda:( ‘info is %s’ % expensiveFunction()) ) Op 14 feb. 2017 18:42 schreef "Kyle Lahnakoski" : Can you wrap the expensive functions in lambdas? And have your logger evaluate it, only if req

Re: [Python-ideas] Efficient debug logging

2017-02-14 Thread Kyle Lahnakoski
Can you wrap the expensive functions in lambdas? And have your logger evaluate it, only if required? > debugLog( ‘info is %r’ % (lambda: expensiveFunction(),) ) On 2017-02-14 10:51, Barry Scott wrote: > A common pattern I use is to have logging calls for debug and information > with my applica

Re: [Python-ideas] Efficient debug logging

2017-02-14 Thread Cory Benfield
> On 14 Feb 2017, at 15:51, Barry Scott wrote: > And have the evaluation of the argument skipped unless its dbg_log is enabled. > > I cannot see how to do this with python as it stands. > > Something would have to be added to allow python to short circuit the > argument tuple evaluation. > >

[Python-ideas] Efficient debug logging

2017-02-14 Thread Barry Scott
A common pattern I use is to have logging calls for debug and information with my applications. The logging calls can be separately enabled and disabled. For example: debug_log_enabled = False def debugLog( msg ): If debug_log_enabled: print( ‘Debug: %s’ % (msg,) ) Then the ca