[issue41906] logging.config.dictConfig does not work with callable filters

[Vinay Sajip]

Change by Vinay Sajip :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.11 -Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41906] logging.config.dictConfig does not work with callable filters

[miss-islington]

miss-islington  added the comment:


New changeset d7c68639795a576ff58b6479c8bb34c113df3618 by Mario Corchero in 
branch 'main':
bpo-41906: Accept built filters in dictConfig (GH-30756)
https://github.com/python/cpython/commit/d7c68639795a576ff58b6479c8bb34c113df3618


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41906] logging.config.dictConfig does not work with callable filters

[Mario Corchero]

Change by Mario Corchero :


--
keywords: +patch
pull_requests: +28943
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/30756

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41906] logging.config.dictConfig does not work with callable filters

[Vinay Sajip]

Vinay Sajip  added the comment:

> If you have any preference about the implementation or any pointer

Nothing particular, just try to fit in with the existing code conventions even 
where they don't follow modern idioms (e.g. a lot of the code in this package 
predates PEP 8 and new styles of string formatting).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41906] logging.config.dictConfig does not work with callable filters

[Mario Corchero]

Mario Corchero  added the comment:

Great! I'll put something together then. If you have any preference about the 
implementation or any pointer on the way you think should be done, please let 
me know.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41906] logging.config.dictConfig does not work with callable filters

[Vinay Sajip]

Vinay Sajip  added the comment:

> Vinay would you consider a patch for logging where dictConfig allows taking 
> objects directly in addition to the reference id?

Sorry I didn't respond to this; it dropped off my radar. Certainly, I would 
consider such a patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41906] logging.config.dictConfig does not work with callable filters

[Petr]

Petr  added the comment:

I would definitely vote for implementing this enhancement. I have just ran into 
the very same issue and my search ended here. Using dictConfig e.g. with 
lambdas seems very natural to me and I understood the docs incorrectly exactly 
as had been reported.

--
nosy: +PetrPy111

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41906] logging.config.dictConfig does not work with callable filters

[László Kiss Kollár]

Change by László Kiss Kollár :


--
nosy: +lkollar

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41906] logging.config.dictConfig does not work with callable filters

[Mario Corchero]

Mario Corchero  added the comment:

Vinay would you consider a patch for logging where dictConfig allows taking 
objects directly in addition to the reference id?

That would allow the following:

```

def noErrorLogs(param):
return 1 if param.levelno < 40 else 0

logconfig_dict = {
"handlers": {
"console": {
"class": "logging.StreamHandler",
"level": "DEBUG",
"stream": "ext://sys.stdout",
"filters": [noErrorLogs]
}
},
"root": {"level": "DEBUG", "handlers": ["console"]},
"version": 1,
}
dictConfig(logconfig_dict)
```

or alternatively passing them on declaration:

```
logconfig_dict = {
'filters': {
'myfilter': noErrorLogs,
},
"handlers": {
"console": {
"class": "logging.StreamHandler",
"level": "DEBUG",
"stream": "ext://sys.stdout",
"filters": ["myfilter"]
}
},
"root": {"level": "DEBUG", "handlers": ["console"]},
"version": 1,
}
dictConfig(logconfig_dict)
```

I'm happy to put a patch together if that looks good to you.

--
nosy: +mariocj89

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41906] logging.config.dictConfig does not work with callable filters

[lorb]

lorb  added the comment:

I looked into implementing this and it's not entirely clear how it could work. 
The main issue I encountered is how to distinguish between a callable that is a 
factory that takes one argument and a callable that is a filter.

One possible approach I see is to rely on the absence of any other keys in the 
configuration sub-directory. Not entirely happy with that (and have to think 
how to treat factory with optional parameter vs filter with optional parameter).
So this would be assumed to be a filter, if inspection reveals that fltr takes 
an argument:
'myfilter': {
'()': fltr,
}
while this would be assumed to be a factory:
'myfilter': {
'()': fctry,
'bar': 'baz',
}

A different approach that I think would be not optimal is to just call the 
callable with a dummy LogRecord passed ind and see if it returns a boolean or 
callable (or throws).

--
nosy: +lorb

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41906] logging.config.dictConfig does not work with callable filters

[Vinay Sajip]

Vinay Sajip  added the comment:

You make reasonable points. I won't close this issue, and get to those updates 
when I can/as time allows. Meanwhile, if you or someone else proposes specific 
changes by way of a pull request, I'll look at that too. And thanks for your 
kind words about the logging docs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41906] logging.config.dictConfig does not work with callable filters

[raybb]

raybb  added the comment:

Thank you for the clarification.

I think I was most confused by the docs on this page (which I should have 
included in my initial post): https://docs.python.org/3/howto/logging.html

It says: 

"In Python 3.2, a new means of configuring logging has been introduced, using 
dictionaries to hold configuration information. This provides a superset of the 
functionality of the config-file-based approach outlined above, and is the 
recommended configuration method for new applications and deployments."


Since it is the recommended configuration method I had naively assumed that it 
would be compatible with the feature of just using callables instead which was 
mentioned here https://docs.python.org/3/library/logging.html. 


I think it would be a nice enhancement long term to be able too support 
callables in dictconfigs.

In the short term do you think it is reasonable to update the first page 
mentioned in this comment to clarify that even though dictConfig is recommended 
it is not a feature parity with the object oriented api?

It would also be nice to clarify this on the second page to say that callables 
don't work if using dictConfig.


I understand it does say a factory is required on the page you linked. I think 
anyone who reads the docs as well as they should will find it. But I think 
adding information about this too other parts of the docs will make it much 
easier for newer folks to catch the difference early on.


Thanks for your work on the logging parts of the api in python. I've read many 
of your docs and they are very helpful.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41906] logging.config.dictConfig does not work with callable filters

[Vinay Sajip]

Vinay Sajip  added the comment:

The value of the '()' key should be a factory - something that returns a filter 
(either something that has a filter method or a callable). That returned 
callable, or the filter method, will be called with a LogRecord and its return 
value used to determine whether the event should be processed. An example of 
using factories is here:

https://docs.python.org/3/library/logging.config.html#user-defined-objects

In your first example, you're passing the filter where you need to pass the 
factory. If you change it to e.g. return a lambda that does the filtering, it 
should work as expected.

At the moment you can't directly pass a filter callable in the config 
dictionary - only a factory that returns a filter. I'll look at this as a 
possible enhancement.

--
type: behavior -> enhancement
versions: +Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41906] logging.config.dictConfig does not work with callable filters

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +vinay.sajip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41906] logging.config.dictConfig does not work with callable filters

[raybb]

New submission from raybb :

According to the docs here (https://docs.python.org/3/library/logging.html):

"You don’t need to create specialized Filter classes, or use other classes with 
a filter method: you can use a function (or other callable) as a filter. The 
filtering logic will check to see if the filter object has a filter attribute: 
if it does, it’s assumed to be a Filter and its filter() method is called. 
Otherwise, it’s assumed to be a callable and called with the record as the 
single parameter."


If I use this code:

def noErrorLogs(param):
return 1 if param.levelno < 40 else 0

logconfig_dict = {
'filters': {
'myfilter': {
'()': noErrorLogs,
}
},
"handlers": {
"console": {
"class": "logging.StreamHandler",
"level": "DEBUG",
"stream": "ext://sys.stdout",
"filters": ["myfilter"]
}
},
"root": {"level": "DEBUG", "handlers": ["console"]},
"version": 1,
}
dictConfig(logconfig_dict)

I get the error "Unable to configure filter 'myfilter'" because "noErrorLogs() 
missing 1 required positional argument: 'param'"


However, If I use this code:


def noErrorLogs(param):
return 1 if param.levelno < 40 else 0

logconfig_dict = {
"handlers": {
"console": {
"class": "logging.StreamHandler",
"level": "DEBUG",
"stream": "ext://sys.stdout",
}
},
"root": {"level": "DEBUG", "handlers": ["console"]},
"version": 1,
}

logging.basicConfig()
dictConfig(logconfig_dict)
l = logging.getLogger()
l.handlers[0].addFilter(noErrorLogs)

Then the filter works correctly.




The bug I am reporting is that when using logging.config.dictConfig you cannot 
pass in a callable that acts a filter. You can only pass in a class with a 
filter method or a function that returns a callable filter.

If this is the expected behavior perhaps the docs should make it clear that 
callables cannot be used with dictConfig.

--
messages: 377791
nosy: raybb
priority: normal
severity: normal
status: open
title: logging.config.dictConfig does not work with callable filters
type: behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com