Sorry for late response. The reason that unused configs is in WARN is that if 
you misspell a config, it means that it will not apply. In some cases (default 
retention) you want know until too late. We wanted to warn admins about 
possible misconfigurations.

In the context of a company supporting Kafka - customers run logs at INFO level 
normally, so if we suspect a misconfiguration, we don't want to ask the 
customer to change level to DEBUG and bounce the broker. It is time consuming 
and can be risky.

*Gwen Shapira*
Product Manager | Confluent
650.450.2760 | @gwenshap
Follow us: Twitter ( https://twitter.com/ConfluentInc ) | blog ( 
http://www.confluent.io/blog )

Sent via Superhuman ( https://sprh.mn/?vip=g...@confluent.io )

On Mon, Jan 06, 2020 at 4:21 AM, Stanislav Kozlovski < stanis...@confluent.io > 
wrote:

> 
> 
> 
> Hey Artur,
> 
> 
> 
> Perhaps changing the log level to DEBUG is the simplest approach.
> 
> 
> 
> I wonder if other people know what the motivation behind the WARN log was?
> I'm struggling to think up of a scenario where I'd like to see unused
> values printed in anything above DEBUG.
> 
> 
> 
> Best,
> Stanislav
> 
> 
> 
> On Mon, Dec 30, 2019 at 12:52 PM Artur Burtsev < artjock@ gmail. com (
> artj...@gmail.com ) > wrote:
> 
> 
>> 
>> 
>> Hi,
>> 
>> 
>> 
>> Indeed changing the log level for the whole AbstractConfig is not an
>> option, because logAll is extremely useful.
>> 
>> 
>> 
>> Grouping warnings into 1 (with the count of unused only) will not be a
>> good option for us either. It will still be pretty noisy. Imagine we have
>> 32 partitions and scaled up the application to 32 instances then we still
>> have 32 warnings per application (instead of 96 now) while we would like
>> to have 0 warnings because we are perfectly aware of using
>> schema.registry.url and its totally fine, and we don't have to be warned
>> every time we start the application. Now imagine we use more than one
>> consumer per application, then it will add another multiplication factor
>> to these grouped warnings and we still have a lot of those. So I would say
>> grouping doesn't help much.
>> 
>> 
>> 
>> I think adding extra logger like
>> "org.apache.kafka.clients.producer.ProducerConfig.unused" could be another
>> good option. That would leave the existing interface untouched and give
>> everyone an option to mute irrelevant warnings.
>> 
>> 
>> 
>> To summarize, I still can see 3 options with its pros and cons discussed
>> in the thread:
>> 1) extra config with interface to handle unused
>> 2) change unused warn to debug
>> 3) add extra logger for unused
>> 
>> 
>> 
>> Please let me know what do you think.
>> 
>> 
>> 
>> Thanks,
>> Artur
>> 
>> 
>> 
>> On Mon, Dec 30, 2019 at 11:07 AM Stanislav Kozlovski
>> < stanislav@ confluent. io ( stanis...@confluent.io ) > wrote:
>> 
>> 
>>> 
>>> 
>>> Hi all,
>>> 
>>> 
>>> 
>>> Would printing all the unused configurations in one line, versus N lines,
>>> be more helpful? I know that it would greatly reduce the verbosity in log
>>> visualization tools like Kibana while still allowing us to see all the
>>> relevant information without the need for an explicit action (e.g changing
>>> the log level)
>>> 
>>> 
>>> 
>>> Best,
>>> Stanislav
>>> 
>>> 
>>> 
>>> On Sat, Dec 28, 2019 at 3:13 PM John Roesler < vvcephei@ apache. org (
>>> vvcep...@apache.org ) >
>>> 
>>> 
>> 
>> 
>> 
>> wrote:
>> 
>> 
>>> 
>>>> 
>>>> 
>>>> Hi Artur,
>>>> 
>>>> 
>>>> 
>>>> That’s a good point.
>>>> 
>>>> 
>>>> 
>>>> One thing you can do is log a summary at WARN level, like “27
>>>> configurations were ignored. Ignored configurations are logged at DEBUG
>>>> level.”
>>>> 
>>>> 
>>>> 
>>>> I looked into the code a little, and these log messages are generated
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
>> 
>> in
>> 
>> 
>>> 
>>>> 
>>>> 
>>>> AbstractConfig (logAll and logUnused). They both use the logger
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
>> 
>> associated
>> 
>> 
>>> 
>>>> 
>>>> 
>>>> with the relevant config class (StreamsConfig, ProducerConfig, etc.).
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
>> 
>> The
>> 
>> 
>>> 
>>>> 
>>>> 
>>>> list of all configs is logged at INFO level, and the list of unused
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
>> 
>> configs
>> 
>> 
>>> 
>>>> 
>>>> 
>>>> is logged at WARN level. This means that it's not possible to silence
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
>> 
>> the
>> 
>> 
>>> 
>>>> 
>>>> 
>>>> unused config messages while still logging the list of all configs. You
>>>> could only silence both by setting (for example) ProducerConfig logger
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
>> 
>> to
>> 
>> 
>>> 
>>>> 
>>>> 
>>>> ERROR or OFF.
>>>> 
>>>> 
>>>> 
>>>> If it's desirable to be able to toggle them independently, then you can
>>>> create a separate logger for unused configs, named something like
>>>> "org.apache.kafka.clients.producer.ProducerConfig.unused". Then, you
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
>> 
>> can
>> 
>> 
>>> 
>>>> 
>>>> 
>>>> leave the log at WARN, so it would continue to be printed by default,
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
>> 
>> and
>> 
>> 
>>> 
>>>> 
>>>> 
>>>> anyone could disable it by setting
>>>> "org.apache.kafka.clients.producer.ProducerConfig.unused" to ERROR or
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
>> 
>> OFF,
>> 
>> 
>>> 
>>>> 
>>>> 
>>>> without disturbing the rest of the config log messages.
>>>> 
>>>> 
>>>> 
>>>> It's simpler without the extra logger, but you also get less control.
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
>> 
>> Do
>> 
>> 
>>> 
>>>> 
>>>> 
>>>> you think the extra control is necessary, versus printing a summary at
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
>> 
>> WARN
>> 
>> 
>>> 
>>>> 
>>>> 
>>>> level?
>>>> -John
>>>> 
>>>> 
>>>> 
>>>> On Fri, Dec 27, 2019, at 04:26, Artur Burtsev wrote:
>>>> 
>>>> 
>>>>> 
>>>>> 
>>>>> Hi,
>>>>> 
>>>>> 
>>>>> 
>>>>> Indeed changing log level to debug would be the easiest and I think that
>>>>> would be a good solution. When no one object I'm ready to move forward
>>>>> with this approach and submit a MR.
>>>>> 
>>>>> 
>>>>> 
>>>>> The only minor thing I have – having it at debug log level might make it a
>>>>> bit less friendly for developers, especially for those who just do the
>>>>> first steps in Kafka. For example, if you misspelled the property name and
>>>>> trying to understand why things don't do what you expect. Having a warning
>>>>> might save some time in this case. Other
>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
>> 
>> than
>> 
>> 
>>> 
>>>> 
>>>>> 
>>>>> 
>>>>> that I cannot see any reasons to have warnings there.
>>>>> 
>>>>> 
>>>>> 
>>>>> Thanks,
>>>>> Artur
>>>>> 
>>>>> 
>>>>> 
>>>>> On Thu, Dec 26, 2019 at 10:01 PM John Roesler < vvcephei@ apache. org (
>>>>> vvcep...@apache.org ) >
>>>>> 
>>>>> 
>>>> 
>>>> 
>>>> 
>>>> wrote:
>>>> 
>>>> 
>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Thanks for the KIP, Artur!
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> For reference, here is the kip:
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
>> 
>> https:/ / cwiki. apache. org/ confluence/ display/ KAFKA/ 
>> KIP-552%3A+Add+interface+to+handle+unused+config
>> (
>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-552%3A+Add+interface+to+handle+unused+config
>> )
>> 
>> 
>>> 
>>>> 
>>>>> 
>>>>>> 
>>>>>> 
>>>>>> I agree, these warnings are kind of a nuisance. Would it be
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
>> 
>> feasible
>> 
>> 
>>> 
>>>> 
>>>> 
>>>> just to leverage log4j in some way to make it easy to filter these
>>>> messages? For example, we could move those warnings to debug level, or
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
>> 
>> even
>> 
>> 
>>> 
>>>> 
>>>> 
>>>> use a separate logger for them.
>>>> 
>>>> 
>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Thanks for starting the discussion.
>>>>>> -John
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Tue, Dec 24, 2019, at 07:23, Artur Burtsev wrote:
>>>>>> 
>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> Hi,
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> This KIP provides a way to deal with a warning "The
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
>> 
>> configuration {}
>> 
>> 
>>> 
>>>> 
>>>>> 
>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> was supplied but isn't a known config." when it is not relevant.
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> Cheers,
>>>>>>> Artur
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>>> 
>>> 
>>> --
>>> Best,
>>> Stanislav
>>> 
>>> 
>> 
>> 
> 
> 
> 
> --
> Best,
> Stanislav
> 
> 
>

Reply via email to