Hi, I updated KIP https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=142641934 and submitted PR accordingly to the discussion in this thread https://github.com/apache/kafka/pull/8043
Please have a look, Artur On Mon, Jan 6, 2020 at 1:22 PM Stanislav Kozlovski <stanis...@confluent.io> wrote: > > Hey Artur, > > Perhaps changing the log level to DEBUG is the simplest approach. > > I wonder if other people know what the motivation behind the WARN log was? > I'm struggling to think up of a scenario where I'd like to see unused > values printed in anything above DEBUG. > > Best, > Stanislav > > On Mon, Dec 30, 2019 at 12:52 PM Artur Burtsev <artj...@gmail.com> wrote: > > > Hi, > > > > Indeed changing the log level for the whole AbstractConfig is not an > > option, because logAll is extremely useful. > > > > Grouping warnings into 1 (with the count of unused only) will not be a > > good option for us either. It will still be pretty noisy. Imagine we > > have 32 partitions and scaled up the application to 32 instances then > > we still have 32 warnings per application (instead of 96 now) while we > > would like to have 0 warnings because we are perfectly aware of using > > schema.registry.url and its totally fine, and we don't have to be > > warned every time we start the application. Now imagine we use more > > than one consumer per application, then it will add another > > multiplication factor to these grouped warnings and we still have a > > lot of those. So I would say grouping doesn't help much. > > > > I think adding extra logger like > > "org.apache.kafka.clients.producer.ProducerConfig.unused" could be > > another good option. That would leave the existing interface untouched > > and give everyone an option to mute irrelevant warnings. > > > > To summarize, I still can see 3 options with its pros and cons > > discussed in the thread: > > 1) extra config with interface to handle unused > > 2) change unused warn to debug > > 3) add extra logger for unused > > > > Please let me know what do you think. > > > > Thanks, > > Artur > > > > On Mon, Dec 30, 2019 at 11:07 AM Stanislav Kozlovski > > <stanis...@confluent.io> wrote: > > > > > > Hi all, > > > > > > Would printing all the unused configurations in one line, versus N lines, > > > be more helpful? I know that it would greatly reduce the verbosity in log > > > visualization tools like Kibana while still allowing us to see all the > > > relevant information without the need for an explicit action (e.g > > > changing the log level) > > > > > > Best, > > > Stanislav > > > > > > On Sat, Dec 28, 2019 at 3:13 PM John Roesler <vvcep...@apache.org> > > wrote: > > > > > > > Hi Artur, > > > > > > > > That’s a good point. > > > > > > > > One thing you can do is log a summary at WARN level, like “27 > > > > configurations were ignored. Ignored configurations are logged at DEBUG > > > > level.” > > > > > > > > I looked into the code a little, and these log messages are generated > > in > > > > AbstractConfig (logAll and logUnused). They both use the logger > > associated > > > > with the relevant config class (StreamsConfig, ProducerConfig, etc.). > > The > > > > list of all configs is logged at INFO level, and the list of unused > > configs > > > > is logged at WARN level. This means that it's not possible to silence > > the > > > > unused config messages while still logging the list of all configs. You > > > > could only silence both by setting (for example) ProducerConfig logger > > to > > > > ERROR or OFF. > > > > > > > > If it's desirable to be able to toggle them independently, then you can > > > > create a separate logger for unused configs, named something like > > > > "org.apache.kafka.clients.producer.ProducerConfig.unused". Then, you > > can > > > > leave the log at WARN, so it would continue to be printed by default, > > and > > > > anyone could disable it by setting > > > > "org.apache.kafka.clients.producer.ProducerConfig.unused" to ERROR or > > OFF, > > > > without disturbing the rest of the config log messages. > > > > > > > > It's simpler without the extra logger, but you also get less control. > > Do > > > > you think the extra control is necessary, versus printing a summary at > > WARN > > > > level? > > > > -John > > > > > > > > > > > > On Fri, Dec 27, 2019, at 04:26, Artur Burtsev wrote: > > > > > Hi, > > > > > > > > > > Indeed changing log level to debug would be the easiest and I think > > > > > that would be a good solution. When no one object I'm ready to move > > > > > forward with this approach and submit a MR. > > > > > > > > > > The only minor thing I have – having it at debug log level might make > > > > > it a bit less friendly for developers, especially for those who just > > > > > do the first steps in Kafka. For example, if you misspelled the > > > > > property name and trying to understand why things don't do what you > > > > > expect. Having a warning might save some time in this case. Other > > than > > > > > that I cannot see any reasons to have warnings there. > > > > > > > > > > Thanks, > > > > > Artur > > > > > > > > > > On Thu, Dec 26, 2019 at 10:01 PM John Roesler <vvcep...@apache.org> > > > > wrote: > > > > > > > > > > > > Thanks for the KIP, Artur! > > > > > > > > > > > > For reference, here is the kip: > > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-552%3A+Add+interface+to+handle+unused+config > > > > > > > > > > > > I agree, these warnings are kind of a nuisance. Would it be > > feasible > > > > just to leverage log4j in some way to make it easy to filter these > > > > messages? For example, we could move those warnings to debug level, or > > even > > > > use a separate logger for them. > > > > > > > > > > > > Thanks for starting the discussion. > > > > > > -John > > > > > > > > > > > > On Tue, Dec 24, 2019, at 07:23, Artur Burtsev wrote: > > > > > > > Hi, > > > > > > > > > > > > > > This KIP provides a way to deal with a warning "The > > configuration {} > > > > > > > was supplied but isn't a known config." when it is not relevant. > > > > > > > > > > > > > > Cheers, > > > > > > > Artur > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > Best, > > > Stanislav > > > > > -- > Best, > Stanislav
