[ https://issues.apache.org/jira/browse/KAFKA-8258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Shixiong Zhu updated KAFKA-8258: -------------------------------- Description: We noticed that the Spark's Kafka connector outputs a lot of following verbose logs: {code:java} 19/04/18 04:31:06 INFO Fetcher: [Consumer clientId=consumer-2, groupId=...] Resetting offset for partition ... to offset .... {code} It comes from [https://github.com/hachikuji/kafka/blob/76c796ca128c3c97231f3ebda994a07bb06b26aa/clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java#L567] This log was added in [https://github.com/apache/kafka/pull/4557] In Spark, we use `seekToEnd` to discover latest offsets of a topic. If there are thousands of partitions in this topic, it will output thousands of INFO logs every call Is it intentional? If not, can we change it to DEBUG? was: We noticed that the Spark's Kafka connector outputs a lot of following verbose logs: {code} 19/04/18 04:31:06 INFO Fetcher: [Consumer clientId=consumer-2, groupId=...] Resetting offset for partition ... to offset .... {code} It comes from https://github.com/hachikuji/kafka/blob/76c796ca128c3c97231f3ebda994a07bb06b26aa/clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java#L567 This log was added in https://github.com/apache/kafka/pull/4557 In Spark, we use `seekToEnd` to discover latest offsets of a topic. If there are thousands of partitions in this topic, it will output thousands of INFO logs. Is it intentional? If not, can we change it to DEBUG? > Verbose logs in org.apache.kafka.clients.consumer.internals.Fetcher > ------------------------------------------------------------------- > > Key: KAFKA-8258 > URL: https://issues.apache.org/jira/browse/KAFKA-8258 > Project: Kafka > Issue Type: Bug > Affects Versions: 1.1.0 > Reporter: Shixiong Zhu > Priority: Major > > We noticed that the Spark's Kafka connector outputs a lot of following > verbose logs: > {code:java} > 19/04/18 04:31:06 INFO Fetcher: [Consumer clientId=consumer-2, groupId=...] > Resetting offset for partition ... to offset .... > {code} > It comes from > [https://github.com/hachikuji/kafka/blob/76c796ca128c3c97231f3ebda994a07bb06b26aa/clients/src/main/java/org/apache/kafka/clients/consumer/internals/Fetcher.java#L567] > This log was added in [https://github.com/apache/kafka/pull/4557] > In Spark, we use `seekToEnd` to discover latest offsets of a topic. If there > are thousands of partitions in this topic, it will output thousands of INFO > logs every call > Is it intentional? If not, can we change it to DEBUG? -- This message was sent by Atlassian JIRA (v7.6.3#76005)