Hello Henry, What's your server-side log4j settings? Could you use WARN on these two classes: kafka.server.epoch.LeaderEpochFileCache and kafka.log.Log.
Guozhang On Fri, Jul 6, 2018 at 3:08 PM, Henry Cai <h...@pinterest.com.invalid> wrote: > @guozhang > > After we moved to kafka-1.1.0 for our Kafka streams application, our broker > logs are polluted with loggings such as: > > [2018-07-06 21:59:26,170] INFO Cleared earliest 0 entries from epoch cache > based on passed offset 301483601 leaving 1 in EpochFile for partition > inflight_spend_unified_staging-single_spend_agg_ > window_AD_GROUP-repartition-26 > (kafka.server.epoch.LeaderEpochFileCache) > > [2018-07-06 21:59:26,170] INFO [Log > partition=inflight_spend_unified_staging-single_spend_agg_window_AD_GROUP- > repartition-1, > dir=/mnt/kafka] Incrementing log start offset to 240548684 (kafka.log.Log) > > > Thousands of them keeping rolling in the broker logs which makes server > side log unusable. > > > Looks like this is triggered by DELETE_RECORDS requests from StreamsThread > from 'KAFKA-6150: Make Repartition Topics Transient' > > > Can you suppress these two INFO loggings on the server side if they are > triggered by AdminClient.deleteRecords() > > > We have thousands of partitions per broker, those deletes were happening > too frequent. > -- -- Guozhang
