Re: Broker side partition round robin

Hi Liam, (+adding imgur links to the images) First of all, thanks for checking my doubt. I understand that the reason I notice this behavior is because our case differs from yours in one point: I'm not writing terabytes in a single day. I'm writing MB, but distribution matters in these MBs, this

Re: How to manually start ingesting in kafka source connector ?

Thank you for the link. In my environment I use strimzi (0.17.0). I realized that connector offset is created when kafka connect start up . So before creating connector , I am able to set offset for Kafka Connect. In addition to you link ,scenario in below link also helped my idea.

Re: Disk space - sharp increase in usage

WMF recently had an issue where Kafka broker disks were filling up with log segment data. It turned out that Kafka was not deleting old log segments because the oldest log segment had a message with a Kafka timestamp a year in the future. Since

Re: Broker side partition round robin

Hi, I can't see your images, they're not making it through the mailing list, sorry. On Tue, 2 Jun. 2020, 11:09 pm Vinicius Scheidegger, < vinicius.scheideg...@gmail.com> wrote: > Hi Liam, > > First of all, thanks for checking my doubt. > I understand that the reason I notice this behavior is

Re: Broker side partition round robin

Hi Liam, First of all, thanks for checking my doubt. I understand that the reason I notice this behavior is because our case differs from yours in one point: I'm not writing terabytes in a single day. I'm writing MB, but distribution matters in these MBs, this because processing in the consumer

Re: Disk space - sharp increase in usage

Hi Victoria, There are no metrics of when a config was changed. However, if you've been capturing the JMX metrics from the brokers, the metric kafka.cluster:name=ReplicasCount,partition=*,topic=*,type=Partition would show if replication factor was increased. As for retention time, if you're sure

Re: Disk space - sharp increase in usage

Regards kafka-logs directory, it was an interesting lead, we checked and it is the same. Regards replication factor and retention, I am not looking for current information, I am look for metrics that can give me information about a change. Still looking for more ideas On 02/06/2020, 11:31,

Re: Disk space - sharp increase in usage

> On Jun 2, 2020, at 12:56 AM, Victoria Zuberman > wrote: > > Hi, > > Background: > Kafka cluster > 7 brokers, with 4T disk each > version 2.3 (recently upgraded from 0.1.0 via 1.0.1) > > Problem: > Used disk space went from 40% to 80%. > Looking for root cause. > > Suspects: > > 1.

Disk space - sharp increase in usage

Hi, Background: Kafka cluster 7 brokers, with 4T disk each version 2.3 (recently upgraded from 0.1.0 via 1.0.1) Problem: Used disk space went from 40% to 80%. Looking for root cause. Suspects: 1. Incoming traffic Ruled out, according to metrics no significant change in “bytes in” for