Hi, Background: Kafka cluster 7 brokers, with 4T disk each version 2.3 (recently upgraded from 0.1.0 via 1.0.1)
Problem: Used disk space went from 40% to 80%. Looking for root cause. Suspects: 1. Incoming traffic Ruled out, according to metrics no significant change in “bytes in” for topics in cluster 1. Upgrade The raise started on the day of upgrade to 2.3 But we upgraded another cluster in the same way and we don’t see similar issue there Is there a known change or issue at 2.3 related to disk space usage? 1. Replication factor Is there a way to see whether replication factor of any topic was changed recently? Didn’t find in metrics... 1. Retention Is there a way to see whether retention was changed recently? Didn’t find in metrics... Would appreciate any other ideas or investigation leads Thanks, Victoria ------------------------------------------- NOTICE: This email and all attachments are confidential, may be proprietary, and may be privileged or otherwise protected from disclosure. They are intended solely for the individual or entity to whom the email is addressed. However, mistakes sometimes happen in addressing emails. If you believe that you are not an intended recipient, please stop reading immediately. Do not copy, forward, or rely on the contents in any way. Notify the sender and/or Imperva, Inc. by telephone at +1 (650) 832-6006 and then delete or destroy any copy of this email and its attachments. The sender reserves and asserts all rights to confidentiality, as well as any privileges that may apply. Any disclosure, copying, distribution or action taken or omitted to be taken by an unintended recipient in reliance on this message is prohibited and may be unlawful. Please consider the environment before printing this email.
