Re: Handling load distribution evenly, with ordering

2024-08-21 Thread Karthick
gt; Hi Karthick, > > The choice has to be yours depending on what you want to achieve. I > understand you want to achieve even distribution of messages across your > partitions. This depends on the following factors: > >- The frequency of keys >- Hashing logic itself >

Re: Handling load distribution evenly, with ordering

2024-08-20 Thread Karthick
Hi Akash Jain Thanks for the reply seeking help for the same to choose hashing logics. Please refer/suggest any. On Sat, Aug 17, 2024 at 10:21 AM Akash Jain wrote: > Hi Karthick. You could implement your own custom partitioner. > > On Saturday, August 17, 2024, Karthick wrote: >

Handling load distribution evenly, with ordering

2024-08-16 Thread Karthick
Hi Team, I'm using Kafka partitioning to maintain field-based ordering across partitions, but I'm experiencing data skewness among the partitions. I have 96 partitions, and I'm sending data with 500 distinct keys that are used for partitioning. While monitoring the Kafka cluster, I noticed that a

Re: Requesting Feedback for a backup solution

2024-03-08 Thread Karthick
Hi guys, I'm still seeking for a solution, I hope many face this problem in production. Any working solutions will help us. On Sat, Feb 17, 2024 at 2:16 PM Karthick wrote: > Dear Kafka Community, > > I am reaching out to seek your valuable feedback and insights on a > propose

Requesting Feedback for a backup solution

2024-02-17 Thread Karthick
look forward to hearing from you soon. Thanks and regards, Karthick

Having issue with Kafka partition

2018-10-29 Thread Karthick Kumar
Hi All, Recently I faced the Below issue WARN Fetcher:679 - Received unknown topic or partition error in ListOffset request for partition messages-2-1. The topic/partition may not exist or the user may not have Describe access to it My consumer is not working for the particular topic. When I ch

zookeeper connection issue

2018-06-28 Thread Karthick Kumar
Hi, I'm using Kafka cluster on three different servers, Recently my servers went down when I start the server and then start the services, One of the Zookeeper is not connected to the Kafka cluster and it stayed there for two days... So I killed the stack in Rancher and then start the new one all

Facing Duplication Issue in kakfa

2018-05-28 Thread Karthick Kumar
Hi, Facing Duplication inconsistently while bouncing Kafka producer and consumer in tomcat node. any help will be appreciated to find out the root cause. -- With Regards, Karthick.K

How to achieve exactly one semantic in Kafka consumer

2018-05-22 Thread Karthick Kumar
Hi, Currently, I'm using kafka_2.11-0.10.2.0. In this working in almost once semantic in Kafka consumer, But I need to change that in exactly once semantics. -- With Regards, Karthick. K

Re: Removing the Kafka DEBUG logs in catalina.out using log4j.properties

2018-05-21 Thread Karthick Kumar
Okay Kaufman, will check on that On Sun, May 20, 2018 at 12:08 AM, Kaufman Ng wrote: > Karthick, > > I am suspecting that it could be the log4j.properties in your webapp is not > picked up by Tomcat, or somehow Tomcat is overriding your webapp's log4j > setup. > > On

Re: Removing the Kafka DEBUG logs in catalina.out using log4j.properties

2018-05-14 Thread Karthick Kumar
is will remove all DEBUG lines where the logger name starts with > > org.apache.kafka. > > > > HTH, > > Andras > > > > On Fri, May 11, 2018 at 9:28 AM, Karthick Kumar > > wrote: > > > > > Hi, > > > > > > I'm using tomcat no

Re: Kafka consumer slowly consume the data

2018-05-14 Thread Karthick Kumar
anks > > On Fri, May 11, 2018 at 6:41 AM, Karthick Kumar > wrote: > > > Hi, > > > > I'm using tomcat nodes for Kafka producer and consumer, Recently I faced > > some issues with it. > > Normally the producer and consumer counts matched in the tomcat

Re: Kafka consumer slowly consume the data

2018-05-11 Thread Karthick Kumar
Hi, I'm using tomcat nodes for Kafka producer and consumer, Recently I faced some issues with it. Normally the producer and consumer counts matched in the tomcat nodes. After some time the produced data is consumed with delay, I'm not sure where to check. The data which was delayed is dumped over

Re: Removing the Kafka DEBUG logs in catalina.out using log4j.properties

2018-05-11 Thread Karthick Kumar
Hi, I'm using tomcat node as a Kafka consumer, It prints the INFO, DEBUG and ERROR logs. When I analyzed in log file debug logs are taking more space. So i'm having disk space issue. I'm using *log4j.properties* for managing the logs, Now I want to remove the DEBUG logs from my logger file. Any