Re: Facing issue while producing

2024-10-01 Thread Akash Jain
like a 3-node kafka cluster > > and replication_factor was configured 2 for the topic. > > > > On Wed, Aug 28, 2024 at 2:55 PM Akash Jain > > wrote: > > > >> Also what is the version of Java you are running? > >> > >> On Wed, Aug 28, 2024 at 1

Re: Kafka old messages

2024-10-01 Thread Akash Jain
Hello Kiran, Define old. In Kafka when you have a topic with: 1. cleanup.policy=delete - messages older than than the retention.ms will be deleted. For example - if you have a retention of 7 days, all messages older than 7 days will be deleted meaning the oldest message that you can recover is 7 d

Re: Issue - Kafka server shutting down automatically

2024-09-06 Thread Akash Jain
Hi Brijendra, I suspect that the path you are using for log.dir is being updated/modified/deleted etc. Can you ensure that this path -> D:\kafka\kafka_logs is not being cleaned/modified in anyway (manually/automatically/by some application)? Are you going to use windows server or linux in producti

Re: Facing issue while producing

2024-08-28 Thread Akash Jain
Also what is the version of Java you are running? On Wed, Aug 28, 2024 at 11:44 AM Akash Jain wrote: > Hi Vikram, please share some more details: > >1. producer version >2. broker version >3. broker side config you mentioned is "snappy", can you check what

Re: Facing issue while producing

2024-08-28 Thread Akash Jain
;far away' from the version of snappy in broker. On Tue, Aug 27, 2024 at 8:00 AM Vikram Singh wrote: > Hello Akash, > > It's the same on the broker side as well. > > *Broker side config:-* compression.type=snappy > > On Mon, Aug 26, 2024 at 9:30 PM Akash Jain >

Re: Facing issue while producing

2024-08-26 Thread Akash Jain
:-* > > producerProperties.put("compression.type",CloudKafkaProducerConfig. > COMPRESSION_TYPE); > > On Fri, Aug 23, 2024 at 5:56 PM Akash Jain > wrote: > > > Hi Vikram. Can you share you code snippet? Are you using compression on > > producer/broker side

Re: Facing issue while producing

2024-08-23 Thread Akash Jain
Hi Vikram. Can you share you code snippet? Are you using compression on producer/broker side? On Friday, August 23, 2024, Vikram Singh wrote: > Hello, > > I am facing issues while producing messages on kafka topics. I am facing > this issue randomly. Please help me by referring to the logs below

Re: Facing large election time in Kafka

2024-08-23 Thread Akash Jain
as yours. This is one thing you should consider - upgrade to a newer Kraft based cluster. On the current setup you can try to optimize zookeeper for better throughput/latency. On Fri, Aug 23, 2024 at 12:40 PM Akash Jain wrote: > HI Atul you use the word 'leader'. You mean the 'c

Re: Facing large election time in Kafka

2024-08-23 Thread Akash Jain
HI Atul you use the word 'leader'. You mean the 'controller'? Or you referring to the leader for each of the partitions? On Fri, Aug 23, 2024 at 7:44 AM Atul Sharma wrote: > Hi, > We are currently facing a prolonged leader election time, approx 2 mins, in > a Kafka cluster (version 2.8.2) that i

Re: Handling load distribution evenly, with ordering

2024-08-23 Thread Akash Jain
t come without its downsides - here is the link <https://www.confluent.io/blog/how-choose-number-topics-partitions-kafka-cluster/> for your reference. On Thu, Aug 22, 2024 at 8:04 AM Karthick wrote: > Thanks Akash Jain for your detailed explanation. I have answered your > queries. &

Re: Handling load distribution evenly, with ordering

2024-08-21 Thread Akash Jain
ote: > Hi Akash Jain > Thanks for the reply seeking help for the same to choose hashing logics. > Please refer/suggest any. > > On Sat, Aug 17, 2024 at 10:21 AM Akash Jain > wrote: > > > Hi Karthick. You could implement your own custom partitioner. > > > >

Re: Handling load distribution evenly, with ordering

2024-08-16 Thread Akash Jain
Hi Karthick. You could implement your own custom partitioner. On Saturday, August 17, 2024, Karthick wrote: > Hi Team, > > I'm using Kafka partitioning to maintain field-based ordering across > partitions, but I'm experiencing data skewness among the partitions. I have > 96 partitions, and I'm s

Visualizing kafka topics data

2018-08-03 Thread Akash Jain
In my Java application, I am using a third party library to collect the memory profile, and send to a Kafka topic as JSON data. How can I make sense out of that JSON data? Is there a tool which can ingest data from a kafka topic and visualize it? Thanks, Akash