Hi SenthilKumar,
thanks a lot .
Yeah I set up local set up and print the log with partition info which
shows as below
2019-07-03 02:48:28.822 INFO 7092 --- [lt-dispatcher-2]
c.s.c.p.p.CompanyInfoPartitioner : Topic : inbound_topic Key = 597736248-
Entropy Cayman Solar Ltd.-null-null-null *Partition = -1* 2019-07-03
02:48:28.931 ERROR 7092 --- [ad | producer-1]
o.s.k.support.LoggingProducerListener : Exception thrown when sending a
message with key='597736248- Entropy Cayman Solar Ltd.-null-null-null' and
payload='com.spgmi.ca.prescore.model.Company@8b12343' to topic
inbound_topic :
org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for
inbound_topic --1: 104 ms has passed since batch creation plus linger time
My topics inbound_topic has two partitions as you see below
C:\Software\kafka\kafka_2.11-2.1.1\bin\windows>kafka-topics.bat --describe
--zookeeper localhost:2181 --topic inbound_topic Topic:inbound_topic
PartitionCount:2 ReplicationFactor:1 Configs: Topic: inbound_topic
Partition: 0 Leader: 0 Replicas: 0 Isr: 0 Topic: inbound_topic Partition: 1
Leader: 0 Replicas: 0 Isr: 0
*But my producer seems to trying to send to Partition = -1.*
My partition logic is as below
int p = (((String)key).hashCode() * Integer.MAX_VALUE) % numPartitions;
logger.info("Topic : "+ topic + "\t Key = " + (String)key + "
Partition = " + p );
On key i am doing hashCode(). What need to be corrected here to avoid this
negative number partition number ? i.e. Partition = -1
What should be my partition key logic like ?
any help highly appreciated.
Regards,
Shyam
On Tue, Jul 2, 2019 at 8:48 PM SenthilKumar K <senthilec...@gmail.com>
wrote:
> Does it happen to all partitions or only few partitions ? Can you make
> sure your local setup working fine ? Were you able to produce using
> console-producer ?
>
> Example :
> EVERE: Expiring 7 record(s) for topic-9{partition:9}: 30022 ms has passed
> since last append
> Expiring 9 record(s) for topic-2{partition:2}: 30015 ms has passed since
> batch creation plus linger time
>
> --Senthil
>
> On Tue, Jul 2, 2019 at 5:34 PM Shyam P <shyamabigd...@gmail.com> wrote:
>
>> Thanks a lot Senthil for quick reply.
>> I am using kafka_2.11-2.1.1 .
>> In your case Kafka Producer Client in One DataCenter and Kafka Broker in
>> other DataCenter but in my case I installed Kafka on the same machine
>> where Producer is running.
>> i.e. currently I am in development mode , so everything now on my local
>> for timebeing ...i.e. Kafka broker , zk and my producer code in eclipse.
>>
>> If is is a set up issue at least it should run fine in my local right.
>> I tried several producer configurations combinations as explained in the
>> SOF link.
>>
>> So not sure now what is the issue and how to fix it ?
>>
>> Is in your case the issue fixed ?
>>
>> Regards,
>> Shyam
>>
>> On Tue, Jul 2, 2019 at 5:12 PM SenthilKumar K <senthilec...@gmail.com>
>> wrote:
>>
>>> Hi Shyam, We also faced `TimeoutException: Expiring 1 record(s)` issue in
>>> our Kafka Producer Client. As described here
>>> <
>>> https://stackoverflow.com/questions/56807188/how-to-fix-kafka-common-errors-timeoutexception-expiring-1-records-xxx-ms-has
>>> >
>>> ,
>>> first we tried increasing request timeout but that didn't help. We had
>>> setup like Kafka Producer Client in One DataCenter and Kafka Broker in
>>> other DataCenter & thats why the producer failed to push records to
>>> brokers
>>> on time due to network issue. In your case , Could be setup issue ?
>>>
>>> --Senthil
>>>
>>> On Tue, Jul 2, 2019 at 3:57 PM Shyam P <shyamabigd...@gmail.com> wrote:
>>>
>>> > Hi,
>>> > I am facing the below issue.
>>> >
>>> > org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s)
>>> > for <topic-name> 229 ms has passed since batch creation plus linger
>>> > time
>>> >
>>> >
>>> > I tried many producer configuration settings. more details below :
>>> >
>>> >
>>> https://stackoverflow.com/questions/56807188/how-to-fix-kafka-common-errors-timeoutexception-expiring-1-records-xxx-ms-has
>>> >
>>> > But nothing working.
>>> >
>>> > Can anyone plz help me , what is wrong here and how to fix it ?
>>> >
>>> > thanks,
>>> > Shyam
>>> >
>>>
>>
