Re: Consumer not receiving messages after connection loss

2020-05-03 Thread Goran Sliskovic
Hi, It's the client that closes the connection, you can see some disconnectExceptions in the client log, however it would be nice if there was a log: "Closing connection due to request timeout" somwhere. This would save me some time parsing wireshark logs, though when you look at that carefully

Re: Kafka: Messages disappearing from topics, largestTime=0

2020-05-01 Thread Goran Sliskovic
d > > > > 17 -> Exception in thread "main" java.io.IOException: Permission > denied > > > > > > > > > So partition 11 and 14 timeindex files were the ones with the timestamp > > > mismatch error that didn't lose messages immediately after

Re: Kafka: Messages disappearing from topics, largestTime=0

2020-04-30 Thread Goran Sliskovic
Hi, It does look as index corruption... Can you post script that stops kafka? On Wednesday, April 29, 2020, 06:38:18 PM GMT+2, JP MB wrote: > > Can you try using the console consumer to display messages/keys and > timestamps ? > --property print.key=true --property print.timestamp=tru

Re: Kafka: Messages disappearing from topics, largestTime=0

2020-04-29 Thread Goran Sliskovic
Hi, When lastModifiedTime on that segment is converted to human readable time: Monday, April 27, 2020 9:14:19 AM UTC In what time zone is the server (IOW: [2020-04-27 10:36:40,386] from the log is in what time zone)?  It looks as largestTime is property of log record and 0 means the log record i

Re: Kafka Connect ignores the bootstrap.servers parameter

2020-04-28 Thread Goran Sliskovic
Client connects to IP address defined by bootstrap.servers and gets metada that contain IP address where kafka is available. That is configured in server.properties: # Hostname and port the broker will advertise to producers and consumers. If not set, # it uses the value for "listeners" if conf

Re: Consumer not receiving messages after connection loss

2020-04-27 Thread Goran Sliskovic
max.partion.fetch,bytes or increasing request.timeout.ms should allow kafka client to consume under low speed condition. Diagnostic would be nice in the kafka log though, at least at the info level.. .   On Sunday, April 26, 2020, 01:41:34 PM GMT+2, Goran Sliskovic wrote: Sorry, apparently logs

Re: Consumer not receiving messages after connection loss

2020-04-26 Thread Goran Sliskovic
Sorry, apparently logs get formatted to unreadable state. Attached this time. On Sunday, April 26, 2020, 01:35:38 PM GMT+2, Goran Sliskovic wrote: Hi, I'm tracking an issue in production system (data flow stops suddenly for hours). That seems to bi triggered by network e

Consumer not receiving messages after connection loss

2020-04-26 Thread Goran Sliskovic
Hi, I'm tracking an issue in production system (data flow stops suddenly for hours). That seems to bi triggered by network errors, however communication is not restored for hours event though networking is restored.I have managed to put test system in similar state. Symptoms: Client (java applic