Re: How do we usually handle Node disconnected issue for kafka producer

[Sachin Mittal]

Hi Kamal,
I understand this, however the connections are maintained by a vertx Kafka
client and I am not able to find a way to catch the closed connection and
reopen it.

Would setting the
connections.max.idle.ms = -1 or max int/long help here.

Thanks
Sachin


On Sat, 4 May 2024 at 11:06 PM, Kamal Chandraprakash <
kamal.chandraprak...@gmail.com> wrote:

> Hi Sachin,
>
> Why do you want to change the default settings? If the connection is open
> and unused,
> then it is fair to close the connection after the timeout and reopen it
> when required.
>
> On Fri, May 3, 2024 at 1:06 PM Sachin Mittal  wrote:
>
> > Hi,
> > I am using a Kafka producer java client by vert.x framework.
> >
> >
> https://vertx.io/docs/apidocs/io/vertx/kafka/client/producer/KafkaProducer.html
> >
> > There is a producer setting in kafka:
> > connections.max.idle.ms = 54
> >
> > So if there are no records to produce then after 9 minutes I get this in
> my
> > logs:
> > [kafka-producer-network-thread | RecordProducer]
> [NetworkClient.java:977] -
> > [Producer clientId=RecordProducer] Node -1 disconnected.
> >
> > What it looks like is the Kafka producer object I had created has lost
> its
> > connection due to this setting.
> >
> > What are my options to ensure that Kafka producer client does not close
> > idle connections or reconnects or keeps alive even when no records to
> > produce arrive for a long time?
> >
> > Thanks
> > Sachin
> >
>

Re: How do we usually handle Node disconnected issue for kafka producer

[Kamal Chandraprakash]

Hi Sachin,

Why do you want to change the default settings? If the connection is open
and unused,
then it is fair to close the connection after the timeout and reopen it
when required.

On Fri, May 3, 2024 at 1:06 PM Sachin Mittal  wrote:

> Hi,
> I am using a Kafka producer java client by vert.x framework.
>
> https://vertx.io/docs/apidocs/io/vertx/kafka/client/producer/KafkaProducer.html
>
> There is a producer setting in kafka:
> connections.max.idle.ms = 54
>
> So if there are no records to produce then after 9 minutes I get this in my
> logs:
> [kafka-producer-network-thread | RecordProducer] [NetworkClient.java:977] -
> [Producer clientId=RecordProducer] Node -1 disconnected.
>
> What it looks like is the Kafka producer object I had created has lost its
> connection due to this setting.
>
> What are my options to ensure that Kafka producer client does not close
> idle connections or reconnects or keeps alive even when no records to
> produce arrive for a long time?
>
> Thanks
> Sachin
>

Re: Apache zookeeper going down every 168 hours

[Haruki Okada]

I mean, just applying the PR linked to the ticket (
https://github.com/apache/kafka/pull/6517) locally. Or just cloning the
branch of this PR.
Note that the patch is not merged to upstream yet so you should try it at
your own risk.


Thanks,

2024年5月4日(土) 15:52 Yogeshkumar Annadurai :

> Hello,
>
> Thanks for your prompt response.
>
> How to apply patch for this? Could you please provide further more details?
>
> Regards
> Yogeshkumar A
>
> On Sat, 4 May 2024 at 9:33 AM, Haruki Okada  wrote:
>
> > Thanks for sharing logs.
> >
> > Kafka has a mechanism to mark log dir as "failed" when IOException
> happens
> > on I/O operations, and it will shut down when all log dirs have marked as
> > failed. (Kafka allows to set multiple log dirs for JBOD)
> > From server.log, we can see that the Kafka broker shut down because of
> > this.
> >
> > `[2024-04-28 11:50:46,466] ERROR Shutdown broker because all log dirs in
> > C:\kafka\data\logs have failed (kafka.log.LogManager)`
> >
> > And, seems the IOException caused this situation shows below message:
> > `The process cannot access the file because it is being used by another
> > process.`
> >
> > I can find some similar issues on windows on Kafka JIRA (e.g.
> > https://issues.apache.org/jira/browse/KAFKA-8172)
> > I never run Kafka on windows so I'm not sure if it works though, you may
> > try patch on this ticket.
> >
> > By the way, running Kafka on windows might be challenging (especially on
> > Production environment) so I recommend you to try on Linux (or on WSL at
> > least)
> >
> >
> > Thanks,
> >
> > 2024年5月4日(土) 10:20 Yogeshkumar Annadurai  >:
> >
> > > Hello,
> > >
> > > We see timeout error in server.log
> > > log files and properties files are attached for your reference
> > >
> > > regards
> > > Yogeshkumar A
> > >
> > > On Sat, May 4, 2024 at 5:27 AM Haruki Okada 
> wrote:
> > >
> > >> Hi.
> > >>
> > >> log.retention shouldn't be related to the phenomenon.
> > >> Sounds like we should understand the situation more precisely to
> answer.
> > >>
> > >> > apache zookeeper connection is going down automatically
> > >>
> > >> How did you confirm this? On ZooKeeper log?
> > >>
> > >> Also, did you see any logs on Kafka side? (on stdout or server.log,
> ...
> > >> etc)
> > >>
> > >>
> > >> Thanks,
> > >>
> > >> 2024年5月4日(土) 6:48 Yogeshkumar Annadurai <
> yogeshkumarannadu...@gmail.com
> > >:
> > >>
> > >> > Hello,
> > >> >
> > >> > We are using Apache kakfa in a development environment, where apache
> > >> > zookeeper connection is going down automatically every 168 hours. we
> > >> > observed that, log.retention.hours is set as 168 hours (7 days).
> > >> >
> > >> > I would like to understand the configuration for this kind of
> scenario
> > >> > (automatic kafka server down - It says, broker connection cannot be
> > >> > established)
> > >> >
> > >> >
> > >> > Regards
> > >> > Yogeshkumar A
> > >> >
> > >>
> > >>
> > >> --
> > >> 
> > >> Okada Haruki
> > >> ocadar...@gmail.com
> > >> 
> > >>
> > >
> >
> > --
> > 
> > Okada Haruki
> > ocadar...@gmail.com
> > 
> >
>


-- 

Okada Haruki
ocadar...@gmail.com