pache.org"
> *Date: *Friday, 18 January 2019 at 03:19
> *To: *"user@kudu.apache.org"
> *Subject: *Re: close Kudu client on timeout
>
>
>
> I did not want to overload my question with details but since you asked :)
> We use NiFi to consume data from 700+ topics
…
Cheers Josef
From: Boris Tyukin
Reply-To: "user@kudu.apache.org"
Date: Friday, 18 January 2019 at 03:19
To: "user@kudu.apache.org"
Subject: Re: close Kudu client on timeout
I did not want to overload my question with details but since you asked :) We
use NiFi to consume d
I did not want to overload my question with details but since you asked :)
We use NiFi to consume data from 700+ topics. Each message is a json
object, produced by GoldenGate.
NiFi has ability to call a custom script, written in Groovy, and we use
that feature to parse json out, apply some logic l
On Thu, Jan 17, 2019 at 1:46 PM Boris Tyukin wrote:
> Hi Alexey,
>
> it was "single idle Kudu Java client that created so many threads".
> 20,000 threads in a few days to be precise :) that code is running
> non-stop and basically listens to kafka topics, then for every batch from
> kafka, we cr
Hi Alexey,
it was "single idle Kudu Java client that created so many threads". 20,000
threads in a few days to be precise :) that code is running non-stop and
basically listens to kafka topics, then for every batch from kafka, we
create new kudu client instance, upsert data and close client.
the
I have a couple more questions:
- Did you get a jstack of the process? If so I assume you saw lots of
Netty threads like "New I/O boss", "New I/O worker", etc. because of having
many KuduClient instances. Is that right?
- Just curious: are your edge node clients in the same data center as Kudu
o
Hi Boris,
Kudu servers have a setting for connection inactivity period: idle
connections to the servers will be automatically closed after the specified
time (--rpc_default_keepalive_time_ms is the flag). So, from that
perspective idle clients is not a big concern to the Kudu server side.
As for
sorry it is Java
On Wed, Jan 16, 2019 at 3:32 PM Mike Percy wrote:
> Java or C++ / Python client?
>
> Mike
>
> Sent from my iPhone
>
> > On Jan 16, 2019, at 12:27 PM, Boris Tyukin
> wrote:
> >
> > Hi guys,
> >
> > is there a setting on Kudu server to close/clean-up inactive Kudu
> clients?
> >
Java or C++ / Python client?
Mike
Sent from my iPhone
> On Jan 16, 2019, at 12:27 PM, Boris Tyukin wrote:
>
> Hi guys,
>
> is there a setting on Kudu server to close/clean-up inactive Kudu clients?
>
> we just found some rogue code that did not close client on code completion
> and wonderi
Hi guys,
is there a setting on Kudu server to close/clean-up inactive Kudu clients?
we just found some rogue code that did not close client on code completion
and wondering if we can prevent this in future on Kudu server level rather
than relying on good developers.
That code caused 22,000 threa
10 matches
Mail list logo
