Kurian [ra...@signalfuse.com]
Received: Sunday, 21 Dec 2014, 12:25PM
To: users@kafka.apache.org [users@kafka.apache.org]
Subject: Re: Trying to figure out kafka latency issues
I'll take a look at the GC profile of the brokers Right now I
keep
a tab
To: users@kafka.apache.org [users@kafka.apache.org]
Subject: Re: Trying to figure out kafka latency issues
I'll take a look at the GC profile of the brokers Right now
I
keep
a tab
on
the CPU, Messages in, Bytes in, Bytes out, free memory (on
the
machine
,
Thunder
-Original Message-
From: Rajiv Kurian [ra...@signalfuse.com]
Received: Saturday, 20 Dec 2014, 10:24PM
To: users@kafka.apache.org [users@kafka.apache.org]
Subject: Re: Trying to figure out kafka latency issues
!
-Original Message-
From: Rajiv Kurian [ra...@signalfuse.com]
Received: Sunday, 21 Dec 2014, 12:25PM
To: users@kafka.apache.org [users@kafka.apache.org]
Subject: Re: Trying to figure out kafka latency issues
I'll take a look at the GC profile
[users@kafka.apache.org]
Subject: Re: Trying to figure out kafka latency issues
I'll take a look at the GC profile of the brokers Right
now
I
keep
a tab
on
the CPU, Messages in, Bytes in, Bytes out, free memory
(on
the
machine
to figure out kafka latency issues
I'll take a look at the GC profile of the brokers Right now I keep a tab
on
the CPU, Messages in, Bytes in, Bytes out, free memory (on the machine
not
JVM heap) free disk space on the broker. I'll need to take a look at the
JVM metrics too. What seemed
]
Subject: Re: Trying to figure out kafka latency issues
I'll take a look at the GC profile of the brokers Right now I keep a
tab
on
the CPU, Messages in, Bytes in, Bytes out, free memory (on the machine
not
JVM heap) free disk space on the broker. I'll need to take a look at
the
JVM
: Sunday, 21 Dec 2014, 12:25PM
To: users@kafka.apache.org [users@kafka.apache.org]
Subject: Re: Trying to figure out kafka latency issues
I'll take a look at the GC profile of the brokers Right now I keep a
tab
on
the CPU, Messages in, Bytes in, Bytes out, free memory (on the machine
luck!
-Original Message-
From: Rajiv Kurian [ra...@signalfuse.com]
Received: Sunday, 21 Dec 2014, 12:25PM
To: users@kafka.apache.org [users@kafka.apache.org]
Subject: Re: Trying to figure out kafka latency issues
I'll take a look at the GC profile of the brokers Right
, 12:25PM
To: users@kafka.apache.org [users@kafka.apache.org]
Subject: Re: Trying to figure out kafka latency issues
I'll take a look at the GC profile of the brokers Right now I keep
a tab
on
the CPU, Messages in, Bytes in, Bytes out, free memory (on the
machine
not
JVM heap
: Sunday, 21 Dec 2014, 12:25PM
To: users@kafka.apache.org [users@kafka.apache.org]
Subject: Re: Trying to figure out kafka latency issues
I'll take a look at the GC profile of the brokers Right now I keep
a tab
on
the CPU, Messages in, Bytes in, Bytes out, free memory
...@signalfuse.com]
Received: Saturday, 20 Dec 2014, 10:24PM
To: users@kafka.apache.org [users@kafka.apache.org]
Subject: Re: Trying to figure out kafka latency issues
Some more work tells me that the end to end latency numbers
vary
with
the
number of partitions I am
with the Java client so can't really help there.
Good luck!
-Original Message-
From: Rajiv Kurian [ra...@signalfuse.com]
Received: Sunday, 21 Dec 2014, 12:25PM
To: users@kafka.apache.org [users@kafka.apache.org]
Subject: Re: Trying to figure out kafka latency issues
I'll take a look at the GC
@kafka.apache.org [users@kafka.apache.org]
Subject: Re: Trying to figure out kafka latency issues
Some more work tells me that the end to end latency numbers vary with the
number of partitions I am writing to. I did an experiment, where based on a
run time flag I would dynamically select how many of the *1024
-Original Message-
From: Rajiv Kurian [ra...@signalfuse.com]
Received: Saturday, 20 Dec 2014, 10:24PM
To: users@kafka.apache.org [users@kafka.apache.org]
Subject: Re: Trying to figure out kafka latency issues
Some more work tells me that the end to end latency numbers vary with the
number
[users@kafka.apache.org]
Subject: Re: Trying to figure out kafka latency issues
I'll take a look at the GC profile of the brokers Right now I keep a tab on
the CPU, Messages in, Bytes in, Bytes out, free memory (on the machine not
JVM heap) free disk space on the broker. I'll need to take a look
That's a pretty detailed analysis, I'll be very interested to see what the root
cause is.
Have you had a look at the broker GC logs? The spike in cpu and the long tail
on the latency make me think garbage collection pauses.
I suppose the large number of partitions may have increased the memory
On Sat, Dec 20, 2014 at 3:49 PM, Rajiv Kurian ra...@signalfuse.com wrote:
I am trying to replace a Thrift peer to peer API with kafka for a
particular work flow. I am finding the 99th percentile latency to be
unacceptable at this time. This entire work load runs in an Amazon VPC. I'd
greatly
I've done some more measurements. I've also started measuring the latency
between when I ask my producer to send a message and when I get an
acknowledgement via the callback. Here is my code:
// This function is called on every producer once every 30 seconds.
public void addLagMarkers(final
Some more work tells me that the end to end latency numbers vary with the
number of partitions I am writing to. I did an experiment, where based on a
run time flag I would dynamically select how many of the *1024 partitions*
I write to. So say I decide I'll write to at most 256 partitions I mod
20 matches
Mail list logo
