Thanks. Here is the source code snippet of EndtoEndLatency test:
for (i<- 0 until numMessages) { val begin = System.nanoTime producer.send(
new ProducerRecord[Array[Byte],Array[Byte]](topic, message)) val received =
iter.next val elapsed = System.nanoTime - begin // poor man's progress bar
if (i % 1000 == 0) println(i + "\t" + elapsed / 1000.0 / 1000.0) totalTime +
= elapsed latencies(i) = (elapsed / 1000 / 1000) }
iter.next reads a message using a consumer process. The
producer.send()function, is it blocking? I think the message should still
matters here, am I right?
In the source code, it is "val message = "hello there beautiful".getBytes"
is around 21 bytes.
I see that in this EndtoEndLatency test, no acks is involved, right?
Thanks.
Yuheng
On Wed, Jul 15, 2015 at 3:43 PM, Guozhang Wang <wangg...@gmail.com> wrote:
> The end-to-end latency record the transferring of a message from producer
> to broker, then to consumer.
>
> I cannot remember the details not but I think the EndtoEndLatency test
> record the latency as average, hence it is small.
>
> Guozhang
>
> On Wed, Jul 15, 2015 at 12:28 PM, Yuheng Du <yuheng.du.h...@gmail.com>
> wrote:
>
> > Guozhang,
> >
> > Thank you for explaining. I see that in ProducerPerformance call back
> > functions were used to get the latency metrics.
> > For the TestEndtoEndLatency, does message size matter? What this
> end-to-end
> > latency comprise of, besides transferring a package from source to
> > destination (typically around 0.2 ms for 100bytes message)?
> >
> > Thanks.
> >
> > Yuheng
> >
> > On Wed, Jul 15, 2015 at 3:01 PM, Guozhang Wang <wangg...@gmail.com>
> wrote:
> >
> > > Yuheng,
> > >
> > > Only TestEndtoEndLatency's number are end to end, for
> ProducerPerformance
> > > the latency is for the send-to-ack latency, which increases as batch
> size
> > > increases.
> > >
> > > Guozhang
> > >
> > > On Wed, Jul 15, 2015 at 11:36 AM, Yuheng Du <yuheng.du.h...@gmail.com>
> > > wrote:
> > >
> > > > In kafka performance tests https://gist.github.com/jkreps
> > > > /c7ddb4041ef62a900e6c
> > > >
> > > > The TestEndtoEndLatency results are typically around 2ms, while the
> > > > ProducerPerformance normally has "average latency"around several
> > hundres
> > > ms
> > > > when using batch size 8196.
> > > >
> > > > Are both results talking about end to end latency? and the
> > > > ProducerPerformance results' latency is just larger because of
> batching
> > > of
> > > > several messages?
> > > >
> > > > What is the message size of the TestEndtoEndLatency test uses?
> > > >
> > > > Thanks for any ideas.
> > > >
> > > > best,
> > > >
> > >
> > >
> > >
> > > --
> > > -- Guozhang
> > >
> >
>
>
>
> --
> -- Guozhang
>
