In this case producer.send() should not be blocking since it does not check
on the returned future (i.e. not waiting on the ack).
On Wed, Jul 15, 2015 at 12:57 PM, Yuheng Du <yuheng.du.h...@gmail.com>
wrote:
> Thanks. Here is the source code snippet of EndtoEndLatency test:
>
>
> for (i<- 0 until numMessages) { val begin = System.nanoTime producer.send(
> new ProducerRecord[Array[Byte],Array[Byte]](topic, message)) val received =
> iter.next val elapsed = System.nanoTime - begin // poor man's progress bar
> if (i % 1000 == 0) println(i + "\t" + elapsed / 1000.0 / 1000.0) totalTime
> +
> = elapsed latencies(i) = (elapsed / 1000 / 1000) }
>
>
>
>
>
>
>
>
>
>
>
>
> iter.next reads a message using a consumer process. The
> producer.send()function, is it blocking? I think the message should still
> matters here, am I right?
> In the source code, it is "val message = "hello there beautiful".getBytes"
> is around 21 bytes.
>
> I see that in this EndtoEndLatency test, no acks is involved, right?
>
> Thanks.
> Yuheng
>
> On Wed, Jul 15, 2015 at 3:43 PM, Guozhang Wang <wangg...@gmail.com> wrote:
>
> > The end-to-end latency record the transferring of a message from producer
> > to broker, then to consumer.
> >
> > I cannot remember the details not but I think the EndtoEndLatency test
> > record the latency as average, hence it is small.
> >
> > Guozhang
> >
> > On Wed, Jul 15, 2015 at 12:28 PM, Yuheng Du <yuheng.du.h...@gmail.com>
> > wrote:
> >
> > > Guozhang,
> > >
> > > Thank you for explaining. I see that in ProducerPerformance call back
> > > functions were used to get the latency metrics.
> > > For the TestEndtoEndLatency, does message size matter? What this
> > end-to-end
> > > latency comprise of, besides transferring a package from source to
> > > destination (typically around 0.2 ms for 100bytes message)?
> > >
> > > Thanks.
> > >
> > > Yuheng
> > >
> > > On Wed, Jul 15, 2015 at 3:01 PM, Guozhang Wang <wangg...@gmail.com>
> > wrote:
> > >
> > > > Yuheng,
> > > >
> > > > Only TestEndtoEndLatency's number are end to end, for
> > ProducerPerformance
> > > > the latency is for the send-to-ack latency, which increases as batch
> > size
> > > > increases.
> > > >
> > > > Guozhang
> > > >
> > > > On Wed, Jul 15, 2015 at 11:36 AM, Yuheng Du <
> yuheng.du.h...@gmail.com>
> > > > wrote:
> > > >
> > > > > In kafka performance tests https://gist.github.com/jkreps
> > > > > /c7ddb4041ef62a900e6c
> > > > >
> > > > > The TestEndtoEndLatency results are typically around 2ms, while the
> > > > > ProducerPerformance normally has "average latency"around several
> > > hundres
> > > > ms
> > > > > when using batch size 8196.
> > > > >
> > > > > Are both results talking about end to end latency? and the
> > > > > ProducerPerformance results' latency is just larger because of
> > batching
> > > > of
> > > > > several messages?
> > > > >
> > > > > What is the message size of the TestEndtoEndLatency test uses?
> > > > >
> > > > > Thanks for any ideas.
> > > > >
> > > > > best,
> > > > >
> > > >
> > > >
> > > >
> > > > --
> > > > -- Guozhang
> > > >
> > >
> >
> >
> >
> > --
> > -- Guozhang
> >
>
--
-- Guozhang
