@googlegroups.com
Subject: Re: Performance: Sending a message with ~150k items,
approx 3.3mb, can I do better than 100ms?
So, 172 MB/s for composition + serialization. Sounds about
right.
On Tue, Jul 14, 2009 at 10:46 AM, Alex Black a...@alexblack.ca
Varda [mailto:ken...@google.com]
*Sent:* Tuesday, July 14, 2009 8:06 PM
*To:* Alex Black
*Cc:* protobuf@googlegroups.com
*Subject:* Re: Performance: Sending a message with ~150k items, approx
3.3mb, can I do better than 100ms?
So, 172 MB/s for composition + serialization. Sounds about right
ok, I took I/O out of the picture by serializing each message into a
pre-allocated buffer, and this time I did a more through measurement.
Benchmark 1: Complete scenario
- average time 262ms (100 runs)
Benchmark 2: Same as # 1 but no IO
- average time 250ms (100 runs)
Benchmark 3: Same as 2
: Sending a message with ~150k items, approx 3.3mb, can
I do better than 100ms?
ok, I took I/O out of the picture by serializing each message into a
pre-allocated buffer, and this time I did a more through measurement.
Benchmark 1: Complete scenario
- average time 262ms (100 runs)
Benchmark 2
revisit tomorrow.
Thx.
-Original Message-
From: protobuf@googlegroups.com [mailto:proto...@googlegroups.com] On
Behalf Of Alex Black
Sent: Tuesday, July 14, 2009 2:05 AM
To: Protocol Buffers
Subject: Re: Performance: Sending a message with ~150k items, approx 3.3mb,
can I do better
:26 AM
To: Alex Black
Cc: Protocol Buffers
Subject: Re: Performance: Sending a message with ~150k items, approx
3.3mb, can I do better than 100ms?
OK. If your message composition (or parsing, on the receiving end)
takes a lot of time, you might look into how much of that is due to
memory
Speed varies a lot depending on the precise content. My benchmarks
generally show serialization performance somewhere between 100 MB/s and 1
GB/s, whereas you're seeing 33MB/s, but my benchmarks do not include any
kind of I/O. Maybe you could separate the serialization step from the I/O
(by
If I comment out the actual serialization and sending of the message
(so I am just composing messages, and clearing them each batch) then
the 100ms drops to about 50ms.
On Jul 14, 12:36 am, Alex Black a...@alexblack.ca wrote:
I'm sending a message with about ~150k repeated items in it, total
Oh, I didn't even know you were including composition in there. My
benchmarks are only for serialization of already-composed messages.
But this still doesn't tell us how much time is spent on network I/O vs.
protobuf serialization. My guess is that once you factor that out, your
performance is
