Re: Java deserialization - any best practices for performances?

2009-07-18 Thread Alex Black
to the end of each message every time it sees a > length. > > Alek > > On Fri, Jul 17, 2009 at 8:13 PM, Alex Black wrote: > > > When I write out messages using C++ I'm careful to clear messages and > > re-use them, is there something equivalent on the java side

Java deserialization - any best practices for performances?

2009-07-17 Thread Alex Black
When I write out messages using C++ I'm careful to clear messages and re-use them, is there something equivalent on the java side when reading those same messages in? My code looks like: CodedInputStream stream = CodedInputStream.newInstance(inputStream); while ( !stream.isAtEnd() ) { MyMe

RE: Performance: Sending a message with ~150k items, approx 3.3mb, can I do better than 100ms?

2009-07-15 Thread Alex Black
yeah, I'll have it running twice as fast by the time you are back from your holiday. ;) From: Kenton Varda [mailto:ken...@google.com] Sent: Wednesday, July 15, 2009 7:48 PM To: Alex Black Cc: protobuf@googlegroup

RE: Performance: Sending a message with ~150k items, approx 3.3mb, can I do better than 100ms?

2009-07-15 Thread Alex Black
Thanks, yes performance seems really good, though I wouldn't mind seeing the java deserialization faster. From: Kenton Varda [mailto:ken...@google.com] Sent: Tuesday, July 14, 2009 8:06 PM To: Alex Black Cc: pro

RE: Performance: Sending a message with ~150k items, approx 3.3mb, can I do better than 100ms?

2009-07-14 Thread Alex Black
009 3:26 AM To: Alex Black Cc: Protocol Buffers Subject: Re: Performance: Sending a message with ~150k items, approx 3.3mb, can I do better than 100ms? OK. If your message composition (or parsing, on the receiving end) takes a lot of time, you might look into how much of that is due to m

RE: Performance: Sending a message with ~150k items, approx 3.3mb, can I do better than 100ms?

2009-07-13 Thread Alex Black
Kenton: I made a mistake with these numbers - pls ignore them - I'll revisit tomorrow. Thx. -Original Message- From: protobuf@googlegroups.com [mailto:proto...@googlegroups.com] On Behalf Of Alex Black Sent: Tuesday, July 14, 2009 2:05 AM To: Protocol Buffers Subject: Re: Perfor

Re: Performance: Sending a message with ~150k items, approx 3.3mb, can I do better than 100ms?

2009-07-13 Thread Alex Black
posed messages. > But this still doesn't tell us how much time is spent on network I/O vs. > protobuf serialization.  My guess is that once you factor that out, your > performance is pretty close to the benchmarks. > > On Mon, Jul 13, 2009 at 10:11 PM, Alex Black wrote: >

Re: Performance: Sending a message with ~150k items, approx 3.3mb, can I do better than 100ms?

2009-07-13 Thread Alex Black
If I comment out the actual serialization and sending of the message (so I am just composing messages, and clearing them each batch) then the 100ms drops to about 50ms. On Jul 14, 12:36 am, Alex Black wrote: > I'm sending a message with about ~150k repeated items in it, total > si

Performance: Sending a message with ~150k items, approx 3.3mb, can I do better than 100ms?

2009-07-13 Thread Alex Black
I'm sending a message with about ~150k repeated items in it, total size is about 3.3mb, and its taking me about 100ms to serialize it and send it out. Can I expect to do any better than this? What could I look into to improve this? - I have "option optimize_for = SPEED;" set in my proto file - I'

RE: Howto build protobuf not as library, but directly linking object files?

2009-07-09 Thread Alex Black
Hey Bart, I'm not sure I'm seeing the full details of your situation, But on windows I compiled a library libprotobuf.lib, and linked to it statically, meaning it gets built into the exe (or dll) you're building. (I built 2 actually, a release one and a debug one). - Alex -Original Message

RE: GoogleOnceType showing up as a memory leak

2009-07-06 Thread Alex Black
Thanks. From: Kenton Varda [mailto:ken...@google.com] Sent: Monday, July 06, 2009 1:32 PM To: Alex Black Cc: Protocol Buffers Subject: Re: GoogleOnceType showing up as a memory leak Yes, this is a known bug resulting from the fact that I developed the shutdown

GoogleOnceType showing up as a memory leak

2009-07-06 Thread Alex Black
I'm using a memory leak detection tool (ms crtdbg). It runs at the end of the program (after main has exited), and is reporting 7 40byte leaks, each of which corresponds to a new call in GoogleOnceType, they're each one of these two: GoogleOnceType::GoogleOnceType() { // internal_ may be non-N

RE: libprotobuf.lib, 29mb?

2009-07-02 Thread Alex Black
Thanks Kenton, thats reassuring. From: Kenton Varda [mailto:ken...@google.com] Sent: Thursday, July 02, 2009 8:56 PM To: Alex Black Cc: Protocol Buffers Subject: Re: libprotobuf.lib, 29mb? On Thu, Jul 2, 2009 at 5:33 PM, Alex Black wrote: I'

libprotobuf.lib, 29mb?

2009-07-02 Thread Alex Black
I'm on windows, using Visual Studio 2008 SP1, compiling a Release x64 build of libprotobuf. The lib file is 29,570KB big, is that expected? I'm using the provided visual studio projects, but I added the x64 configuration to them, perhaps I have some options set incorrectly or something. (The de

RE: Using CopyingOutputStreamAdaptor and CodedOutputStream

2009-06-29 Thread Alex Black
e 29, 2009 7:08 PM To: Alex Black Cc: Protocol Buffers Subject: Re: Using CopyingOutputStreamAdaptor and CodedOutputStream On Sun, Jun 28, 2009 at 7:30 AM, Alex Black wrote: Hi, I'm trying to use these to serialize messages to my own stream. I have a couple of

Using CopyingOutputStreamAdaptor and CodedOutputStream

2009-06-28 Thread Alex Black
Hi, I'm trying to use these to serialize messages to my own stream. I have a couple of questions: 1. If I use message.SerializeToCodedStream() do I still need to write the size of the message to the stream on my own first? Note, you can see the code I was previously using here: http://groups.g

RE: 'Streaming' messages (say over a socket)

2009-06-17 Thread Alex Black
From: Kenton Varda [mailto:ken...@google.com] Sent: Wednesday, June 17, 2009 4:42 PM To: Alex Black Cc: Christopher Smith; Protocol Buffers Subject: Re: 'Streaming' messages (say over a socket) Mostly looks fine. Note that a varint can be up to 5 bytes. You should probably just use Co

RE: 'Streaming' messages (say over a socket)

2009-06-16 Thread Alex Black
!stream.isAtEnd() ) { Foor.Bar.Builder builder = Foo.Bar.newBuilder(); stream.readMessage(builder, null); Foo.Bar message = builder.build(); } From: Christopher Smith [mailto:cbsm...@gmail.com] Sent: Monday, June 15, 2009 2:58 PM To

'Streaming' messages (say over a socket)

2009-06-14 Thread Alex Black
Is there a way to start sending a message before its fully composed? Say we have messages like this: message Entity { required int32 id = 1; required string name = 2; } message Entities { repeated Entity entity = 1; } If we're sending a message Entities with 1,000 Entity objects in