Re: Java deserialization - any best practices for performances?

2009-07-18 Thread Kenton Varda
On Fri, Jul 17, 2009 at 8:13 PM, Alex Black  wrote:

>
> When I write out messages using C++ I'm careful to clear messages and
> re-use them, is there something equivalent on the java side when
> reading those same messages in?


No.  Sorry.  This just doesn't fit at all with the Java library's design,
and even if it did, you cannot reuse Java String objects, which often
account for most of the memory usage.  However, memory allocation is cheaper
in Java than in C++, so there's less to gain from it.


>
>
> My code looks like:
>
> CodedInputStream stream = CodedInputStream.newInstance(inputStream);
>
> while ( !stream.isAtEnd() )
> {
> MyMessage.Builder builder = MyMessage.newBuilder();
> stream.readMessage(builder, null);
> MyMessage myMessage = builder.build();
>
> for ( MessageValue messageValue : myMessage.getValuesList() )
> {
>..
> }
> }
>
> I'm passing 150 messages each with 1000 items, so presumably memory is
> allocated 150 times for each of the messages...
>
> - Alex
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---



Re: Java deserialization - any best practices for performances?

2009-07-18 Thread Alex Black

Hi Alek, can you elaborate a bit on what you mean by lazy parsing?

I think what I want is to be able to *reuse* my objects, specifically
the instances of MyMessage, instead of allocating new ones each time
through the loop.  This is analagous to what my C++ code does when
writing out messages, it re-uses the same message object, clearing it
between uses.

On Jul 18, 12:25 am, Alek Storm  wrote:
> I think what you want is lazy parsing, which unfortunately isn't available
> yet.  You could always read bytes off the stream in chunks, or write your
> own CodedInputStream to skip to the end of each message every time it sees a
> length.
>
> Alek
>
> On Fri, Jul 17, 2009 at 8:13 PM, Alex Black  wrote:
>
> > When I write out messages using C++ I'm careful to clear messages and
> > re-use them, is there something equivalent on the java side when
> > reading those same messages in?
>
> > My code looks like:
>
> > CodedInputStream stream = CodedInputStream.newInstance(inputStream);
>
> > while ( !stream.isAtEnd() )
> > {
> >     MyMessage.Builder builder = MyMessage.newBuilder();
> >     stream.readMessage(builder, null);
> >     MyMessage myMessage = builder.build();
>
> >     for ( MessageValue messageValue : myMessage.getValuesList() )
> >     {
> >        ..
> >     }
> > }
>
> > I'm passing 150 messages each with 1000 items, so presumably memory is
> > allocated 150 times for each of the messages...
>
> > - Alex
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---