[protobuf] Increasing memory usage with Protobuf

[Øyvind Bakken]

Hi there

We have an issue with increasing memory usage of an application using 
Protobuf. The application has been tested with Valgrind in Ubuntu to find 
the reason for this increase. The results shown with Massif Visualiser is 
shown below:

<https://lh6.googleusercontent.com/-wmtsYjFOW_M/Ucrxa0J4E8I/AAAAAAAAABw/9itx_i5csOI/s1600/Memory+usage.png>

Here is a snapshot of the part of code highlighted by Valgrind:

<https://lh4.googleusercontent.com/-C0QgKsXxnJo/Ucr4wEPtvdI/AAAAAAAAACI/9cf44aekIsc/s1600/Snapshot%2360-Protobuf.png>

The figure is suggesting that the issue is related to Protobuf library 
functions, and further studies of the source code have strongened these 
suspicions. 

Here are the functions in ProtobufReceiver:

void ProtobufReceiver::runClient() 

{

        string receivedData;

        while(!terminationRequested)

        {

                string tmpData;

                *socket >> tmpData;

                receivedData += tmpData;


                while(parseContent(receivedData));

        }

}

bool ProtobufReceiver::parseContent(string &receivedData){
    ....
    ....
    ....

    if(dataSize >= packetSize)    {        DataList *dl = new DataList();       
 string dataString = receivedData.substr(packetSize-messageSize, messageSize);  
      dl->ParseFromString(dataString);
        
        ...
        ...
       
        if(dataSize > packetSize)        {           retval = true;           
receivedData = receivedData.substr(packetSize);        }        else        {   
        receivedData.clear();        }
    }

}

The DataList is the protobuf message, and even though this is not seen deleted 
here, it has been verified that every object is correctly deleted later in a 
getMessage() function.

The memory usage has been shown to increase up to values of around 1GB, before 
flattening out (increases at an almost negligible rate). At this stage, the 
program behaviour still seems normal.
No crashes have been observed because of this, but it would still be nice to 
figure out the reason for this behaviour. 

Also, a strange thing is that when running the program on a newer version of 
the HP EliteBook series, the issue seems to not appear. The program has been 
runned on a few other computers 
(mine is an EliteBook 8740w), on both Windows 7 and Ubuntu, with the same 
memory issues. I guess this could imply that the issue is HW dependant.

We are using Protobuf version 2.4.1.

Does anybody have a clue what's going on here? Any feedback would be 
appreciated.

Regards










-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/groups/opt_out.