ParseFromFileDescriptor will read until there is no more input. This is consistent with behavior when parsing from other types of input streams, as the message wire format is not self delimiting. For file descriptors, that means you have to reach EOF. As such, ParseFromFileDescriptor is only useful when the whole file contains a single message.
One approach to writing multiple messages to the same stream is to use a length-delimited format: write the size of the message, then serialize the message itself. On the receiver side, you would set up a FileInputStream, and wrap a CodedInputStream around that. You can read the size of the messages from the stream and then use PushLimit and PopLimit to control how much data is read. On Wed, Aug 11, 2010 at 2:19 PM, nirajshr <niraj...@gmail.com> wrote: > Hi Kenton, > > I really appreciate your quick response. As you mentioned in your > response, the protobuf library does write all the data to the file > descriptor before returning from SerializeToFileDescriptor(). But, I > have to explicitly close the file descriptor (using close(fd) ) before > the data gets flushed. As a result, ParseFromFileDescriptor keeps > waiting for the input until the file descriptor has been closed. > > I tried breaking this process into two parts to see if the problem > persists: > In the first program, > 1) Firstly, I used the SerializeToString() function to write to a > string, > 2) Then, I wrote the string to a file descriptor using the > write(char*, buffer length) function. > > Then, I tested the following two ways to receive the data at the other > end of the socket: > In second program, > 1) I used ParseFromFileDescriptor() function to receive the data. The > problem persisted in this case. The first program had to call > close(fd) in order for the second program to receive the data. > > 2) Alternatively, I used read(char* str, buff_len) & > ParseFromString() functions to receive the data. In this case, the > program immediately received the data, without having to close the > file descriptor in the first program. > > I know that this issue might be outside the scope of google protocol > buffers. But, this contradictory results show that the > ParseFromFileDescriptor() function expects an explicit close of the > filedescriptor before it can retrieve all the data. I guess this makes > sense as ParseFromFileDescriptor() function has no way of knowing the > length of the data. Having to close the file descriptor as a way to > mark the end of the transmission of data is confusing. The > SerializeToFileDescriptor() function has to take care of this so that > ParseFromFileDescriptor() function has an idea about the size of data > being received. > > I am sorry to bother you with this long post, but, your insight will > be very helpful. > > On Aug 10, 5:40 pm, Kenton Varda <ken...@google.com> wrote: > > The protobuf library writes all data to the file descriptor before > returning > > from SerializeToFileDescriptor(). So, the problem is not in protobufs. > > Sorry. > > > > On Tue, Aug 10, 2010 at 10:58 AM, nirajshr <niraj...@gmail.com> wrote: > > > > > Particularly, I am using the SerializeToFileDescriptor() function to > > > send the serialized data over a socket interface. The problem is that > > > the data does not get sent before I explicitly close the socket using > > > close(fileDescriptor). I even tried letting the google protocol buffer > > > object go out of scope (hoping the destructor would be called, and > > > hence flushing of the data). I am sending large amounts of data > > > (around 1MB). I did not face this problem while sending smaller amount > > > of data. > > > > > Isn't there any way I can force flushing of data without destroying > > > the google protocol buffer object? > > > > > -- > > > You received this message because you are subscribed to the Google > Groups > > > "Protocol Buffers" group. > > > To post to this group, send email to proto...@googlegroups.com. > > > To unsubscribe from this group, send email to > > > protobuf+unsubscr...@googlegroups.com<protobuf%2bunsubscr...@googlegroups.com> > <protobuf%2bunsubscr...@googlegroups.com<protobuf%252bunsubscr...@googlegroups.com> > > > > > . > > > For more options, visit this group at > > >http://groups.google.com/group/protobuf?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To post to this group, send email to proto...@googlegroups.com. > To unsubscribe from this group, send email to > protobuf+unsubscr...@googlegroups.com<protobuf%2bunsubscr...@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/protobuf?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
