Could you post some code showing the problem? I still suspect that it's probably an issue involving how the messages are framed, or maybe some kind of buffering issue. You could also try serializing the message to a string (e.g. with message.SerializeToString(&s)) and see if you can successfully parse the message again. That could help narrow down whether it's a problem with protobuf serialization or something else.
On Thu, Feb 8, 2018 at 9:29 PM, hce h <jupiter....@gmail.com> wrote: > Hi Adam, I built from the git source, but that did not work. After several > hours debugging, I found the problem but not quite sure the which library > causes the problem: > > I sent two protobuf messages consecutively in server side within the same > thread, it uses boost::asio::streambuf and std::ostream to call > message.SerializeToOstream(&os), then to call boost::asio::async_write to > send the message. For each sendMessage, a new shared pointer streambuf is > used, it should not cause buffer overlap. It works file when the message > smaller then 17 KB, when the message is larger, it corrupted the serialized > message, I suspect it is the protobuf serialization issue rather than boost > asio asyc write issue, but I don't know it enough, you might give me some > insight... > > > > On Wednesday, February 7, 2018 at 12:10:30 PM UTC+11, hce h wrote: >> >> Thanks Adam, good to know I don't need to change my protocol to set >> syntax = "proto2", I am building the code right now, will let you know if >> the new library fix the issue or not. >> >> >> >> On Wednesday, February 7, 2018 at 11:51:02 AM UTC+11, Adam Cozzette wrote: >>> >>> You can still upgrade to our most recent code on GitHub. As long as you >>> don't set syntax = "proto3" in your .proto files, your code should work >>> fine without needing any modifications. >>> >>> On Tue, Feb 6, 2018 at 4:25 PM, hce h <jupit...@gmail.com> wrote: >>> >>>> I am going to try to build protobuf from git source for armfs to verify >>>> if the old version protobuf caused the issue or not, the trouble is my >>>> protobuf code as all defined for protobuf v2, I don't want to run into the >>>> trouble with protobuf v3 source code, anyway I can defined to build v2 only >>>> during my build process? >>>> >>>> Thank you >>>> >>>> >>>> On Wednesday, February 7, 2018 at 10:44:14 AM UTC+11, hce h wrote: >>>>> >>>>> Thanks Adam, I had further investigation, the original post of error >>>>> was not correct, it was not the problems of length, the file length was >>>>> correct, but the file was corrupted, there were some junk characters like >>>>> "^@^@^@^L^B^@^B^P^D^H<80><9b>^F^P]^X<80><9b>^F" inserted to the >>>>> middle of protobuf bytes / string and chopped off the connect string after >>>>> serialisation. I don't think there are any issues of coding and >>>>> serialisation / deserialisation in general, I have used the same code for >>>>> years without any issues until when the files increased to 17 KB, it >>>>> corrupted. >>>>> >>>>> Thank you. >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Wednesday, February 7, 2018 at 4:13:22 AM UTC+11, Adam Cozzette >>>>> wrote: >>>>>> >>>>>> Could you post some example code showing the problem? I would guess >>>>>> that it's an issue with how your code is framing the message. Serialized >>>>>> protos do not indicate their own length, so when you parse a message you >>>>>> have to know how many bytes to expect. If you try to parse the wrong >>>>>> number >>>>>> of bytes, you can inadvertently parse the wrong thing or parsing might >>>>>> fail >>>>>> entirely. >>>>>> >>>>>> On Mon, Feb 5, 2018 at 5:04 PM, hce h <jupit...@gmail.com> wrote: >>>>>> >>>>>>> Hi, >>>>>>> >>>>>>> I am running protobuf v2 C++ library on Debian 8 ARM processor, I >>>>>>> can only send a text file in bytes field about 17797 bytes, if the file >>>>>>> size is more than 17797 bytes, the file will be corrupted and chopped >>>>>>> off >>>>>>> to 17797 bytes. Are there any workarounds to fix this issue? The >>>>>>> detailed >>>>>>> the package information: >>>>>>> >>>>>>> Package: libprotobuf9 >>>>>>> Source: protobuf >>>>>>> Version: 2.6.1-1 >>>>>>> Installed-Size: 866 >>>>>>> Maintainer: Robert Edmonds <edm...@debian.org> >>>>>>> Architecture: armhf >>>>>>> Depends: libc6 (>= 2.4), libgcc1 (>= 1:4.4.0), libstdc++6 (>= 4.9), >>>>>>> zlib1g (>= 1:1.1.4) >>>>>>> Size: 312100 >>>>>>> SHA256: 7e6a4ffa5c4ccea20a339425c5fe51c0954b6f222960ac87e26d9d7ae5b3 >>>>>>> e79b >>>>>>> SHA1: 807d3ed197797546f87d050cf02ea3d3c96fc4ee >>>>>>> MD5sum: 58bfed0950ad07ef1b06d82a42f24eae >>>>>>> Description: protocol buffers C++ library >>>>>>> Description-md5: c69f5af146ab9bc64e44b912e5af457e >>>>>>> Multi-Arch: same >>>>>>> Homepage: https://code.google.com/p/protobuf/ >>>>>>> Tag: implemented-in::c++, role::shared-lib >>>>>>> Section: libs >>>>>>> Priority: optional >>>>>>> Filename: pool/main/p/protobuf/libprotobuf9_2.6.1-1_armhf.deb >>>>>>> >>>>>>> Thank you >>>>>>> >>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Protocol Buffers" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to protobuf+u...@googlegroups.com. >>>>>>> To post to this group, send email to prot...@googlegroups.com. >>>>>>> Visit this group at https://groups.google.com/group/protobuf. >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> >>>>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Protocol Buffers" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to protobuf+u...@googlegroups.com. >>>> To post to this group, send email to prot...@googlegroups.com. >>>> Visit this group at https://groups.google.com/group/protobuf. >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to protobuf+unsubscr...@googlegroups.com. > To post to this group, send email to protobuf@googlegroups.com. > Visit this group at https://groups.google.com/group/protobuf. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at https://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/d/optout.
