[protobuf] ParseDelimited from unreliable comm channel

[David L]

When serializing a message with SerializeDelimitedToOstream, is the 
expectation that parsing the resulting serial stream will re-sync if some 
data is lost (eg, if the serialized stream is sent over a radio modem)? 
Some preliminary testing that I did showed that parsing data after some 
lost bytes failed continuously thereafter. Is that expected? I had hoped 
that a parsing from a delimited stream would have some re-sync logic so 
only the message with the missing bytes would be lost.

Thanks,

 David

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/protobuf/f05befea-8b86-4742-bf44-4cefb2733bd8n%40googlegroups.com.

[protobuf] Re: Question about pr #7470

['David L. Jones' via Protocol Buffers]

Hey Alex --

I just posted a reply on the PR. Sorry for the delay.

--dlj

On Wednesday, July 1, 2020 at 12:45:46 PM UTC-7 stackedsax wrote:

> Can I buymeacoffee.com for someone to put a CI label on this PR?  Or is 
> there somewhere else I should try and reach out to folks?
>
> Cheers,
>
> -Alex
>
> On Friday, June 26, 2020 at 3:16:29 PM UTC-7 stackedsax wrote:
>
>> Hey again,
>>
>> PR #7470  is up 
>> to 17 thumbs up and 4 hooray emojis.  Is this enough to get a CI label put 
>> on the PR?  We'd just like to see if it passes CI or if there is more work 
>> to be done at this point.
>>
>> Thanks very much,
>>
>> -Alex
>>
>> On Wednesday, June 24, 2020 at 3:09:12 PM UTC-7 stackedsax wrote:
>>
>>> Hi there,
>>>
>>> A member of my team recently submitted this PR:
>>>
>>>- https://github.com/protocolbuffers/protobuf/pull/7470
>>>
>>> We'd love it if someone could put the right labels on the PR so that we 
>>> can see the results of a CI run.  We'd also love some input on the PR 
>>> itself, of course.  Is this the correct forum to ask for these types of 
>>> things?
>>>
>>> Thanks very much,
>>>
>>> -Alex
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/protobuf/9040b167-3487-48f4-a997-20ac4e239d30n%40googlegroups.com.

[protobuf] Re: generate big endian protobuf code on little endian machine

['David L. Jones' via Protocol Buffers]

The sources generated by protoc do not vary by platform... protoc isn't a 
cross-compiler itself, it's just a source code generator.

The goal would be to get the protoc binary for your build environment, and 
use it to generate the sources to feed into your host/target platform.
In other words, you'll need to build the protobuf suite once to get 
`protoc` for the builder. Then, you'll use that `protoc` to build a second 
time for the target.

If you are using the autoconf-based build, you can pass `--with-protoc` to 
the configure script to point to pick the correct protoc binary to use.

I was able to do this using two build trees. The "-build" tree is x86-64, 
and targets the builder:

~/protobuf-2.6.0-build$ (./autogen.sh && ./configure && make -j32) >& 
make.log

Then, use that to build for ppc:

~/protobuf-2.6.0-build$ PROTOC=$PWD/src/protoc
~/protobuf-2.6.0-build$ cd ../protobuf-2.6.0
~/protobuf-2.6.0$ (./autogen.sh && ./configure --with-protoc=$PROTOC 
--host=powerpc-linux-gnu && make -j32) >& make.log
~/protobuf-2.6.0$ readelf -h src/.libs/libprotobuf.so.9.0.0
ELF Header:
  Magic:   7f 45 4c 46 01 02 01 00 00 00 00 00 00 00 00 00
  Class: ELF32
  Data:  2's complement, big endian
  Version:   1 (current)
  OS/ABI:UNIX - System V
  ABI Version:   0
  Type:  DYN (Shared object file)
  Machine:   PowerPC
(etc.)


(Of course, you can install the x86-64 build if you want, but I would 
recommend keeping it separate. Version 2.6.0 is really old, so you wouldn't 
want to use it for new projects.)


On Friday, April 24, 2020 at 6:17:49 AM UTC-7, kerry crouse wrote:
>
> If I run the PowerPC cross compiler and create a Protoc with it, that 
> Protoc will only run on a PowerPC. I am trying to set up the PowerPC 
> Protobuf files, the .pb.cc files, on the x86 build machine to cross compile 
> the image for the PowerPC machine. Is not the cross compiled Protoc meant 
> to run directly on the PowerPC machine?
>
> On Thursday, April 23, 2020 at 1:36:40 PM UTC-4, kerry crouse wrote:
>>
>> I have been through the online documentation, readmes, and protoc help 
>> and I'm missing how to do this: generate big endian protobuf code on little 
>> endian machine. Can someone point me to a howto? My build machine is a 
>> Linux Debian x86 machine. My target machine is a 32 bit PowerPC device. 
>> Thanks!
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/protobuf/e9937bb5-caab-4d3f-b244-e7efe631b86d%40googlegroups.com.

[protobuf] Re: Getting access to the Developers Guide pages

['David L. Jones' via Protocol Buffers]

If there's an error, please let us know...

On Friday, April 24, 2020 at 11:55:06 AM UTC-7, omár sweidán wrote:
>
> Hello, Team!
>
> I want to know if a normal human being is able to edit the Developers 
> Guide page, even with the supervision of the Google team?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/protobuf/820d0d2e-e91e-42c0-a356-d978f941cd6f%40googlegroups.com.

[protobuf] Re: Can custom option be set to repeated?

['David L. Jones' via Protocol Buffers]

It looks like this:

  types: ["1"]

should be:

  type: ["1"]

Does that change fix the error?

On Thursday, April 23, 2020 at 7:16:31 PM UTC-7, mustache_master wrote:
>
>
> Can custom option be set to repeated?
>
> sample code
> syntax = "proto3";
>
> package sample;
>
> import "google/protobuf/descriptor.proto";
>
> extend google.protobuf.FieldOptions {
>   Scope scope = 5;
> }
>
> message Scope {
>   repeated string type = 1;
> }
>
> message Sample {
>   uint64 id = 1 [
> (scope) = {
>   types: ["1"]
> }
>   ];
> }ここにコードを入力...
>
> sample repository
> https://github.com/activeguild/use-proto-fieldoption
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/protobuf/b13330cd-f3c0-47ed-a5aa-092b492edda4%40googlegroups.com.

[protobuf] Re: generate big endian protobuf code on little endian machine

['David L. Jones' via Protocol Buffers]

(I think Kerry's reply went to a different thread...)



On Thursday, April 23, 2020 at 5:11:05 PM UTC-7, Crouse, Kerry wrote:
>
> David,
>
> Thank you for your very prompt reply! I am required (for 
> compatibility) to use version 2.6.0. I'm not sure this makes much of a 
> difference. What I am attempting to ultimately accomplish is turn the 
> .proto files into code I can feed into the PowerPC cross compiler to 
> generate code as part of the static image to load onto the PowerPC. 
>

Yes, this is something that (conceptually) should work. (But there are more 
details...)
 

> Perhaps I'm not thinking about this process in the correct manner. If I 
> generate Protoc using that cross compiler, Protoc will have to be run on 
> the PowerPC? If so, my goal is defeated. I don't think, however, that 
> running the x86 Protoc will give me the correct data structures and code. 
> Perhaps I'm completely backwards. 
>
>
>
This is notionally correct. However, the compatibility and ABI surfaces 
within the protobuf suite are ... let's say, "complex." :-)

If the PPC32 target uses version 2.6.0, then you'll need to use the protoc 
from that version. The generated protobuf code is version-locked, in the 
sense that the generated code is effectively API- and ABI-locked to 
internal APIs in the (non-generated) protobuf runtime libraries. (This is 
somewhat similar to Qt and moc, if you are familiar with cross-building 
Qt-based projects... protobuf has a few more notional "API surfaces," but 
the idea is similar.)

If you have the sources that were used to build the ppc32 runtimes, you 
should be able to build for your host using those. Alternatively, you can 
grab a tarball from that release here:
https://github.com/protocolbuffers/protobuf/releases/tag/v2.6.0

I would also suggest upgrading from 2.6.0 if you can (but it sounds like 
this may be a hard constraint for you... in which case, you have my 
sympathies).

--dlj

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/protobuf/4de8622c-7ad4-4dd8-b3ed-0effa4702dd6%40googlegroups.com.

[protobuf] Re: Convert a Protobuf into array of JSONs

['David L. Jones' via Protocol Buffers]

The fundamental unit for protobufs (across all serialization and 
presentation formats, including JSON) is a message.

The semantics of the general JSON format do not map to protobuf semantics 
very well, but the proto3 language guide has a section on the JSON 
structures that can interoperate with protobufs:
https://developers.google.com/protocol-buffers/docs/proto3#json

The impetus for JSON interop is for servers which can understand either 
JSON- or wire-formatted requests, so a top-level list would pretty 
fundamentally break semantics. Given that pitfall, it's pretty unlikely 
that we would add support for anything other than a top-level message. The 
example you gave is how I would recommend representing "just" a list.

On Wednesday, April 22, 2020 at 5:18:24 PM UTC-7, Guilherme Costa wrote:
>
> I have a question regarding Protobufs, since I want to generate an array 
> of JSONs by using them, such as in this example
>
>> [
>>{
>> "var": ["myVariable1","myVariable2"], 
>> "key1": 123123,
>> "key2": 1122,
>> "key3": "abcd-0101"
>>},
>>   { 
>> "var": ["myVariable1"], 
>> "key1": 123124,
>> "key2": 1123,
>> "key3": "abcd-0102"
>>   },] 
>>
>>
> However, I just can get the example below:
>
>> {
>>  list: [
>>{
>> "var": ["myVariable1","myVariable2"], 
>> "key1": 123123,
>> "key2": 1122,
>> "key3": "abcd-0101"
>>},
>>   { 
>> "var": ["myVariable1"], 
>> "key1": 123124,
>> "key2": 1123,
>> "key3": "abcd-0102"
>>   },]  
>>
>> }
>>
>>
> It is possible to generate the first example (without the brackets and the 
> first key) using protobufs?
> Some of this issue are also discussed in: 
> https://stackoverflow.com/questions/4694/how-to-model-json-array-as-protobuf-definition,
>  
> they also had the same question, but I tried and did not work at all!
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/protobuf/8a095f8d-a572-41fb-8fb7-72de6208874e%40googlegroups.com.

[protobuf] Re: generate big endian protobuf code on little endian machine

['David L. Jones' via Protocol Buffers]

(I'm assuming this is for C++...)

The protobuf libraries (including the code generated by the protobuf 
compiler) should internally contain the necessary #ifdef/etc. guards to 
handle endian issues. For example:
https://github.com/protocolbuffers/protobuf/blob/master/src/google/protobuf/wire_format_lite.cc#L319-L369

In other words, it should Just Work (TM) with your cross-build toolchain.

If you use the Debian packages, make sure you grab the backports (otherwise 
you'll likely get a very old version).

If you want to build completely from source, both the autoconf and CMake 
builds should work as usual for cross-buliding. (If you also build and 
install natively, you can pass `--with-protoc` to configure.)

If you use a different build system, the changes are a tiny bit out of 
scope for our instructions. Again, the major host tool consideration is the 
`protoc` binary, which generates .pb.cc, etc. files.



On Thursday, April 23, 2020 at 10:36:40 AM UTC-7, kerry crouse wrote:
>
> I have been through the online documentation, readmes, and protoc help and 
> I'm missing how to do this: generate big endian protobuf code on little 
> endian machine. Can someone point me to a howto? My build machine is a 
> Linux Debian x86 machine. My target machine is a 32 bit PowerPC device. 
> Thanks!
>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to protobuf+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/protobuf/18efce8c-9bb6-4c11-b9b3-4b556bc4ad6f%40googlegroups.com.