Re: [protobuf] Any vs. oneof in message wrappers
Hi Feng,
Thank you for the answer. Good point about redundant type information in
the header message.
Regards,
/David
On Monday, May 16, 2016 at 11:01:47 PM UTC+2, Feng Xiao wrote:
>
>
>
> On Mon, May 16, 2016 at 11:30 AM, 'David Koch' via Protocol Buffers <
> prot...@googlegroups.com > wrote:
>
>> Our platform generates/processes different types of proto3 messages with
>> Kafka as buffer. We currently store this type as part of the Kafka message
>> key so message consumers know which class to use for de-serialization.
>>
>> I want to add the type to the serialized message in order to not rely on
>> the existence of a separate key. I thought about creating a wrapper message
>> like this:
>>
>> message Header {
>> enum PayloadType {
>> TYPE_ONE = 1;
>> TYPE_TWO = 2;
>> }
>>
>> int64 tmst = 1;
>> string message_id = 2;
>> string producer_id = 3;
>> PayloadType payload_type = 4;
>> }
>>
>>
>> // Any payload.
>> message MessageWrapper {
>> Header header = 1;
>> Any payload = 2;
>> }
>>
>>
>> or alternatively:
>>
>> // List different payload options.
>> message MessageWrapper {
>> Header header = 1;
>> oneof payload {
>> PayloadTypeOne payload_type_one = 2;
>> PayloadTypeTwo payload_type_two = 3;
>> }
>> }
>>
>>
>> My question: Is one of the two alternatives to be preferred?
>>
> I don't think you need the header in either case. If you are using an Any
> message as payload, the payload already has the message type encoded in the
> type URL. If you are using an oneof, the receiver can also query which
> payload is set by using the oneof case API.
>
> Comparing the two (using Any and using oneof), using Any would give your
> users more flexibility to use their own message types, while using oneof
> will require all users to define their message types in a single place (all
> imported by the MessageWrapper type). The API for oneof is straightforward
> then Any though. If you need the flexibility to allow different components
> to define different payload types, Any is probably better. Otherwise I
> would favor the simpler oenof API.
>
>
>>
>> Using Any leads to more complex setting/getting in the application code
>> but it is more generic from the message definition point of view. Dropping
>> the Header.PayloadType field in the definition using Any would allow me
>> to not modify the MessageWrapper definition at all when adding new
>> payload types.
>>
>> Thanks.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Protocol Buffers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to protobuf+u...@googlegroups.com .
>> To post to this group, send email to prot...@googlegroups.com
>> .
>> Visit this group at https://groups.google.com/group/protobuf.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
Re: [protobuf] Any vs. oneof in message wrappers
On Mon, May 16, 2016 at 11:30 AM, 'David Koch' via Protocol Buffers <
protobuf@googlegroups.com> wrote:
> Our platform generates/processes different types of proto3 messages with
> Kafka as buffer. We currently store this type as part of the Kafka message
> key so message consumers know which class to use for de-serialization.
>
> I want to add the type to the serialized message in order to not rely on
> the existence of a separate key. I thought about creating a wrapper message
> like this:
>
> message Header {
> enum PayloadType {
> TYPE_ONE = 1;
> TYPE_TWO = 2;
> }
>
> int64 tmst = 1;
> string message_id = 2;
> string producer_id = 3;
> PayloadType payload_type = 4;
> }
>
>
> // Any payload.
> message MessageWrapper {
> Header header = 1;
> Any payload = 2;
> }
>
>
> or alternatively:
>
> // List different payload options.
> message MessageWrapper {
> Header header = 1;
> oneof payload {
> PayloadTypeOne payload_type_one = 2;
> PayloadTypeTwo payload_type_two = 3;
> }
> }
>
>
> My question: Is one of the two alternatives to be preferred?
>
I don't think you need the header in either case. If you are using an Any
message as payload, the payload already has the message type encoded in the
type URL. If you are using an oneof, the receiver can also query which
payload is set by using the oneof case API.
Comparing the two (using Any and using oneof), using Any would give your
users more flexibility to use their own message types, while using oneof
will require all users to define their message types in a single place (all
imported by the MessageWrapper type). The API for oneof is straightforward
then Any though. If you need the flexibility to allow different components
to define different payload types, Any is probably better. Otherwise I
would favor the simpler oenof API.
>
> Using Any leads to more complex setting/getting in the application code
> but it is more generic from the message definition point of view. Dropping
> the Header.PayloadType field in the definition using Any would allow me
> to not modify the MessageWrapper definition at all when adding new
> payload types.
>
> Thanks.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to protobuf+unsubscr...@googlegroups.com.
> To post to this group, send email to protobuf@googlegroups.com.
> Visit this group at https://groups.google.com/group/protobuf.
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
[protobuf] Any vs. oneof in message wrappers
Our platform generates/processes different types of proto3 messages with
Kafka as buffer. We currently store this type as part of the Kafka message
key so message consumers know which class to use for de-serialization.
I want to add the type to the serialized message in order to not rely on
the existence of a separate key. I thought about creating a wrapper message
like this:
message Header {
enum PayloadType {
TYPE_ONE = 1;
TYPE_TWO = 2;
}
int64 tmst = 1;
string message_id = 2;
string producer_id = 3;
PayloadType payload_type = 4;
}
// Any payload.
message MessageWrapper {
Header header = 1;
Any payload = 2;
}
or alternatively:
// List different payload options.
message MessageWrapper {
Header header = 1;
oneof payload {
PayloadTypeOne payload_type_one = 2;
PayloadTypeTwo payload_type_two = 3;
}
}
My question: Is one of the two alternatives to be preferred?
Using Any leads to more complex setting/getting in the application code but
it is more generic from the message definition point of view. Dropping the
Header.PayloadType field in the definition using Any would allow me to not
modify the MessageWrapper definition at all when adding new payload types.
Thanks.
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
