Ok… so I was finally able to parse a dynamic message and it looks good. It
looks like it was just a user error on my part… after a little bit of
digging around, I found the right APIs to call. Now my code looks like:
Descriptors.FileDescriptor fieldOptionsDesc =
DescriptorProtos.FieldOptions.getDescriptor().getFile();
DescriptorProtos.FileDescriptorSet set =
DescriptorProtos.FileDescriptorSet.parseFrom(
PBnJ.class.getResourceAsStream("/messages.desc"));
Descriptors.Descriptor md =
Descriptors.FileDescriptor.buildFrom(set.getFile(0),
new
Descriptors.FileDescriptor[]{fieldOptionsDesc}).findMessageTypeByName("MessagePublish");
Messages.MessagePublish event = Messages.MessagePublish.newBuilder()
.setUuid(UUID.randomUUID().toString())
.setTimestamp(System.currentTimeMillis())
.setEmail("he...@example.com")
.setMessageAuthorUid(1)
.setMessageContent("hello world!")
.setMessageUid(1)
.build();
DynamicMessage dynamicMessage = DynamicMessage.parseFrom(md,
event.toByteArray());
// Parse worked!
for (Descriptors.FieldDescriptor fieldDescriptor : md.getFields()) {
Boolean extension =
fieldDescriptor.getOptions().getExtension(Messages.isPii);
System.out.println(fieldDescriptor.getName() + " isPii = " +
extension);
}
The output is:
uuid isPii = false
timestamp isPii = false
message_uid isPii = false
message_content isPii = false
message_author_uid isPii = false
email isPii = false
For some reason, this is incorrectly showing “isPii = false” for the email
field when it should be “isPii = true” (as it is in the .proto file). Any
thoughts on this?
Thanks again all!
On Friday, October 31, 2014 2:18:44 PM UTC-7, Ilia Mirkin wrote:
At no point are you specifying that you want to use the
> "MessagePublish" descriptor, so you must still be using the API
> incorrectly...
>
> On Fri, Oct 31, 2014 at 5:10 PM, Pradeep Gollakota <prade...@gmail.com
> <javascript:>> wrote:
> > Ok… awesome… I do have the .proto’s ahead of time, so I can have them
> > compiled to the .desc files and store those.
> >
> > Here’s my .proto file:
> >
> > package com.lithum.pbnj;
> >
> > import "google/protobuf/descriptor.proto";
> >
> > option java_package = "com.lithium.pbnj";
> >
> > extend google.protobuf.FieldOptions {
> > optional bool isPii = 50101;
> > }
> >
> > message MessagePublish {
> > required string uuid = 1;
> > required int64 timestamp = 2;
> > required int64 message_uid = 3;
> > required string message_content = 4;
> > required int64 message_author_uid = 5;
> > optional string email = 6 [(isPii) = true];
> > }
> >
> > I compiled this .proto file into a .desc file using the command you gave
> me.
> > I’m now trying to parse a DynamicMessage from the .desc file. Here’s the
> > code I have so far.
> >
> > DescriptorProtos.FileDescriptorSet descriptorSet =
> >
> DescriptorProtos.FileDescriptorSet.parseFrom(PBnJ.class.getResourceAsStream("/messages.desc"));
>
>
> > Descriptors.Descriptor desc =
> > descriptorSet.getFile(0).getDescriptorForType();
> >
> > Messages.MessagePublish event =
> Messages.MessagePublish.newBuilder()
> > .setUuid(UUID.randomUUID().toString())
> > .setTimestamp(System.currentTimeMillis())
> > .setEmail("he...@example.com <javascript:>")
> > .setMessageAuthorUid(1)
> > .setMessageContent("hello world!")
> > .setMessageUid(1)
> > .build();
> >
> > DynamicMessage dynamicMessage = DynamicMessage.parseFrom(desc,
> > event.toByteArray());
> >
> > The final line in the above code is throwing the following exception:
> >
> > Exception in thread "main"
> > com.google.protobuf.InvalidProtocolBufferException: Protocol message
> > end-group tag did not match expected tag.
> > at
> >
> com.google.protobuf.InvalidProtocolBufferException.invalidEndTag(InvalidProtocolBufferException.java:94)
>
>
> > at
> >
> com.google.protobuf.CodedInputStream.checkLastTagWas(CodedInputStream.java:174)
>
>
> > at
> >
> com.google.protobuf.CodedInputStream.readMessage(CodedInputStream.java:478)
> > at
> >
> com.google.protobuf.MessageReflection$BuilderAdapter.parseMessage(MessageReflection.java:482)
>
>
> > at
> >
> com.google.protobuf.MessageReflection.mergeFieldFrom(MessageReflection.java:780)
>
>
> > at
> >
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:336)
>
>
> > at
> >
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:318)
>
>
> > at
> >
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:229)
>
>
> > at
> >
> com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:180)
>
>
> > at
> >
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:419)
>
>
> > at
> >
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:229)
>
>
> > at
> >
> com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:171)
>
>
> > at
> >
> com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:412)
>
>
> > at
> com.google.protobuf.DynamicMessage.parseFrom(DynamicMessage.java:119)
> > at com.lithium.pbnj.PBnJ.main(PBnJ.java:36)
> > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
>
> > at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
>
> > at java.lang.reflect.Method.invoke(Method.java:483)
> > at
> com.intellij.rt.execution.application.AppMain.main(AppMain.java:134)
> >
> > On Friday, October 31, 2014 11:39:27 AM UTC-7, Oliver wrote:
> >>
> >> Basically, you can't do that in pure Java - the compiler is a C++
> >> binary, there is no Java version.
> >>
> >> Still, working with the output of --descriptor_set_out is probably the
> >> way to go here. If you have the .proto file ahead of time, you can
> >> pregenerate the descriptor output at build time and store it instead
> >> of the .proto file. If you don't have the .proto file ahead of time
> >> (and you can't redesign - this is not a good design) then you could
> >> run the compiler at runtime and read the output. Either way, now you
> >> have a parsed version of the message format as a protobuf-encoded
> >> message that you can read into your Java program and extract the
> >> Descriptors you need.
> >>
> >> If you're looking at a selfdescribing message format, then I'd go with
> >> using the parsed descriptors as your format description, not the text
> >> .proto file.
> >>
> >> Oliver
> >>
> >>
> >> On 31 October 2014 17:56, Pradeep Gollakota <prade...@gmail.com>
> wrote:
> >> > Hi Oliver,
> >> >
> >> > Thanks for the response! I guess my question wasn't quite clear. In
> my
> >> > java
> >> > code I have a string which contains the content of a .proto file.
> Given
> >> > this
> >> > string, how can I create an instance of a Descriptor class so I can
> do
> >> > DynamicMessage parsing.
> >> >
> >> > Thanks!
> >> > - Pradeep
> >> >
> >> > On Thursday, October 30, 2014 2:41:19 PM UTC-7, Oliver wrote:
> >> >>
> >> >> On 30 October 2014 02:53, Pradeep Gollakota <prade...@gmail.com>
> wrote:
> >> >>
> >> >> > I have a use case where I need to parse messages without having
> the
> >> >> > corresponding precompiled classes in Java. So the DynamicMessage
> >> >> > seems
> >> >> > to be
> >> >> > the correct fit, but I'm not sure how I can generate the
> >> >> > DescriptorSet
> >> >> > from
> >> >> > the ".proto" definition.
> >> >>
> >> >> protoc --descriptor_set_out=FILE ?
> >> >>
> >> >> Oliver
> >> >
> >> > --
> >> > You received this message because you are subscribed to the Google
> >> > Groups
> >> > "Protocol Buffers" group.
> >> > To unsubscribe from this group and stop receiving emails from it,
> send
> >> > an
> >> > email to protobuf+u...@googlegroups.com.
> >> > To post to this group, send email to prot...@googlegroups.com.
> >> > Visit this group at http://groups.google.com/group/protobuf.
> >> > For more options, visit https://groups.google.com/d/optout.
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups
> > "Protocol Buffers" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an
> > email to protobuf+u...@googlegroups.com <javascript:>.
> > To post to this group, send email to prot...@googlegroups.com
> <javascript:>.
> > Visit this group at http://groups.google.com/group/protobuf.
> > For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at http://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
