Re: [protobuf] How to handle options in compiler plugin?
Hi Kenton,
I am trying to do something similiar. I am writing a java plugin and I have
defined my custom service options and method options but inside my plugin I
see I only have access to FileDescriptorProto and not the FileDescriptor
itself. when I try to access these options as you just suggested from the
FileDecriptorProto they are null. Please recommend on what is the suggested
solution to make this work. Thanks
On Tuesday, April 20, 2010 at 5:53:37 PM UTC-7, Kenton Varda wrote:
>
> You do not need to compile your option declarations into protoc. You only
> need to compile them into your plugin binary. Files which use your options
> must import the .proto file that defines them. protoc will then parse the
> option definitions dynamically.
>
> For example, you could create a file php_options.proto:
>
> import "google/protobuf/descriptor.proto";
>
> package protobuf_php;
>
> message PhpFileOptions {
> optional bool skip_unknown = 1;
> optional string namespace = 2;
> }
>
> extend google.protobuf.FileOptions {
> optional PhpFieldOptions file_opt = 1004;
> }
>
> Now users could do:
>
> import "php_options.proto"
>
> option (protobuf_php.file_opt.skip_unknown) = true;
> option (protobuf_php.file_opt.namespace) = "My/Namespace";
>
> In your plugin, you can read these options like:
>
>
>
> file_descriptor->options().GetExtension(protobuf_php::file_opt).skip_unknown()
>
> Note that the number 1004 above is a number that I have assigned
> specifically to you! Usually people have to e-mail me to request these (as
> the comments in descriptor.proto say), but rather than wait for you to
> actually do so I went ahead and assigned you a number. Note that you can
> also define other kinds of options (e.g. field options); use extension
> number 1004 for these as well. Please let me know your project URL once it
> has one so that I can keep track of it.
>
> On Tue, Apr 20, 2010 at 3:08 PM, Andrew Brampton > wrote:
>
>> Thanks for your reply Kenton,
>>
>> I've been going round in circles with the documentation but completely
>> missed the section on custom options. Ok, so that has helped a little
>> but I'm still not sure how to apply it. So here is some more
>> information. I have the sample addressbook.proto file and at the top
>> it has a couple of options:
>>
>> option java_package = "com.example.tutorial";
>> option java_outer_classname = "AddressBookProtos";
>>
>> All I want to do is add another line like so:
>>
>> option php_skip_unknown = true;
>> or something like:
>> option php_namespace = "My/Namespace";
>>
>> Which, in the first case, would tell my compiler to skip over unknown
>> fields instead of storing them, and the second put the generated files
>> in a particular namespace.
>>
>> Now I see from the documentation I can extend FileOptions, but where
>> do I save the proto file with my new options in it? Would I need to
>> recompile the protoc compiler? At the moment I'm using protoc
>> installed from apt, and giving it the following command line:
>>
>> protoc --php_out . --plugin=protoc-gen-php=./protoc-gen-php my.proto
>>
>> Thanks for any guidance you can give me.
>> Andrew
>>
>> On Tue, Apr 20, 2010 at 10:38 PM, Kenton Varda > > wrote:
>> > It sounds like you are using custom options incorrectly, independent of
>> your
>> > plugin. That error message is being produced by protoc before it even
>> tries
>> > to call your plugin. Please double-check the docs on custom options.
>> If
>> > you can't figure out what's wrong, show us the .proto code you are
>> using to
>> > define and then use your option.
>> >
>> > On Tue, Apr 20, 2010 at 4:24 AM, Andrew Brampton > > wrote:
>> >>
>> >> Hi,
>> >> I've nearly finished writing a compiler plugin for generating PHP
>> >> code. I now wanted to support some additional options. However, I'm
>> >> not sure how to begin. When I have a custom option in my proto file,
>> >> protoc seems to die with this error:
>> >>
>> >> my.proto:2:8: Option "php_skip_unknown" unknown.
>> >>
>> >> This seems to happen before CodeGenerator's generate method is called.
>> >> So I'm not sure how I handle this.
>> >>
>> >> Also, are there some test proto files, with binary data, which I can
>> >> use to fully validate my compiler?
>> >>
>> >> thanks
>> >> Andrew
>> >>
>> >> --
>> >> You received this message because you are subscribed to the Google
>> Groups
>> >> "Protocol Buffers" group.
>> >> To post to this group, send email to prot...@googlegroups.com
>> .
>> >> To unsubscribe from this group, send email to
>> >> protobuf+u...@googlegroups.com .
>> >> For more options, visit this group at
>> >> http://groups.google.com/group/protobuf?hl=en.
>> >>
>> >
>> >
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Protocol Buffers" group.
> To post to this group, send email to prot...@googlegroups.com
> .
> To unsubscribe from this group, send email to
> protobuf+u...@googlegroups.com
Re: [protobuf] How to generate FileDescriptor Object which has custom options?
Hi Josh,
I tried your suggested approach, the result is still the same.
helloD.findServiceByName("HelloWorldService").findMethodByName("sayHello").getOptions()
still populates them as unknown fields instead of matching them up to its
descriptors from Annotations FileDescriptor.
Ankit
On Monday, July 23, 2018 at 11:52:56 AM UTC-7, Josh Humphries wrote:
>
> On Mon, Jul 23, 2018 at 12:56 PM Ankit Patel > wrote:
>
>> Hi Josh,
>>
>> Thanks for responding. I do not have access to the codegen classes from
>> the proto compiler. My requirement is to use the proto files and create
>> descriptors from it and rely on that. I have the proto file where the
>> extension is defined and I am using that to create a registry and then feed
>> it to the File descriptor of my proto as shown in the above code. But It is
>> still not reading that registry and updating itself. What am I doing wrong?
>>
>
> Hi, Ankit,
> I apologize that my first response was incorrect. I clearly didn't ingest
> your entire code example. After reviewing more closely, I'll admit, it
> looks like you are doing everything correctly. Hopefully someone else on
> the list can spot the issue.
>
> In the meantime, I'd probably use a debugger to step through it,
> particularly the call to internalUpdateFileDescriptor, to see what's
> going on internally. An alternative you might try is just parsing
> "Hello.pb" with your initialized extension registry, instead of parsing it
> up front without and then having to call the internal update method later.
> (Not that it should behave differently -- but it could be interesting to
> verify whether or not it behaves differently.)
>
>
>>
>> Thanks,
>> Ankit
>>
>> On Friday, July 20, 2018 at 5:56:35 PM UTC-7, Josh Humphries wrote:
>>>
>>> The issue is that when you access the descriptor that way, it does not
>>> know about the custom options.
>>>
>>> You need to create an ExtensionRegistry. You'll also need the Java code
>>> produced from the annotations.proto file: that generated code will have a
>>> way to access the extensions (to add them to your registry), and IIRC even
>>> includes a way to register all extensions in the file.
>>>
>>> Then you'll have to re-parse the descriptor proto with the
>>> ExtensionRegistry -- that way, the field will be recognized and parsed
>>> correctly instead of considered an unknown field. IIRC (I've been doing Go
>>> for a while, so my recollection of the Java protobuf runtime might be
>>> fading a little), to re-parse them you have to serialize it to bytes (in
>>> this case, the method descriptor proto) and then de-serialize again. The
>>> resulting de-serialized message will now have the options in a queryable
>>> form.
>>>
>>>
>>> *Josh Humphries*
>>> jh...@bluegosling.com
>>>
>>>
>>>
>>> On Fri, Jul 20, 2018 at 8:30 PM Ankit Patel
>>> wrote:
>>>
>>>> Hi All,
>>>>
>>>> I have a simple .proto file
>>>>
>>>> syntax = "proto3";
>>>>
>>>> package helloworld.v1;
>>>>
>>>> option java_package = "helloworld.v1";
>>>> option java_multiple_files = true;
>>>>
>>>> import "google/api/annotations.proto";
>>>>
>>>> service HelloWorldService {
>>>> rpc sayHello (SayHelloRequest) returns (SayHelloResponse) {
>>>> option (google.api.http) = {
>>>> post: "/v1/example/echo"
>>>> body: "*"
>>>> };
>>>> }
>>>> }
>>>>
>>>> message SayHelloRequest {
>>>> string favourite_thing = 1;
>>>> }
>>>>
>>>> message SayHelloResponse {
>>>> string reply = 1;
>>>> }
>>>>
>>>> I am generating a proto descriptor set from it using the proto compiler.
>>>>
>>>> After that I am trying to read the proto descriptor programatically in
>>>> the java code like this
>>>> FileDescriptorSet Hello = FileDescriptorSet.parseFrom(new
>>>> FileInputStream(new File("D:\\temp\\descriptor\\Hello.pb")));
>>>> FileDescriptorSet Annotations = FileDescriptorSet.parseFrom(new
>>>> FileInputStream(new File("D:\\temp\\descriptor\\annotations.pb")));
>>>> FileDescriptorSet Http = FileDescriptorSet.p
Re: [protobuf] How to generate FileDescriptor Object which has custom options?
Hi Josh,
Thanks for responding. I do not have access to the codegen classes from the
proto compiler. My requirement is to use the proto files and create
descriptors from it and rely on that. I have the proto file where the
extension is defined and I am using that to create a registry and then feed
it to the File descriptor of my proto as shown in the above code. But It is
still not reading that registry and updating itself. What am I doing wrong?
Thanks,
Ankit
On Friday, July 20, 2018 at 5:56:35 PM UTC-7, Josh Humphries wrote:
>
> The issue is that when you access the descriptor that way, it does not
> know about the custom options.
>
> You need to create an ExtensionRegistry. You'll also need the Java code
> produced from the annotations.proto file: that generated code will have a
> way to access the extensions (to add them to your registry), and IIRC even
> includes a way to register all extensions in the file.
>
> Then you'll have to re-parse the descriptor proto with the
> ExtensionRegistry -- that way, the field will be recognized and parsed
> correctly instead of considered an unknown field. IIRC (I've been doing Go
> for a while, so my recollection of the Java protobuf runtime might be
> fading a little), to re-parse them you have to serialize it to bytes (in
> this case, the method descriptor proto) and then de-serialize again. The
> resulting de-serialized message will now have the options in a queryable
> form.
>
>
> *Josh Humphries*
> jh...@bluegosling.com
>
>
>
> On Fri, Jul 20, 2018 at 8:30 PM Ankit Patel > wrote:
>
>> Hi All,
>>
>> I have a simple .proto file
>>
>> syntax = "proto3";
>>
>> package helloworld.v1;
>>
>> option java_package = "helloworld.v1";
>> option java_multiple_files = true;
>>
>> import "google/api/annotations.proto";
>>
>> service HelloWorldService {
>> rpc sayHello (SayHelloRequest) returns (SayHelloResponse) {
>> option (google.api.http) = {
>> post: "/v1/example/echo"
>> body: "*"
>> };
>> }
>> }
>>
>> message SayHelloRequest {
>> string favourite_thing = 1;
>> }
>>
>> message SayHelloResponse {
>> string reply = 1;
>> }
>>
>> I am generating a proto descriptor set from it using the proto compiler.
>>
>> After that I am trying to read the proto descriptor programatically in
>> the java code like this
>> FileDescriptorSet Hello = FileDescriptorSet.parseFrom(new
>> FileInputStream(new File("D:\\temp\\descriptor\\Hello.pb")));
>> FileDescriptorSet Annotations = FileDescriptorSet.parseFrom(new
>> FileInputStream(new File("D:\\temp\\descriptor\\annotations.pb")));
>> FileDescriptorSet Http = FileDescriptorSet.parseFrom(new
>> FileInputStream(new File("D:\\temp\\descriptor\\http.pb")));
>> FileDescriptorSet Descriptor = FileDescriptorSet.parseFrom(new
>> FileInputStream(new File("D:\\temp\\descriptor\\descriptor.pb")));
>>
>> FileDescriptor httpD = FileDescriptor.buildFrom(Http.getFile(0), new
>> FileDescriptor[] {}, true);
>> FileDescriptor descriptorD =
>> FileDescriptor.buildFrom(Descriptor.getFile(0), new FileDescriptor[] {},
>> true);
>> FileDescriptor annotationsD =
>> FileDescriptor.buildFrom(Annotations.getFile(0), new FileDescriptor[]
>> {httpD, descriptorD});
>>
>> ExtensionRegistry registry = ExtensionRegistry.newInstance();
>> FieldDescriptor httpFD = annotationsD.findExtensionByName("http");
>> Message httpRuleM =
>> DynamicMessage.newBuilder(httpD.findMessageTypeByName("HttpRule")).build();
>> registry.add(httpFD, httpRuleM);
>>
>> FileDescriptor helloD = FileDescriptor.buildFrom(Hello.getFile(0), new
>> FileDescriptor[] {annotationsD});
>> FileDescriptor.internalUpdateFileDescriptor(helloD, registry);
>>
>> helloD.findServiceByName("HelloWorldService").findMethodByName("sayHello").getOptions();
>>
>> The api call in the last line puts the "http" in the unknown field, and
>> the reason is because its a custom option and not registered. But as you
>> can see I am doing that.
>>
>> Can someone please guide me on what am I doing wrong?
>>
>> Thanks!
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Protocol Buffers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to protobuf+u...@googlegroups.com .
>> To post to this group, send email to prot...@googlegroups.com
>> .
>> Visit this group at https://groups.google.com/group/protobuf.
>> For more options, visit https://groups.google.com/d/optout.
>>
>
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
[protobuf] How to generate FileDescriptor Object which has custom options?
Hi All,
I have a simple .proto file
syntax = "proto3";
package helloworld.v1;
option java_package = "helloworld.v1";
option java_multiple_files = true;
import "google/api/annotations.proto";
service HelloWorldService {
rpc sayHello (SayHelloRequest) returns (SayHelloResponse) {
option (google.api.http) = {
post: "/v1/example/echo"
body: "*"
};
}
}
message SayHelloRequest {
string favourite_thing = 1;
}
message SayHelloResponse {
string reply = 1;
}
I am generating a proto descriptor set from it using the proto compiler.
After that I am trying to read the proto descriptor programatically in the
java code like this
FileDescriptorSet Hello = FileDescriptorSet.parseFrom(new
FileInputStream(new File("D:\\temp\\descriptor\\Hello.pb")));
FileDescriptorSet Annotations = FileDescriptorSet.parseFrom(new
FileInputStream(new File("D:\\temp\\descriptor\\annotations.pb")));
FileDescriptorSet Http = FileDescriptorSet.parseFrom(new
FileInputStream(new File("D:\\temp\\descriptor\\http.pb")));
FileDescriptorSet Descriptor = FileDescriptorSet.parseFrom(new
FileInputStream(new File("D:\\temp\\descriptor\\descriptor.pb")));
FileDescriptor httpD = FileDescriptor.buildFrom(Http.getFile(0), new
FileDescriptor[] {}, true);
FileDescriptor descriptorD =
FileDescriptor.buildFrom(Descriptor.getFile(0), new FileDescriptor[] {},
true);
FileDescriptor annotationsD =
FileDescriptor.buildFrom(Annotations.getFile(0), new FileDescriptor[]
{httpD, descriptorD});
ExtensionRegistry registry = ExtensionRegistry.newInstance();
FieldDescriptor httpFD = annotationsD.findExtensionByName("http");
Message httpRuleM =
DynamicMessage.newBuilder(httpD.findMessageTypeByName("HttpRule")).build();
registry.add(httpFD, httpRuleM);
FileDescriptor helloD = FileDescriptor.buildFrom(Hello.getFile(0), new
FileDescriptor[] {annotationsD});
FileDescriptor.internalUpdateFileDescriptor(helloD, registry);
helloD.findServiceByName("HelloWorldService").findMethodByName("sayHello").getOptions();
The api call in the last line puts the "http" in the unknown field, and the
reason is because its a custom option and not registered. But as you can
see I am doing that.
Can someone please guide me on what am I doing wrong?
Thanks!
--
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to protobuf+unsubscr...@googlegroups.com.
To post to this group, send email to protobuf@googlegroups.com.
Visit this group at https://groups.google.com/group/protobuf.
For more options, visit https://groups.google.com/d/optout.
