I agree with Louis-Marie: I would like the compiler file(s) included in the Python package.
Every protoc plugin written in Python will need to import the google.protobuf.compiler.plugin_pb2 module. This means that any user wishing to use that plugin (it isn't limited to plugin developers themselves) will be out of luck with a default Python protobuf install, which omits this file. There are a couple of workarounds (non-exhaustive list): 1) compiler plugin packages can copy generated code from plugin_pb2.py to their project/module and distribute it themselves. 2) Python plugins can tell end-users to manually compile and install plugin_pb2.py. 3) Don't use Python for the compiler plugin. So, these workarounds require me to feel dirty that I'm copying code instead of using a library verbatim (1), take a risk that the produced plugin_pb2.py file won't change with a future release (1), force all users of my plugin to jump through extra steps (2), and/or write my compiler plugin in a language in which I'm not as proficient (3). While it is true that the majority of installations will likely never utilize the compiler plugin module, for those that do, the barrier to entry can be increased and resolving the issue can be extremely frustrating, especially if compiler plugins don't have the documentation telling users how to resolve the import failure. Since it doesn't "just work," the experience tarnishes the image of the plugin and of protocol buffers itself. The only argument against inclusion so far is package bloat. On my system, plugin_pb2.py and plugin_pb2.pyc are 6247 and 4750 bytes, respectively. The current uncompressed egg weighs in around 500K. Are we really objecting over a 11k/2% size increase? That seems trivial to me, especially since the added file(s) won't conflict with any existing ones and thus there is little risk to adding it. While I would prefer that the Python installer be updated to include compiler.plugin_pb2, at the least could the documentation be updated to reflect a preferred workaround? (And I don't think forcing the use of a program packager like PyInstaller is the correct route either.) Greg From: Jason Hsueh Sent: Thursday, October 07, 2010 1:55 PM To: Louis-Marie Cc: Kenton Varda ; [email protected] Subject: Re: [protobuf] Python installation does not build plugin_pb2 As Kenton said, including plugin.proto would bloat the core library. Only people implementing proto compilers such as yourself need to use it. On the C++ side, you would typically just build a statically linked binary that has all of the plugin generated code linked in. It's not included in the C++ core library either. There are various mechanisms to package python programs, e.g. http://www.pyinstaller.org/, that would allow you to give users a functioning plugin program without requiring that they install the generated modules for plugin.proto. On Wed, Oct 6, 2010 at 12:42 AM, Louis-Marie <[email protected]> wrote: Thanks for your answer. To give you a little bit more information, here's what I'm trying to do. I want to deliver a tool using a custom protoc plugin implemented in python, so that end user can generate code from its own proto files. There would be nothing special to do before using this plugin, but since it depends on plugin_pb2, I need to find the plugin.proto file (using pkg-config), compile it, and put it in some appropriate location. Also, I can't put it in its "natural" parent python package (google.protobuf) which is already provided by protobuf installation. On the other side, the c++ code is generated (and I guess, compiled into the shared library), which makes it a little bit more straightforward to write c++ plugins. What would be the best way to achieve what I'm trying to do? If the python library size increase is not acceptable there, couldn't the plugin_pb2 file be generated in an independent location, so that one could still rely on it being present on any protobuf install? Thanks for your advices, Louis-Marie 2010/10/6 Kenton Varda <[email protected]>: > It's not generated because none of the python implementation actually uses > it. So, generating it and including it in the egg would just increase the > library size for everyone, when most people don't need it. > What makes you feel uncomfortable about generating it yourself? > > On Fri, Oct 1, 2010 at 6:01 AM, Louis-Marie <[email protected]> wrote: >> >> Hi all, >> >> It looks like python installation of protocol buffers does not >> generate the google.protobuf.compiler.plugin_pb2 python file, while >> google.protobuf.descriptor_pb2 is explicitly generated by >> protobuf/python/setup.py >> >> generate_proto("../src/google/protobuf/descriptor.proto") >> >> Shouldn't the plugin.proto file be compiled and installed the same >> way? Maybe I am missing something there, be I feel very uncomfortable >> recompiling it when I need to write a plugin. >> >> Thanks, >> >> Louis-Marie >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Protocol Buffers" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/protobuf?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/protobuf?hl=en. > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en. -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
