On Wed, Jan 31, 2018 at 2:57 PM, Jonnny <irs...@gmail.com> wrote: > By 'Protobuf Python Library' you mean the Python Package called `protobuf > ? I think It's already included in my package as dependency. But Google > protobuf is just one example. Most people around here using Scala, and they > all have 'scalapb' imported into their proto files. The path is `*scalapb* > /*scalapb*.*proto`. *It's not something I can chance as it's a third > party proto, and it's not a Google protobuf so using the `protobuf` package > won't solve this issue. I can manually add it into my `/usr/local/include` > folder, but this mean the package is not self-contained and the user needs > to compile his own scalapb pb2 file, and manually add it to > `/usr/local/include` if he wants my package to work. > Is there not a Pypi package containing scalapb/scalapb_pb2.py already? How about ask the author of scalapb/scalapb.proto to provide such a python package?
> > It means again, I have to keep the `scalapb` folder set into the root of > my project, or play around with PYTHONPATH. I wasn't able to find online > Python projects/packages that uses Python complied proto code, so I was > thinking perhaps there is a way around it. But if there is not a better > way, and that's consider the 'standard', I'm guessing I can do one of the > three below: > > 1. On a normal project, `scalapb` should be sitting on the root of the > project (ie, the folder where I'm running the script from). > 2. Packages are more complex, because the 'root' includes things like > `setup.py` and `README.srt`, but not the actual package code. The package > sits in a directory inside the root Folder. For example in the Request > package (https://github.com/requests/requests), the actual code run from > within the `request` folder. So even if I'll keep the generated code > directory structure in the `requests` folder, it won't work necause the > relative path starts from the Root folder and not `requests` folder. Adding > the folder structure into the root of the project is messy (because you > need to defines those as package in `setup.py`, and perhaps even supply > __init__.py file). > 3. Playing with PYTHONPATH like so: > > > import sys > sys.path.append('./my_package_name/proto') > > > This code need to be called every-time the package is called (probably need > to sit inside the `__init__.py I'm guessing). > > > There is a 4th option, but one that I do not like much. In case I have > small amount of `pb2` files, I can manually edit the `pb2` imports by > adding my folder after the code is compiled. > > I would just go ahead with (1). What's so bad about having a scalapb directory in your root directory? If you are concerned with conflicts with other project that may also include scalapb, you can just create a separate python package for the scalapb/scalapb_pb2.py, deploy it to pypi, and share it with all other projects that depend on the same thing. > > > > > > On Wednesday, January 31, 2018 at 11:39:08 PM UTC+2, Feng Xiao wrote: >> >> >> >> On Wed, Jan 31, 2018 at 8:54 AM, Jonnny <irs...@gmail.com> wrote: >> >>> Hello everyone, >>> >>> I have a couple of micro-services, and each has couple of proto files. >>> It's common for one micro-service proto file to reference another service >>> proto file using import like so: >>> >>> import "google/protobuf/struct.proto"; >>> import "myService/v1/message.proto"; >>> import "myOtherService/v2/parser.proto"; >>> >>> We use a Gradle script that's responsible on creating the folder >>> structure and fetch all proto dependencies from different git repositories. >>> So in the example above the Gradle script will output will be a folder >>> named `google/protobuff`, `myServicev1/v1` and `myOtherService/v2` - each >>> of the folders will hold the proto files needed. Once the folder structure >>> is OK, I can use `protoc` to generate the Python code. >>> >>> Here's where the problem begins. The generated `_pb2` code import >>> modules based on the folder structure. So the import inside the `_pb2` file >>> will look like so: >>> >>> from google.protobuf import struct_pb2 as >>> google_dot_protobuf_dot_struct__pb2 >>> >>> >>> That's a relative path import, and it means that in order for my Python >>> program to work, the root folder where the Python script running from, >>> should have the `google/protobuff` folder for this import to work.Now >>> because I have multiple micro-services, I don't appreciated the fact I have >>> to add 3 new folders in my main Python project (it's even more painful when >>> you create a Pip package). What I was hoping to do is to open a directory >>> called `proto`, and add all that directory structure into it - but because >>> imports are relative, this won't work. I tried to to play with couple of >>> `protoc` options like `PROTO_PATH`, or run `protoc` from a parent >>> directory, so the `_pb2` files will get created with a different relative >>> path I can work with like so: >>> >>> from proto.google.protobuf import struct_pb2 as >>> google_dot_protobuf_dot_struct__pb2 >>> >>> But I wasn't able to do so. A dirty option would be adding the `proto` >>> folder into my `PYTHONPATH`. It's not pretty, and I'm guessing there is >>> probably a simple solution for that I'm missing with `protoc`. >>> >> You can probably change the path for your own message.proto/parser.proto, >> but for protobuf's google/protobuf/struct.proto (and any other protos under >> google/protobuf), it's the only option to import it as >> "google/protobuf/struct.proto". Nothing else would work. It just has to >> be imported as "google/protobuf/struct.proto" in any .proto file and has to >> be imported as "google.protobuf.struct_pb2" in any python file. It's part >> of protobuf public API and won't (and can't) be changed. >> >> Where is protobuf python library installed on your machine? If you have >> it installed, google.protobuf.struct_pb2 should be part of it and you >> shouldn't need to put google/protobuf/struct_pb2.py in your own project >> directory. >> >> >>> >>> Thanks! >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Protocol Buffers" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to protobuf+u...@googlegroups.com. >>> To post to this group, send email to prot...@googlegroups.com. >>> Visit this group at https://groups.google.com/group/protobuf. >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to protobuf+unsubscr...@googlegroups.com. > To post to this group, send email to protobuf@googlegroups.com. > Visit this group at https://groups.google.com/group/protobuf. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at https://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/d/optout.
