Re: protoc and python imports
That works for me. Cheers, alan On Oct 28, 2:40 pm, "Kenton Varda" <[EMAIL PROTECTED]> wrote: > The model used by the protocol compiler is to assume that the .proto files > are located in a tree that parallels the Python package tree. We don't want > to get into relative imports because they can get complicated and > error-prone. > If you don't want to put your .proto files into a tree matching your Python > package tree, you could alternatively map them into such a tree virtually > like so: > > protoc --proto_path=mypkg=proto > > This maps the virtual directory "mypkg" to the physical directory "proto". > You would then have to write your imports like: > > import "mypkg/a.proto" > > You can also map individual files. > > If this is insufficient then I guess we need a way to specify the python > package explicitly in the .proto file, similar to the java_package option, > rather than just inferring it from the location of the .proto file. > > On Tue, Oct 28, 2008 at 7:49 AM, Alan Kligman <[EMAIL PROTECTED]>wrote: > > > > > I need the line to look like: > > > from .. import a_pb2.py > > > The reason this is a problem is because I'm building the protocol > > buffers into the middle of an existing project. The problem is that > > protoc assumes that the output is either at the top of the package, or > > that the related files are all in the same sub-package (which is > > rarely true). Python2.5 supports relative intra-package imports (like > > the one above). More details here: > >http://docs.python.org/tut/node8.html#SECTION00842. > > > I think this is probably worth fixing. The workaround is to do some > > post-processing on the output from protoc, which could get nasty. > > > On Oct 27, 5:44 pm, "Kenton Varda" <[EMAIL PROTECTED]> wrote: > > > I'm not sure I understand. What would you expect the import line > > importing > > > a_pb2 to look like? My understanding is that Python imports are > > absolute, > > > not relative to the importing file. > > > > On Sat, Oct 25, 2008 at 7:11 PM, Alan Kligman <[EMAIL PROTECTED] > > >wrote: > > > > > I'm having a problem with protoc where python imports are not done > > > > correctly. Here's the situation: > > > > > I have a directory structure like this: > > > > > proto/a.proto > > > > proto/a/b.proto > > > > proto/a/c.proto > > > > > a.proto provides some common definitions for both b.proto and c.proto. > > > > I build the output like this: > > > > > protoc --proto_path=. --python_out=../dist *.proto > > > > protoc --proto_path=. --python_out=../dist a/*.proto > > > > > assuming that proto is the current directory. Because a.proto is > > > > included in both b.proto and c.proto, they both import it like this: > > > > > import "a.proto";# relative to the current directory > > > > > After building the protobuf files with protoc, the resulting python > > > > output has import statements for a_pb2.py that look like: > > > > > import a_pb2.py > > > > > which is wrong, because a_pb2.py is actually in the directory one > > > > above b_pb2.py and c_pb2.py. Is there a way to get protoc to do this > > > > properly? Is it a bug? Python2.5 handles relative imports, but there > > > > is no nice way to do it in python2.4. > > > > > Thoughts? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Standard for RPC proto
Out of the several specifications, a problem I find is they all use
serialized messages as a byte string as part of the message. That's
inefficient and in the case of C++ involves triple-copying - from
socket buffer to kernel buffer to user buffer to parsed message. At
the very least that last copy should be avoided (from what I know the
first two can't be avoided at the current moment unless you have
sophisticated hardware).
Also the java version tries to do too much, adding the ability to
query what methods are available. I think that's best done as a
default RPC service on top of the RPC layer (as in, every rpc server
has a rpc method called "GetKnownMethods()" for example). Agreeing
with most people here, extra layers like security should be done in
the layer above or below (think RPC over https).
After stripping that all down, you can have a simple protocol such as:
message RpcRequestHeader {
required uint32 id = 1;
enum RequestType {
REQUEST = 0;
CANCEL = 1;
}
optional RequestType type = 2 [default = REQUEST];
optional string service_name = 3;
optional string method_name = 4;
}
where every request would send the request header message, followed by
the request message. Each message is sent as a length/serialized-
message pair, where length is a varint64. This is what I'm doing so
far as an experimentation in implementing C++ based rpc.
Frank
On Oct 28, 2:37 pm, "Paul P Komkoff Jr" <[EMAIL PROTECTED]> wrote:
> On Tue, Oct 28, 2008 at 9:07 PM, Pavel Shramov <[EMAIL PROTECTED]> wrote:
> > By the way one of the simpliest ways for RPC is to use HTTP transport.
> > It's have some limitations (e.g large overhead for small messages) but
> > also some benefits (e.g many libraries for performing HTTP calls and
> > simple proxying)
>
> Speaking of ugly hacks, one of the intermediate versions of rpc I used
> were, actually, protocol buffers over xmlrpc. messages were serialized
> and then wrapped into xmlrpc.Binary
>
> --
> This message represents the official view of the voices in my head.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Extensions becoming unknown fields
On Oct 29, 4:42 am, Jon Skeet <[EMAIL PROTECTED]> wrote: > You need to pass in an ExtensionRegistry in the parseFrom call. Have a > look at the unit tests for examples (search for "extension"). Thanks Jon! I think I have a better understanding of how things work now. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: reading one message at a time
thanks & regards On Oct 29, 2:40 am, "Kenton Varda" <[EMAIL PROTECTED]> wrote: > On Tue, Oct 28, 2008 at 6:35 AM, Moonstruck <[EMAIL PROTECTED]> wrote: > > > you mean we should write the file like this? > > (sizeof a message) | (serialized message) | (sizeof another message) > > | (another serialized message) || so on and so forth > > > while reading, we'd first read the message size, then read data with > > the specified size to a stream, after that I can get it parsed; > > > is that so ? > > Yes. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Extensions becoming unknown fields
On Oct 28, 11:23 pm, [EMAIL PROTECTED] wrote: > and on the receiver side: > // receive the message > byte[] buffer = receive(); > // parse the message from the array > Message msg = Message.parseFrom(buffer); > // check if message has the extension > if(msg.hasExtension(Extension.ExtensionName)) // this returns > false = ( > // blah blah blah > > Thanks for any insight! You need to pass in an ExtensionRegistry in the parseFrom call. Have a look at the unit tests for examples (search for "extension"). Jon --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
