Re: Can't seem to make Extension parsing working...
Hi,
See below for more information.
On Fri, 2009-08-07 at 16:58 +0200, Brice Figureau wrote:
Hi,
Using Protobuf 2.1 with the following proto file:
message Message {
enum Type {
AUTH = 100;
}
required Type type = 1;
optional int32 id = 2;
extensions 100 to max;
}
message AuthRequest {
extend Message {
optional AuthRequest auth_request = 100;
}
required string name = 1;
required string pass = 2;
optional string nonce = 3;
optional string hmac = 4;
}
When setting name and pass to arbitray values, I get the following
bytes:
{8, 100, 16, 0, -94, 6, 40, 10, 4, 116, 101, 115, 116, 18, 32, 48, 57,
56, 70, 54, 66, 67, 68, 52, 54, 50, 49, 68, 51, 55, 51, 67, 65, 68, 69,
52, 69, 56, 51, 50, 54, 50, 55, 66, 52, 70, 54}.
If I print to text the corresponding message:
type: AUTH
id: 0
[game.req.AuthRequest.auth_request] {
name: test
pass: 098F6BCD4621D373CADE4E832627B4F6
}
Feeding back this byte array to Message whit this code:
byte[] out = {8, 100, 16, 0, -94, 6, 40, 10, 4, 116, 101, 115, 116, 18,
32, 48, 57, 56, 70, 54, 66, 67, 68, 52, 54, 50, 49, 68, 51, 55, 51, 67,
65, 68, 69, 52, 69, 56, 51, 50, 54, 50, 55, 66, 52, 70, 54};
Message m = Message.parseFrom(out);
assertTrue(m.hasExtension(AuthRequest.auth_request));
I don't get any extension number 100 (getExtension returns null,
hasExtension is false and when I debug the program, there's nothing in
the extension FieldSet).
It all looks like the field 100 is parsed as an Unknown field.
When I text print the message, I get:
type: AUTH
id: 0
100: \n\004test\022 098F6BCD4621D373CADE4E832627B4F6
So yes, the data is here, but it isn't parsed as a known or valid
extension.
So simplifying the code, I end-up with the following non passing unit
test:
AuthRequest req =
AuthRequest.newBuilder().setName(test).setPass(test).build();
Message src =
Message.newBuilder().setType(Type.AUTH).setExtension(AuthRequest.authRequest,
req).build();
Message m = Message.parseFrom(src.toByteArray());
assertTrue(m.hasExtension(AuthRequest.authRequest));
I used protoc to encode the correct textual representation, and I get
the exact same byte array as was produced by my java code, so
serialization seems to be OK.
Using protoc --decode from the same binary content, then it prints the
correct content, so the C version seems to work fine, which means only
the Java version has some issues with deserialization.
--
Brice Figureau
My Blog: http://www.masterzen.fr/
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: TextFormat and optional fields.
Protobuf 2.1.0 throws NPE when you try to set a field to null. So, this has
been long fixed.
On Fri, Aug 7, 2009 at 4:30 AM, Brice Figureau
brice...@daysofwonder.combrice%2...@daysofwonder.com
wrote:
Hi,
Oops, sorry, I was plain wrong.
It turns out the code was setting the nonce parameter to null.
Anyway, maybe it should be necessary to change the TextFormat to better
deal with null fields?
On Fri, 2009-08-07 at 12:07 +0200, Brice Figureau wrote:
Hi,
I have the following message, under Protobuf 2.0.3:
message Message {
required string name = 1;
optional string nonce = 2;
}
When I call toString() on this message it throws a NullPointerException:
Exception in thread main java.lang.NullPointerException
at com.google.protobuf.ByteString.copyFromUtf8(Unknown Source)
at com.google.protobuf.TextFormat.escapeText(Unknown Source)
at com.google.protobuf.TextFormat.printFieldValue(Unknown Source)
at com.google.protobuf.TextFormat.printSingleField(Unknown Source)
at com.google.protobuf.TextFormat.printField(Unknown Source)
at com.google.protobuf.TextFormat.print(Unknown Source)
at com.google.protobuf.TextFormat.print(Unknown Source)
at com.google.protobuf.TextFormat.printToString(Unknown Source)
at com.google.protobuf.AbstractMessage.toString(Unknown Source)
Which means it is impossible to have a null default value in a message,
we have to always have a default value (which is sometimes not
possible).
Is that the expected behaviour?
--
Brice Figureau
My Blog: http://www.masterzen.fr/
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Can't seem to make Extension parsing working...
Have you tried using the overload of parseFrom which takes an extension registry? Make sure you register all the extensions first, of course - call the appropriate registerAllExtensions static method, passing in the extension registry, before calling parseFrom(src.toByteArray(), registry). Jon --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: 2.2.0 release candidate
Kenton Varda wrote: What kind of issues? Unless the ThreadLocals are explicitly cleared, the classloader can not be unloaded/GCed and jars redeployed. The retained memory is not so much an issue for protobuf, but other people have not been so lucky and ended up with leaks of many megabytes of retained references per TL * the number of threads that have the TL. http://cs.oswego.edu/pipermail/concurrency-interest/2007-October/004435.html http://crazybob.org/2006/07/hard-core-java-threadlocal.html TLs are great in theory, but in practice only work well for call chains that can be properly coupled to their client lifecycles. -h --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: 'Search this group' isn't very helpful
I'm having zero results too, searching for DescriptorPool for instance. On Aug 4, 7:54 pm, Kenton Varda ken...@google.com wrote: Interesting. I'll file a bug against google groups. On Tue, Aug 4, 2009 at 3:25 PM, Adam yel...@gmail.com wrote: On Aug 4, 5:16 pm, Kenton Varda ken...@google.com wrote: What were you searching for? Are you sure there should be results? I originally searched for 'rpc' which I'm pretty sure in the past brought up over 3 pages of results. I've been archiving the emails locally from the group for over a year. To test the search I went back and found some keywords in subjects that I would think it should easily locate. For example: ECMAScript (2008-07-11) MacPorts (2008-07-13) Each time I get zero results. -Adam- Hide quoted text - - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: 'Search this group' isn't very helpful
The groups people are working on fixing this. On Fri, Aug 7, 2009 at 12:51 PM, Scott Stafford scott.staff...@gmail.comwrote: I'm having zero results too, searching for DescriptorPool for instance. On Aug 4, 7:54 pm, Kenton Varda ken...@google.com wrote: Interesting. I'll file a bug against google groups. On Tue, Aug 4, 2009 at 3:25 PM, Adam yel...@gmail.com wrote: On Aug 4, 5:16 pm, Kenton Varda ken...@google.com wrote: What were you searching for? Are you sure there should be results? I originally searched for 'rpc' which I'm pretty sure in the past brought up over 3 pages of results. I've been archiving the emails locally from the group for over a year. To test the search I went back and found some keywords in subjects that I would think it should easily locate. For example: ECMAScript (2008-07-11) MacPorts (2008-07-13) Each time I get zero results. -Adam- Hide quoted text - - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Python BuildFile functionality
Well, note that soon (hopefully in the next major release after 2.2.0 -- we actually have someone actively working on it now) the Python implementation will be able to wrap C++ classes for better performance. At that point it will probably be easy to accomplish what you want by handing off most of the work to the C++ library (and I can imagine that we'd provide a Python API for this). But if you need something that works without any C extensions, the more work will have to be done. On Fri, Aug 7, 2009 at 1:03 PM, Scott Stafford scott.staff...@gmail.comwrote: Hi - We need DescriptorPool::BuildFile-like functionality in the protobuf Python library. In other words, something that can take a FileDescriptorProto object and create a bunch of descriptor.Descriptor's that can be made into dynamic loaded classes. We haven't looked very closely at this problem yet, we're still trying to assess exact requirements. I am pretty sure nothing like it exists (correct?). If we built it ourselves, is it something that could be/would be of interest for the official protobuf distro? Does anyone have opinions on the form it should take? Scott --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Python BuildFile functionality
On Fri, Aug 7, 2009 at 1:54 PM, Scott Stafford scott.staff...@gmail.comwrote: Interesting... Are you using SWIG to wrap the C++? No, straight C extensions. (SWIG is just a code generator that generates C extensions.) Would you have to actually compile something to use a message in Python (god forbid)? You'd have to install a general protocol buffers extension, but you would not have to compile any C++ code specific to your message types (unless you want even more speed). Thinking about it a bit, I guess you'd just SWIG the static (non- generated) interfaces and the Python classes would all operate exclusively through reflection, such that any Python operation (or class definition) would just call various DescriptorPool and Reflection methods to perform all operations... ? If so, that changes a lot of things for us, but possibly 90% for the better... That sounds more or less accurate, though we're still not completely sure how it will end up looking. The first priority is to make the Python API faster (without necessarily adding any new functionality), but adding access to DescriptorPool makes some sense once we have that. Scott On Aug 7, 4:20 pm, Kenton Varda ken...@google.com wrote: Well, note that soon (hopefully in the next major release after 2.2.0 -- we actually have someone actively working on it now) the Python implementation will be able to wrap C++ classes for better performance. At that point it will probably be easy to accomplish what you want by handing off most of the work to the C++ library (and I can imagine that we'd provide a Python API for this). But if you need something that works without any C extensions, the more work will have to be done. On Fri, Aug 7, 2009 at 1:03 PM, Scott Stafford scott.staff...@gmail.com wrote: Hi - We need DescriptorPool::BuildFile-like functionality in the protobuf Python library. In other words, something that can take a FileDescriptorProto object and create a bunch of descriptor.Descriptor's that can be made into dynamic loaded classes. We haven't looked very closely at this problem yet, we're still trying to assess exact requirements. I am pretty sure nothing like it exists (correct?). If we built it ourselves, is it something that could be/would be of interest for the official protobuf distro? Does anyone have opinions on the form it should take? Scott- Hide quoted text - - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
