Re: Deserializing a message for which I don't have the .proto (in Java)
On Wed, Jun 24, 2009 at 11:55 AM, Toph wrote: > So if I know that a given byte blob is actually a packed array, how > can I decode it without having a precompiled proto definition? You would have to not only know that the blob is a packed array, but also the wire type of the elements in the array (varint, 32-bit, or 64-bit). Then you could manually decode the elements, using CodedInputStream. > The byte blob in question wouldn't be a complete message, so I can't > use UnknownFieldSet.parseFrom(), can I? > (unlike embedded messages) That is correct. > > > On Jun 25, 1:46 am, Kenton Varda wrote: > > On Wed, Jun 24, 2009 at 6:15 AM, Toph wrote: > > > > > Thanks for the reply Kenton. > > > > > Yes I discovered UnknownFieldSet, although it (and thus its toString > > > () ) isn't able to recurse down a tree of embedded messages stored in > > > length-delimited values (as far as I can see), so I've written a > > > recursive method to do just that (which has to try to parse such a > > > value in order to determine if it is an embedded message, or just a > > > bunch of bytes/text). > > > > Right, there is no way to distinguish between embedded messages and byte > > blobs without having the proto definition. > > > > I believe the C++ TextFormat code heuristically detects embedded messages > > when printing unknown fields, the same way you do. I don't think that's > > been ported to Java, though. > > > > > Is UnknownFieldSet able to handle packed repeated fields correctly and > > > parse them out of the length delimited value into one of the > > > UnknownFieldSet.Field.get*List() Lists, as opposed to just leaving > > > them packed in the value (as it does with embedded messages) ? I > > > assume so, but haven't tried it. > > > > No. Again, this requires knowing the proto definition. There's no way > to > > distinguish between packed arrays and byte blobs given only the encoded > > message. > > > > > I'm ok with the interface changing in a future version as long as it > > > supports extracting all of the data that it does today. > > > > Yes, it will contain all the same data, but it will no longer group the > data > > by field number. Instead it will just have an array of number/value > pairs > > in arbitrary order (with groups still represented as embedded > > UnknownFieldSets). > > > > > > > > > > > > > On Jun 24, 2:12 am, Kenton Varda wrote: > > > > You can use UnknownFieldSet, but be warned that the interface for > that > > > class > > > > is likely to change in a future version (because the current design > is > > > > somewhat inefficient). If you just want to print the contents, you > > > should > > > > be fine -- just parse into an UnknownFieldSet and then call its > > > toString() > > > > method. Those parts won't change. > > > > > > On Tue, Jun 23, 2009 at 7:46 AM, Toph wrote: > > > > > > > Hi folks, > > > > > > > I understand that protocol buffers messages are not fully self- > > > > > describing. > > > > > However, the message contains the field number, wire type, and > value, > > > > > right? > > > > > > > In Java, how can I take a byte[] (array of bytes) that represents a > > > > > message and deserialize it into a list of tuples that contain the > > > > > field number, wire type, and value? I really just want to print > out > > > > > these tuples, even if the value is binary. > > > > > > > Do any of the classes i the Java API provide a mechanism to do > this? > > > > > I know I could just take the documentation of the encoding and > write > > > > > code myself, but I am hoping that one of the API classes exists in > a > > > > > usable or near-usable form to do this for me. Will > CodedInputStream > > > > > do it? Can I use any of the parseFrom() or mergeFrom() methods? > > > > > > > Note that I do not have the corresponding .proto file in source > form, > > > > > compiled form, or available to transmit along with any messages > > > > > > > Thanks > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Deserializing a message for which I don't have the .proto (in Java)
> No. Again, this requires knowing the proto definition. There's no way to > distinguish between packed arrays and byte blobs given only the encoded > message. So if I know that a given byte blob is actually a packed array, how can I decode it without having a precompiled proto definition? The byte blob in question wouldn't be a complete message, so I can't use UnknownFieldSet.parseFrom(), can I? (unlike embedded messages) On Jun 25, 1:46 am, Kenton Varda wrote: > On Wed, Jun 24, 2009 at 6:15 AM, Toph wrote: > > > Thanks for the reply Kenton. > > > Yes I discovered UnknownFieldSet, although it (and thus its toString > > () ) isn't able to recurse down a tree of embedded messages stored in > > length-delimited values (as far as I can see), so I've written a > > recursive method to do just that (which has to try to parse such a > > value in order to determine if it is an embedded message, or just a > > bunch of bytes/text). > > Right, there is no way to distinguish between embedded messages and byte > blobs without having the proto definition. > > I believe the C++ TextFormat code heuristically detects embedded messages > when printing unknown fields, the same way you do. I don't think that's > been ported to Java, though. > > > Is UnknownFieldSet able to handle packed repeated fields correctly and > > parse them out of the length delimited value into one of the > > UnknownFieldSet.Field.get*List() Lists, as opposed to just leaving > > them packed in the value (as it does with embedded messages) ? I > > assume so, but haven't tried it. > > No. Again, this requires knowing the proto definition. There's no way to > distinguish between packed arrays and byte blobs given only the encoded > message. > > > I'm ok with the interface changing in a future version as long as it > > supports extracting all of the data that it does today. > > Yes, it will contain all the same data, but it will no longer group the data > by field number. Instead it will just have an array of number/value pairs > in arbitrary order (with groups still represented as embedded > UnknownFieldSets). > > > > > > > On Jun 24, 2:12 am, Kenton Varda wrote: > > > You can use UnknownFieldSet, but be warned that the interface for that > > class > > > is likely to change in a future version (because the current design is > > > somewhat inefficient). If you just want to print the contents, you > > should > > > be fine -- just parse into an UnknownFieldSet and then call its > > toString() > > > method. Those parts won't change. > > > > On Tue, Jun 23, 2009 at 7:46 AM, Toph wrote: > > > > > Hi folks, > > > > > I understand that protocol buffers messages are not fully self- > > > > describing. > > > > However, the message contains the field number, wire type, and value, > > > > right? > > > > > In Java, how can I take a byte[] (array of bytes) that represents a > > > > message and deserialize it into a list of tuples that contain the > > > > field number, wire type, and value? I really just want to print out > > > > these tuples, even if the value is binary. > > > > > Do any of the classes i the Java API provide a mechanism to do this? > > > > I know I could just take the documentation of the encoding and write > > > > code myself, but I am hoping that one of the API classes exists in a > > > > usable or near-usable form to do this for me. Will CodedInputStream > > > > do it? Can I use any of the parseFrom() or mergeFrom() methods? > > > > > Note that I do not have the corresponding .proto file in source form, > > > > compiled form, or available to transmit along with any messages > > > > > Thanks --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Deserializing a message for which I don't have the .proto (in Java)
On Wed, Jun 24, 2009 at 6:15 AM, Toph wrote: > > Thanks for the reply Kenton. > > Yes I discovered UnknownFieldSet, although it (and thus its toString > () ) isn't able to recurse down a tree of embedded messages stored in > length-delimited values (as far as I can see), so I've written a > recursive method to do just that (which has to try to parse such a > value in order to determine if it is an embedded message, or just a > bunch of bytes/text). Right, there is no way to distinguish between embedded messages and byte blobs without having the proto definition. I believe the C++ TextFormat code heuristically detects embedded messages when printing unknown fields, the same way you do. I don't think that's been ported to Java, though. > Is UnknownFieldSet able to handle packed repeated fields correctly and > parse them out of the length delimited value into one of the > UnknownFieldSet.Field.get*List() Lists, as opposed to just leaving > them packed in the value (as it does with embedded messages) ? I > assume so, but haven't tried it. No. Again, this requires knowing the proto definition. There's no way to distinguish between packed arrays and byte blobs given only the encoded message. > I'm ok with the interface changing in a future version as long as it > supports extracting all of the data that it does today. Yes, it will contain all the same data, but it will no longer group the data by field number. Instead it will just have an array of number/value pairs in arbitrary order (with groups still represented as embedded UnknownFieldSets). > > > On Jun 24, 2:12 am, Kenton Varda wrote: > > You can use UnknownFieldSet, but be warned that the interface for that > class > > is likely to change in a future version (because the current design is > > somewhat inefficient). If you just want to print the contents, you > should > > be fine -- just parse into an UnknownFieldSet and then call its > toString() > > method. Those parts won't change. > > > > > > > > On Tue, Jun 23, 2009 at 7:46 AM, Toph wrote: > > > > > Hi folks, > > > > > I understand that protocol buffers messages are not fully self- > > > describing. > > > However, the message contains the field number, wire type, and value, > > > right? > > > > > In Java, how can I take a byte[] (array of bytes) that represents a > > > message and deserialize it into a list of tuples that contain the > > > field number, wire type, and value? I really just want to print out > > > these tuples, even if the value is binary. > > > > > Do any of the classes i the Java API provide a mechanism to do this? > > > I know I could just take the documentation of the encoding and write > > > code myself, but I am hoping that one of the API classes exists in a > > > usable or near-usable form to do this for me. Will CodedInputStream > > > do it? Can I use any of the parseFrom() or mergeFrom() methods? > > > > > Note that I do not have the corresponding .proto file in source form, > > > compiled form, or available to transmit along with any messages > > > > > Thanks > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Deserializing a message for which I don't have the .proto (in Java)
Thanks for the reply Kenton. Yes I discovered UnknownFieldSet, although it (and thus its toString () ) isn't able to recurse down a tree of embedded messages stored in length-delimited values (as far as I can see), so I've written a recursive method to do just that (which has to try to parse such a value in order to determine if it is an embedded message, or just a bunch of bytes/text). Is UnknownFieldSet able to handle packed repeated fields correctly and parse them out of the length delimited value into one of the UnknownFieldSet.Field.get*List() Lists, as opposed to just leaving them packed in the value (as it does with embedded messages) ? I assume so, but haven't tried it. I'm ok with the interface changing in a future version as long as it supports extracting all of the data that it does today. On Jun 24, 2:12 am, Kenton Varda wrote: > You can use UnknownFieldSet, but be warned that the interface for that class > is likely to change in a future version (because the current design is > somewhat inefficient). If you just want to print the contents, you should > be fine -- just parse into an UnknownFieldSet and then call its toString() > method. Those parts won't change. > > > > On Tue, Jun 23, 2009 at 7:46 AM, Toph wrote: > > > Hi folks, > > > I understand that protocol buffers messages are not fully self- > > describing. > > However, the message contains the field number, wire type, and value, > > right? > > > In Java, how can I take a byte[] (array of bytes) that represents a > > message and deserialize it into a list of tuples that contain the > > field number, wire type, and value? I really just want to print out > > these tuples, even if the value is binary. > > > Do any of the classes i the Java API provide a mechanism to do this? > > I know I could just take the documentation of the encoding and write > > code myself, but I am hoping that one of the API classes exists in a > > usable or near-usable form to do this for me. Will CodedInputStream > > do it? Can I use any of the parseFrom() or mergeFrom() methods? > > > Note that I do not have the corresponding .proto file in source form, > > compiled form, or available to transmit along with any messages > > > Thanks --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Deserializing a message for which I don't have the .proto (in Java)
You can use UnknownFieldSet, but be warned that the interface for that class is likely to change in a future version (because the current design is somewhat inefficient). If you just want to print the contents, you should be fine -- just parse into an UnknownFieldSet and then call its toString() method. Those parts won't change. On Tue, Jun 23, 2009 at 7:46 AM, Toph wrote: > > Hi folks, > > I understand that protocol buffers messages are not fully self- > describing. > However, the message contains the field number, wire type, and value, > right? > > In Java, how can I take a byte[] (array of bytes) that represents a > message and deserialize it into a list of tuples that contain the > field number, wire type, and value? I really just want to print out > these tuples, even if the value is binary. > > Do any of the classes i the Java API provide a mechanism to do this? > I know I could just take the documentation of the encoding and write > code myself, but I am hoping that one of the API classes exists in a > usable or near-usable form to do this for me. Will CodedInputStream > do it? Can I use any of the parseFrom() or mergeFrom() methods? > > Note that I do not have the corresponding .proto file in source form, > compiled form, or available to transmit along with any messages > > Thanks > > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Deserializing a message for which I don't have the .proto (in Java)
Hi folks, I understand that protocol buffers messages are not fully self- describing. However, the message contains the field number, wire type, and value, right? In Java, how can I take a byte[] (array of bytes) that represents a message and deserialize it into a list of tuples that contain the field number, wire type, and value? I really just want to print out these tuples, even if the value is binary. Do any of the classes i the Java API provide a mechanism to do this? I know I could just take the documentation of the encoding and write code myself, but I am hoping that one of the API classes exists in a usable or near-usable form to do this for me. Will CodedInputStream do it? Can I use any of the parseFrom() or mergeFrom() methods? Note that I do not have the corresponding .proto file in source form, compiled form, or available to transmit along with any messages Thanks --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
