[protobuf] How to get the byte[] from a serialized data ?
I am trying to use Protocol buffers to store serialized data in Database for a web application built in java. I have created .proto files and compiled them to get the generated classes. Also I can build the message objects using the setters finally build() method. But to store it to database, I need serialized data as byte[] or byte buffers. How do I finally get that from the message instances ?? So what do i need to do after creating the objects ? -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] How to use the FieldOptions in java
Hi,
I was scanning the Language Guide and saw that there are simple ways
to access the MessageOptions defined in the java code like:
String value =
MyProtoFile.MyMessage.getDescriptor().getOptions().getExtension(MyProtoFile.myOption);
However if i have a field option like (using the e.g. in Language
guide)
extend google.protobuf.FieldOptions {
optional float my_field_option = 50002;
}
optional int32 foo = 1 [(my_field_option) = 4.5];
How do I access the my_field_option value in Java. The
MyProtoFile.myFieldOption static member does exist, but I don't know
how to use it.
I have tried a lot of things, but was unable to figure out a way.
Thanks, Harpreet
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: How to use the FieldOptions in java
You need to use the field descriptor for the field you want to look
at.
You can get ahold of the field descriptor by using findFieldByName
or findFieldByNumber
FieldDescriptor myFieldDescriptor =
MyProtoFileClass.MyMessage.getDescriptor().findFieldByNumber(MyProtoFileClass.MyMessage.MYFIELD_FIELD_NUMBER);
(anybody know a better way to get field descriptors?)
Object myExtensionValue =
myFieldDescriptor.getOptions().getField(MyProtoFileClass.myfieldoption.getDescriptor());
On Mar 4, 8:49 am, Harpreet hpreet.si...@gmail.com wrote:
Hi,
I was scanning the Language Guide and saw that there are simple ways
to access the MessageOptions defined in the java code like:
String value =
MyProtoFile.MyMessage.getDescriptor().getOptions().getExtension(MyProtoFile.myOption);
However if i have a field option like (using the e.g. in Language
guide)
extend google.protobuf.FieldOptions {
optional float my_field_option = 50002;
}
optional int32 foo = 1 [(my_field_option) = 4.5];
How do I access the my_field_option value in Java. The
MyProtoFile.myFieldOption static member does exist, but I don't know
how to use it.
I have tried a lot of things, but was unable to figure out a way.
Thanks, Harpreet
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] How to get the byte[] from a serialized data ?
On Mar 4, 2011, at 7:15 , Aditya Narayan wrote: I have created .proto files and compiled them to get the generated classes. Also I can build the message objects using the setters finally build() method. But to store it to database, I need serialized data as byte[] or byte buffers. How do I finally get that from the message instances ?? You want .toByteArray(): http://code.google.com/apis/protocolbuffers/docs/reference/java/com/google/protobuf/MessageLite.html#toByteArray() Evan -- http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] RuntimeException while parsing back the byte[] to protocol buffer message instance! (deserialization)
I tried following code for encoding and decoding.
Decoding from byte[] fails, however.
Code:-
public static void test(){
Comment comment = Comment.newBuilder()
.setCommentBody(This is 1st comment)
.build();
byte[] bytes = comment.toByteArray();
try {
Comment c = Comment.parseFrom(bytes);
}
catch (InvalidProtocolBufferException ex) {
Logger.getLogger(testing.class.getName()).log(Level.SEVERE,
null, ex);
}
Exception details:-
Exception in thread main java.lang.RuntimeException: Uncompilable
source code
at
com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java)
at
com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:153)
at
com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:709)
at
paratha.serializers.protocolbuffers.CommentProto$Comment.parseFrom(CommentProto.java:186)
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
[protobuf] RuntimeException while parsing back the byte[] to protocol buffer message instance! (deserialization)
I tried following code for encoding and decoding.
Decoding from byte[] fails, however.
Code:-
public static void test(){
Comment comment = Comment.newBuilder()
.setCommentBody(This is 1st comment)
.build();
byte[] bytes = comment.toByteArray();
try {
Comment c = Comment.parseFrom(bytes);
}
catch (InvalidProtocolBufferException ex) {
Logger.getLogger(testing.class.getName()).log(Level.SEVERE,
null, ex);
}
Exception details:-
Exception in thread main java.lang.RuntimeException: Uncompilable
source code
at
com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java)
at
com.google.protobuf.AbstractMessageLite$Builder.mergeFrom(AbstractMessageLite.java:153)
at
com.google.protobuf.AbstractMessage$Builder.mergeFrom(AbstractMessage.java:709)
at
paratha.serializers.protocolbuffers.CommentProto$Comment.parseFrom(CommentProto.java:186)
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Chunking a large message
On Mar 3, 2011, at 15:53 , Linus wrote: I am wondering if there are any examples of chunking large PB messages (about 1MB) into smaller chunks, to transmit over the wire. This is going to be pretty application specific. Typically it involves taking one message with a huge repeated field and sending it / writing it as a sequence of messages with fewer items for each repeated field. So I can't really point you to any examples off the top of my head. That said: the documentation suggests keeping protocol buffers to be ~ 1 MB in size, so if your messages are 1 MB, I personally wouldn't worry about it. Hope this helps, Evan -- http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] RuntimeException while parsing back the byte[] to protocol buffer message instance! (deserialization)
On Mar 4, 2011, at 11:11 , Aditya Narayan wrote: Exception in thread main java.lang.RuntimeException: Uncompilable source code This error means there is a build problem in your Eclipse project. You are trying to call some code that is not building compiled correctly. Fix your build errors and then your example should work. Good luck, Evan -- http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] RuntimeException while parsing back the byte[] to protocol buffer message instance! (deserialization)
Thanks Evan Seems like I am missing some files. I have just created a .proto file and compiled it with protoc to get generated java files. I got .java file compiled and put it with the java's source folder from protobuf 2.4.a in my project folder. But some of the class dependency of the files in this folder seem to be unresolved. I think I would instead need a jar that I could add to classpath of my project. -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
[protobuf] Re: Chunking a large message
Thanks Evan. Your response makes some things clear to me. I meant to say I'd like chunks of 1MB ( I re-read my email and realize it wasn't worded properly). I now have some ideas that I'm going to try out. Thanks again! On Mar 4, 8:37 am, Evan Jones ev...@mit.edu wrote: On Mar 3, 2011, at 15:53 , Linus wrote: I am wondering if there are any examples of chunking large PB messages (about 1MB) into smaller chunks, to transmit over the wire. This is going to be pretty application specific. Typically it involves taking one message with a huge repeated field and sending it / writing it as a sequence of messages with fewer items for each repeated field. So I can't really point you to any examples off the top of my head. That said: the documentation suggests keeping protocol buffers to be ~ 1 MB in size, so if your messages are 1 MB, I personally wouldn't worry about it. Hope this helps, Evan --http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] RPC Methods
http://code.google.com/apis/protocolbuffers/docs/proto.html#services describes
how to define services. However, protobuf does not provide an RPC
implementation. You may set the file option {lang}_generic_services = true
to have the protocol compiler generate a stub that can be used to integrate
with an RPC implementation. The new recommendation is to write a compiler
plugin to generate implementation-specific code using the service
descriptions provided by the protocol compiler:
http://code.google.com/apis/protocolbuffers/docs/reference/other.html
On Fri, Mar 4, 2011 at 4:38 PM, Joshua Partogi joshua.part...@gmail.comwrote:
Hi there,
Is it possible to define RPC methods in protobuf? From what I am
reading it seems that protobuf is only limited to data structured data
serializer. Is there any documentation I read about RPC methods in
protobuf if any is available?
Thank you for the assistance.
Kind regards,
Joshua.
--
http://twitter.com/jpartogi
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
Re: [protobuf] Store number of messages
Varint does an encoding whose length changes with the size of the
number - hence the name. So it can use between 1 and 10 bytes. So when
the value _events_written changes, re-writing the same number at the
beginning of the file will use a different number of bytes.
If you want to write a number that changes like this to a particular
position, you need to use a fixed size number encoding (e.g. fixed32
or fixed64) to do so.
On Fri, Mar 4, 2011 at 16:22, ksamdev samvel.khalat...@gmail.com wrote:
I have an application where number of messages is a priory unknown. I can
open output file and write 0 in the beginning:
Writer::Writer(const string filename):
_output(filename.c_str(), ios::out | ios::trunc | ios::binary),
_events_written(0)
{
_raw_out.reset(new
::google::protobuf::io::OstreamOutputStream(_output));
_coded_out.reset(new
::google::protobuf::io::CodedOutputStream(_raw_out.get()));
_coded_out-WriteVarint32(_events_written);
At this time, _events_written is zero, which, encoded, uses a
different length in the file ...
}
and then in the destructor reset the position in the ofstream to the
beginning to write the number of the events:
Writer::~Writer()
{
_coded_out.reset();
_raw_out.reset();
_output.seekp(0);
// Small trick to save number of the events
//
_raw_out.reset(new
::google::protobuf::io::OstreamOutputStream(_output));
_coded_out.reset(new
::google::protobuf::io::CodedOutputStream(_raw_out.get()));
_coded_out-WriteVarint32(_events_written);
... than here: _events_written is a larger number, encoded with more
bytes, so it will overwrite parts of the following data ..
_coded_out.reset();
_raw_out.reset();
_output.close();
}
Then read the beginning of the file with number of the events:
Reader::Reader(const string filename):
_input(filename.c_str(), ios::in | ios::binary),
_is_good(true),
_events_written(0)
{
_raw_in.reset(new ::google::protobuf::io::IstreamInputStream(_input));
_coded_in.reset(new
::google::protobuf::io::CodedInputStream(_raw_in.get()));
_coded_in-ReadVarint32(_events_written);
}
Everything seems fine. Then events can be read one by one like:
bool Reader::read(Event event)
{
event.Clear();
uint32_t message_size;
if (!_coded_in-ReadVarint32(message_size))
... so here you have a number, whose beginning is partially
overwritten by the end of the re-written events list - the reading
fails.
{
_is_good = false;
return false;
}
if (0 message_size)
{
string message;
if (!_coded_in-ReadString(message, message_size) ||
!event.ParseFromString(message))
return false;
}
return true;
}
Unfortunately, ReadVarint32 fails for some reason in the Reader::read(...)
method.
The code works fine in case I do not use seekp in the Writer::~Writer().
What is the proper way to seek to the beginning of the file and store number
of entries?
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
--
You received this message because you are subscribed to the Google Groups
Protocol Buffers group.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en.
