Re: question about protobuf in VC

[Kenton Varda]

Odd...  it's as if addressbook.pb.cc's static initializers were run twice
(so it was registered in the global descriptor pool twice, and on the second
instance you saw the error).
Can you run in a debugger, set a breakpoint at
DescriptorPool::InternalBuildGeneratedFile(), and see what calls it, and
what the filename is each time it is called?

On Mon, Sep 8, 2008 at 6:23 AM, Mars <[EMAIL PROTECTED]> wrote:

>
> VS express 2008
>
> libprotobuf.lib and libprotobuf.dll are copy from the output of
> libprotobuf.vcproj.
> addressbook.pb.h and addressbook.pb.cc are generated by using protoc --
> cpp_out
> project -> project -> configuration properties -> c/c++ -> general ->
> Addition Include Directories "protobuf-2.0.0.beta\src"
>
> project -> project -> configuration properties -> Linker -> Input ->
> Addition Dependencies "libprotobuf.lib"
>
> == src code ==
>
> #include "stdafx.h"
> #include 
> #include 
> #include 
> #include "addressbook.pb.h"
> using namespace std;
>
>
> int _tmain(int argc, _TCHAR* argv[]) {
>  // Verify that the version of the library that we linked against is
>  // compatible with the version of the headers we compiled against.
>  GOOGLE_PROTOBUF_VERIFY_VERSION;
>  string filename = "t1.ss";
>
>  tutorial::AddressBook address_book;
>  tutorial::Person *person = address_book.add_person();
>
>  person->set_name("bati");
>  person->set_id(9);
>  tutorial::Person::PhoneNumber* phone_number = person->add_phone();
>  phone_number->set_number("1390109");
>  phone_number->set_type(tutorial::Person::MOBILE);
>
>  // Write the new address book back to disk.
>  fstream output(filename.c_str(), ios::out | ios::trunc |
> ios::binary);
>  if (!address_book.SerializeToOstream(&output)) {
>cerr << "Failed to write address book." << endl;
>return -1;
>  }
>
>  return 0;
> }
>
>
> === error message ===
>
> libprotobuf ERROR ..\src\google\protobuf\descriptor.cc:1776] Invalid
> proto descr
> iptor for file "addressbook.proto":
> libprotobuf ERROR ..\src\google\protobuf\descriptor.cc:1779]
> addressbook.proto
> : A file with this name is already in the pool.
> libprotobuf FATAL ..\src\google\protobuf\descriptor.cc:1752] CHECK
> failed: resul
> t != NULL:
>
> This application has requested the Runtime to terminate it in an
> unusual way.
> Please contact the application's support team for more information.
>
>
>
> how to fix that?
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Feature Request

[Kenton Varda]

Sure, I'd be OK with adding methods to Message like:
  bool SerializeDelimitedTo(CodedOutputStream* output);
  bool SerializeDelimitedToZeroCopyStream(ZeroCopyOutputStream* output);
  bool SerializeDelimitedToFile(int file_descriptor);
  bool SerializeDelimitedToOstream(ostream* output);

  bool ParseDelimitedFrom(CodedOutputStream* input);
  bool ParseDelimitedFromZeroCopyStream(ZeroCopyInputStream* input);
  bool MergeDelimitedFrom(CodedOutputStream* input);
  bool MergeDelimitedFromZeroCopyStream(ZeroCopyInputStream* input);
  // Note that we cannot parse a length-delimited message from
  // a file descriptor or an istream since these interfaces don't provide
  // a way to push data back into the stream if we read too far.

I'm pretty swamped, though.  Does someone want to write up a patch (with
unit tests)?

On Sat, Sep 6, 2008 at 2:44 AM, <[EMAIL PROTECTED]> wrote:

> Kenton Varda wrote:
>
>> Feel free to use the above code (note: hasn't been compiled or tested).  I
>> guess the question is whether or not this needs to be in libprotobuf itself.
>>
>
> I am in favor of putting some kind of delimited message format into the
> API, this is the requested feature.  This will keep people from being too
> creative and reinventing incompatible solutions to have delimited messages.
>  Such as the suggestions in this thread.
>
> The LengthDelimited functions are the most obvious: all possible
> implementation of messages must already have the functionality internally
> (as your code shows).  So a few lines of code in the API (not the generated
> code files) takes care of this feature request.  I have not checked if the
> existing API for all languages exposes enough to write the equivalent of the
> few lines of code you showed for C++.
> The most obvious use to me is reading from a continous (e.g. network)
> stream of bytes.  The outermost message needs to be delimited somehow,
> currently by the application inventing more protocol rules.
>
> Cheers,
> Chris
>
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Deserializing Messages of unknown type at compile-time

[Kenton Varda]

On Mon, Sep 8, 2008 at 4:33 AM, <[EMAIL PROTECTED]> wrote:

> More tricky.  Given a stream and an instance, I'm trying to get the
> Descriptor by calling Message#getDescriptorForType() on the instance
> and passing the return value, along with an input stream, to
> DynamicMessage#parseFrom(Descriptor,input).  I then cast the
> DynamicMessage that is returned by parseFrom to the same type of the
> instance that is given to me.


That won't work.  DynamicMessage is a different class; it does not know how
to instantiate the protocol-compiler-generated version of the class.
 Instead, you should do:

  Message result =
messageInstance.newBuilderForType().mergeFrom(input).build();

Actually, you should check isInitialized() before calling build(), or use
buildPartial() instead, but that's a separate issue.


> The problem that I'm encountering is during deserialization.  I'm
> getting an InvalidProtocolBufferException.  Here's the trace:
>
> ...
>
> What's curious about this is that my .proto files each only have one
> field in each Message, and each of those fields has a tag of 1.  None
> of my tags are 0.


The protocol compiler would not allow you to use tag zero anyway.  It looks
like your input data is not identical to the data written by the sender.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: how to handle message spanning multiple packets?

[Kenton Varda]

Protocol Buffers solve the problem of converting structured data into a flat
byte array.  They do not solve the problem of communicating that byte array
over a network; that's the job of some other library.  Sorry to disappoint.

On Sun, Sep 7, 2008 at 10:58 AM, Mateusz Berezecki <[EMAIL PROTECTED]>wrote:

>
> Hello Group,
>
> I have a question that worries me for some time now,
> and I have not found any answer for it.
>
> How am I supposed to handle messages
> spanning multiple packets of a connectionless protocol ?
> I am thinking UDP here.
>
> All is good if the message can be contained inside a single
> UDP packet, but what if it does not ?
>
> What if the message spans K packets ?
> ParseFromString no longer works as I don't know
> the message length , and how many packets it spans.
>
> How do I discover the message length dynamically,
> after only receiving a part of it???
>
> I could of course monitor packets, prepend length information
> on top of protobuf message (but this adds yet another
> data description layer on top of protobuf, so it defeats
> the purpose of having protobufs at all), then start sort of
> "journalling"
> the packets, and start managing incoming fragments, and merge
> them when appropriate and finally deserialize them.
>
>
> But, I thought that it was done within the protobuf (yes, I am
> seriously disappointed,
> because the whole library is now seriously crippled :-( ) !
>
> Apparently it is not.
>
> So, I started looking at RPC services but this what was
> supposed to be a best thing in the whole library,
> suddenly suffers from basically no documentation,
> and no examples whatsoever.
>
>
> Is there a way to solve my problem in an _elegant_ way ?
> I am into a solution that utilizes protobuf's RPC channels/controllers
> even though I have no friggin idea how to use them :-) but I have
> an intuition that tells me that's the way to do it.
>
> So, could anyone please describe their experience and some
> guidelines (an example would be nice too) on
> solving this kind of problem?
>
>
> Mateusz
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: how to handle message spanning multiple packets?

[Kenton Varda]

They are just abstract interfaces.  You have to implement them.  For
examples, I suppose you could look at some of the RPC implementations people
are working on:
http://code.google.com/p/protobuf/wiki/RPCImplementations

On Mon, Sep 8, 2008 at 10:34 AM, Mateusz Berezecki <[EMAIL PROTECTED]>wrote:

> On Mon, Sep 8, 2008 at 7:26 PM, Kenton Varda <[EMAIL PROTECTED]> wrote:
> > Protocol Buffers solve the problem of converting structured data into a
> flat
> > byte array.  They do not solve the problem of communicating that byte
> array
> > over a network; that's the job of some other library.  Sorry to
> disappoint.
>
> Hello Kenton,
>
> It's not that much of a disappointment. I was having a perception
> it does solve this particular problem based on the "media coverage".
>
> Anyway, could you please point me to some examples or perhaps
> explain how to use RpcChannel and RpcController classes?
>
> The information in the documentation is rather scarce.
>
> Mateusz
> >
> > On Sun, Sep 7, 2008 at 10:58 AM, Mateusz Berezecki <[EMAIL PROTECTED]>
> > wrote:
> >>
> >> Hello Group,
> >>
> >> I have a question that worries me for some time now,
> >> and I have not found any answer for it.
> >>
> >> How am I supposed to handle messages
> >> spanning multiple packets of a connectionless protocol ?
> >> I am thinking UDP here.
> >>
> >> All is good if the message can be contained inside a single
> >> UDP packet, but what if it does not ?
> >>
> >> What if the message spans K packets ?
> >> ParseFromString no longer works as I don't know
> >> the message length , and how many packets it spans.
> >>
> >> How do I discover the message length dynamically,
> >> after only receiving a part of it???
> >>
> >> I could of course monitor packets, prepend length information
> >> on top of protobuf message (but this adds yet another
> >> data description layer on top of protobuf, so it defeats
> >> the purpose of having protobufs at all), then start sort of
> >> "journalling"
> >> the packets, and start managing incoming fragments, and merge
> >> them when appropriate and finally deserialize them.
> >>
> >>
> >> But, I thought that it was done within the protobuf (yes, I am
> >> seriously disappointed,
> >> because the whole library is now seriously crippled :-( ) !
> >>
> >> Apparently it is not.
> >>
> >> So, I started looking at RPC services but this what was
> >> supposed to be a best thing in the whole library,
> >> suddenly suffers from basically no documentation,
> >> and no examples whatsoever.
> >>
> >>
> >> Is there a way to solve my problem in an _elegant_ way ?
> >> I am into a solution that utilizes protobuf's RPC channels/controllers
> >> even though I have no friggin idea how to use them :-) but I have
> >> an intuition that tells me that's the way to do it.
> >>
> >> So, could anyone please describe their experience and some
> >> guidelines (an example would be nice too) on
> >> solving this kind of problem?
> >>
> >>
> >> Mateusz
> >> > >>
> >
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Deserializing Messages of unknown type at compile-time

[Kenton Varda]

On Mon, Sep 8, 2008 at 6:18 PM, Alex Loddengaard
<[EMAIL PROTECTED]>wrote:

> On Tue, Sep 9, 2008 at 1:16 AM, Kenton Varda <[EMAIL PROTECTED]> wrote:
>
>> That won't work.  DynamicMessage is a different class; it does not know
>> how to instantiate the protocol-compiler-generated version of the class.
>>  Instead, you should do:
>>
>>   Message result =
>> messageInstance.newBuilderForType().mergeFrom(input).build();
>>
>> Actually, you should check isInitialized() before calling build(), or use
>> buildPartial() instead, but that's a separate issue.
>>
>
> I changed my deserializing code to use the above, but I'm getting the same
> exception.  I also tried to call isInitialized() on the instance given to
> me, and the instance is not initialized.
>

That message instance is probably a default instance.  isInitialized() will
always be false on those, unless it has no required fields at all.


>
>
>> The protocol compiler would not allow you to use tag zero anyway.  It
>> looks like your input data is not identical to the data written by the
>> sender.
>>
>
> I'm confident that the sender data is the same data that is created when I
> serialize.
>

The exceptions that you're reporting strongly suggest that you are *not*
seeing the same data on both ends.  Try this:  serialize to a byte array or
ByteString, compute a checksum of some sort for debugging, then write the
bytes to your output.  On the other end, read the bytes back into a
ByteString or byte array, checksum again, and see if it's the same.  Then
parse from that.  I'm pretty confident that if the checksums are the same,
you will not see the error you're seeing.


>   Perhaps I'm serializing incorrectly?  I'm creating a CodedOutputStream
> given an OutputStream and passing that to writeTo.
>

This is redundant -- you can just pass the OutputStream to writeTo().


>   However, I'm not using a CodedInputStream to deserialize.  Should I be
> using Coded or non-Coded streams?
>

It doesn't matter.  If given a normal stream, mergeFrom() / writeTo() will
wrap it in a coded stream on their own.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Deserializing Messages of unknown type at compile-time

[Kenton Varda]

On Mon, Sep 8, 2008 at 6:27 PM, Kenton Varda <[EMAIL PROTECTED]> wrote:

>
>
> On Mon, Sep 8, 2008 at 6:18 PM, Alex Loddengaard <
> [EMAIL PROTECTED]> wrote:
>
>> On Tue, Sep 9, 2008 at 1:16 AM, Kenton Varda <[EMAIL PROTECTED]> wrote:
>>
>>> That won't work.  DynamicMessage is a different class; it does not know
>>> how to instantiate the protocol-compiler-generated version of the class.
>>>  Instead, you should do:
>>>
>>>   Message result =
>>> messageInstance.newBuilderForType().mergeFrom(input).build();
>>>
>>> Actually, you should check isInitialized() before calling build(), or use
>>> buildPartial() instead, but that's a separate issue.
>>>
>>
>> I changed my deserializing code to use the above, but I'm getting the same
>> exception.  I also tried to call isInitialized() on the instance given to
>> me, and the instance is not initialized.
>>
>
> That message instance is probably a default instance.  isInitialized() will
> always be false on those, unless it has no required fields at all.
>

To clarify:  In my original message I was saying that you should call
isInitialized() on the builder returned by mergeFrom(), to make sure the
parsed message is complete, before you call build().


>
>>
>>
>>>  The protocol compiler would not allow you to use tag zero anyway.  It
>>> looks like your input data is not identical to the data written by the
>>> sender.
>>>
>>
>> I'm confident that the sender data is the same data that is created when I
>> serialize.
>>
>
> The exceptions that you're reporting strongly suggest that you are *not*
> seeing the same data on both ends.  Try this:  serialize to a byte array or
> ByteString, compute a checksum of some sort for debugging, then write the
> bytes to your output.  On the other end, read the bytes back into a
> ByteString or byte array, checksum again, and see if it's the same.  Then
> parse from that.  I'm pretty confident that if the checksums are the same,
> you will not see the error you're seeing.
>
>
>>   Perhaps I'm serializing incorrectly?  I'm creating a CodedOutputStream
>> given an OutputStream and passing that to writeTo.
>>
>
> This is redundant -- you can just pass the OutputStream to writeTo().
>
>
>>   However, I'm not using a CodedInputStream to deserialize.  Should I be
>> using Coded or non-Coded streams?
>>
>
> It doesn't matter.  If given a normal stream, mergeFrom() / writeTo() will
> wrap it in a coded stream on their own.
>
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Deserializing Messages of unknown type at compile-time

[Kenton Varda]

On Mon, Sep 8, 2008 at 9:11 PM, Alex Loddengaard
<[EMAIL PROTECTED]>wrote:

> I have a follow-up question:
>
> Will using 
> *messageInstance.newBuilderForType().mergeFrom(input).build();*work for a 
> stream that contains trailing binary information?
>

No, it won't work.  Protocol buffers are not self-delimiting.  They assume
that the input you provide is supposed to be one complete message, not a
message possibly followed by other stuff.

You will need to somehow communicate the size of the message and make sure
to limit the input to that size.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Deserializing Messages of unknown type at compile-time

[Kenton Varda]

On Mon, Sep 8, 2008 at 10:52 PM, Alex Loddengaard <[EMAIL PROTECTED]
> wrote:

> I should revise my problem slightly.  I had said that I am given an
> instance of a Message class when deserializing.  This is true, though
> sometimes that instance is null.  In the cases when it's null, I'm not able
> to call newBuilderForType() on it.  I'm not able to call
> getDefaultInstance(), either.  This is now problematic, though there may be
> a work around.  Also given to me is a Class instance of the Message.  I'm
> using Reflection to instantiate a new Message instance, then
> getDefaultInstance() to get the default instance, and then I'm calling
> newBuilderForType().  Is this problematic?
>

Hmm, I think the framework you are using is poorly designed -- it should
always give you a non-null default instance.  Using Java reflection is ugly.

getDefaultInstance() is actually a static method.  So, you don't have to
instantiate a new message instance first -- just call the static method
without an instance.  You can't actually instantiate the message class
directly anyway, since the constructors are private.


>
>
> Thanks again.  Sorry for all the spam!
>
> Alex
>
>
> On Tue, Sep 9, 2008 at 12:11 PM, Alex Loddengaard <
> [EMAIL PROTECTED]> wrote:
>
>> I have a follow-up question:
>>
>> Will using *messageInstance.newBuilderForType().mergeFrom(input).build();
>> * work for a stream that contains trailing binary information?
>>
>> I'm asking this question for the following reason: I'm using a very simple
>> example where my Message just contains a single String.  When I print the
>> serialized message with a value of "my_string", I get "my_string".
>> Now, when I see the stream coming in on the deserialization side, I get
>> "my_string"  The leading binary is the same as the original,
>> however the trailing binary is something new entirely.  The trailing binary
>> is probably being created by Hadoop.
>>
>> Kenton, you have made it very clear that *
>> messageInstance.newBuilderForType().mergeFrom(input).build();* is the
>> correct approach.  What could possibly be going wrong if the stream I'm
>> trying to deserialize from contains trailing binary data?
>>
>> Thanks ahead of time for your help.
>>
>> Alex
>>
>>
>> On Tue, Sep 9, 2008 at 10:47 AM, Alex Loddengaard <
>> [EMAIL PROTECTED]> wrote:
>>
>>> After taking my code out of Hadoop, it looks as though my deserializing
>>> mechanism is working fine.  My problem lies with my integration with Hadoop.
>>>
>>> Thanks for resolving this issue, Kenton!
>>>
>>> Alex
>>>
>>>
>>> On Tue, Sep 9, 2008 at 9:42 AM, Alex Loddengaard <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>>> On Tue, Sep 9, 2008 at 9:28 AM, Kenton Varda <[EMAIL PROTECTED]> wrote:
>>>>
>>>>> To clarify:  In my original message I was saying that you should call
>>>>> isInitialized() on the builder returned by mergeFrom(), to make sure the
>>>>> parsed message is complete, before you call build().
>>>>>
>>>>
>>>> Ah.  Now isInitialized() is returning true, though I'm still having
>>>> problems deserializing.  Now that I'm using OutputStream and InputStream,
>>>> I'm getting the following exception:
>>>>
>>>> com.google.protobuf.InvalidProtocolBufferException: Protocol message tag
>>>> had invalid wire type.
>>>>
>>>> I'm going to take my code out of Hadoop to see if Hadoop is causing
>>>> these issues.  I'm still weary of that, though, because other serialization
>>>> frameworks such as Facebook's Thrift seem to work in the framework that I 
>>>> am
>>>> using.
>>>>
>>>> Thanks for your help, Kenton!  I'll check back soon with my progress.
>>>>
>>>
>>>
>>
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Scientific Applications

[Kenton Varda]

Regarding speed, make sure you have:
option optimize_for = SPEED;

in your .proto file.  (The default is to optimize for code size, since
people should only optimize for speed after profiles show that they need
it.)

Regarding reading only part of a file:  Protocol buffers are designed to be
parsed all at once.  If you want to have multiple records in a file which
can be parsed individually, you will need to devise some container format
for them.  Note that each record could still be a protocol buffer.

On Thu, Sep 11, 2008 at 11:55 AM, Nicolas <[EMAIL PROTECTED]> wrote:

>
> Hi,
>
> I have been looking at (and starting to use) protocol buffers to store
> data from scientific applications, typically objects with different
> attributes, and first of all coordinates (of varying length),
> extracted from video or laser scanner data. I did some tests and I
> have been wondering if this is the right tool. What I like is to
> offload the writing of all serializing/de-serializing to an external
> library, and just care about the description of the data format. What
> is inconvenient is the obligation to write a super data type with a
> repeated field to save more than one object of the same type in the
> same message. I was also a bit surprised by the relatively low speed
> to load the data from a file. I also would like to be able to access
> part of the data in a file without loading the whole file. Is it a
> good tool if all one needs is saving/loading data to/from a file?
>
> Can anyone with some experience in these matters, and especially of
> alternative formats, e.g. netCDF, comment on this and recommend a
> standard well-supported solution?
>
> Thanks for any help and for sharing this tool with the community,
>
> Nicolas
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Scientific Applications

[Kenton Varda]

Note that the inefficiency of repeated primitive types will be fixed...
 eventually...  but, yeah, right now it's no good.

On Fri, Sep 12, 2008 at 2:09 AM, Chris <[EMAIL PROTECTED]> wrote:

>
> Nicolas wrote:
> > Can anyone with some experience in these matters, and especially of
> > alternative formats, e.g. netCDF, comment on this and recommend a
> > standard well-supported solution?
> I do not think scientific data should be stored with protocol-buffers.
>
> I would suggest, since netCDF-4 now encloses HDF5, the you also look at
> HDF5 (e.g. wikipedia is always a good
> source of links).
>
> The netCDF and HDF5 are both self-describing data file formats unlike
> protocol-buffer's wire format.
>
> It is impossible to read back and analyze a protocol-buffer without the
> .proto file description because the wire format does not hold the actual
> type of any of the data (e.g. bool vs int vs unsigned int vs
> enumeration, or string vs bytes vs embedded message).
>
> The protocol-buffer wire format for repeated elements, such as you need
> for scientific data vectors and arrays, is relatively inefficient since
> it includes the "field# + wire tag" before each and every single number
> in your file.  This kills efficient bulk reading and writing unless you
> create a new format and embed it as a binary blob.
> So you end up using HDF5 or something else anyway.
>
> Cheers,
>   Chris
>
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Deserializing Messages of unknown type at compile-time

[Kenton Varda]

Even if the message contains only one, non-repeated field, ParseFrom*() will
keep reading until EOF or an error.
At Google, we have lots of various container formats, for streaming,
record-based files, database tables, etc., where each record is a protocol
buffer.  All of these formats store the size of the message before the
message itself.  Our philosophy is that because we have protocol buffers,
all of these *other* formats and protocols can be designed to pass around
arbitrary byte blobs, which greatly simplifies them.  An arbitrary byte blob
is not necessarily self-delimiting, so it's up to these container formats to
keep track of the size separately.

On Thu, Sep 11, 2008 at 8:20 PM, <[EMAIL PROTECTED]> wrote:

>
> Kenton,
>
> > No, it won't work.  Protocol buffers are not self-delimiting.  They
> assume
> > that the input you provide is supposed to be one complete message, not a
> > message possibly followed by other stuff.
>
>
> There are a couple of related threads about delimiting the outer
> message (with either a marker or a length). The need for this seems to
> arise from streaming  (especially when input would block such as on a
> network socket).
>
> Could this not be solved by a simple convention in the proto file ?
> (Maybe I am missing something big here)
>
> Let us say we have a proto as follows
>
> message TRPProtocol
> {
>  message TRPPDU
>  {
>required int32 version;
>required int32 type;
>
>optional HelloRequest   hello_req = 1;
>optional HelloResponse  hello_resp = 2;
>optional ConnectRequest connect_req =
> 3;
> // etc etc
>  };
>  required TRPPDU  thepdu=1;
> };
>
> On the wire the outer message is not length delimited, but the inner
> message is. The inner message is represented by the 'required' field
> 'thepdu'.
>
> It would then be possible to stream instances of the inner message
> "TRPPDU". I hope my understanding is correct. Could you write
> something like the following ?
>
> TRPProtocol::TRPPDU Pdu;
> Pdu.ParseFromFileDescriptor( socket_fd);   // socket_fd has been
> opened and initialized earlier
>
> to read just one message, respond to that if needed, and then read the
> next one.
>
> Is my understanding correct ? Is this how it is done at Google when
> using PB for client - server comms ?
>
> Thanks,
>
> Vivek
>
>
> >
> > You will need to somehow communicate the size of the message and make
> sure
> > to limit the input to that size.
>
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Deleting one of a repeated field?

[Kenton Varda]

On Tue, Sep 16, 2008 at 9:28 AM, <[EMAIL PROTECTED]> wrote:

>
> reply to self: I guess you copy the item from the end of the list into
> the space formerly occupied by the item you want to delete, then call
> RemoveLast()...


Yes, that's what you should do.

I didn't want to provide a Remove(int) because it would be O(n).  Version 1
of protocol buffers had such a function, and we frequently saw people
writing loops like:

for (int i = 0; i < field.size(); i++) {
  if (ShouldFilter(field[i])) {
field.Remove(i);
--i;
  }
}

This loop is O(n^2), which is bad, but it's hard to tell that it is O(n^2).
 The idea behind only providing RemoveLast() is to force you to either do
something clever (like swapping the element with the last element first, as
the documentation suggests) or write your own loop which makes the time
complexity of your code obvious.



>
>
>
> On Sep 16, 4:27 pm, [EMAIL PROTECTED] wrote:
> > OK, rephrase that question. Say you have a releated field containing
> > 10 items.  You wish to delete the 2nd item so that the list contains 9
> > items and foo_size() returns 9. Is that possible?
> >
> > I can't see an erase() function in mutable_foo(), and I can't see
> > anything in RepeatedField that decrements current_size_ other than
> > RemoveLast() which presumably will only remove the _last_ item?
> >
> > Currently the only way to remove elements from a list seems to be
> > awful hacks like a.parse(a.serialise()) after clearing the items you
> > want to delete
> >
> > On Aug 12, 9:05 pm, "Kenton Varda" <[EMAIL PROTECTED]> wrote:
> >
> > > For C++, use the RepeatedField interfaces:
> http://code.google.com/apis/protocolbuffers/docs/reference/cpp/google...
> >
> > > You can get a mutable pointer to a RepeatedField or RepeatedPtrField
> that
> > > represents a field named "foo" by calling the message's "mutable_foo()"
> > > accessor.
> >
> > > For Python, I think you currently have to clear the field and
> repopulate it.
> >
> > > On Tue, Aug 12, 2008 at 12:22 PM, <[EMAIL PROTECTED]> wrote:
> >
> > > > Python or C++
> >
> > > > On Aug 12, 8:12 pm, "Kenton Varda" <[EMAIL PROTECTED]> wrote:
> > > > > In what language?
> >
> > > > > On Tue, Aug 12, 2008 at 12:11 PM, <[EMAIL PROTECTED]> wrote:
> >
> > > > > > Given an object containing a repeated field, is it possible to
> remove
> > > > > > some arbitrary element from that list?
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: How to properly implement RpcChannel::CallMethod

[Kenton Varda]

You should send the method's name along with the request.  The receiver can
then look up the MethodDescriptor from the service's ServiceDescriptor,
using FindMethodByName().
You might even encode your messages like this:

message RpcRequest {
  required string method_name = 1;

  // The serialized request message.
  required bytes request_data = 2;
}

On Tue, Sep 16, 2008 at 8:54 AM, <[EMAIL PROTECTED]> wrote:

>
> Hi!
>
> I have some problems setting up a simple IPC system using protocol
> buffers for sending the parameters and results between two processes.
> I have written a protocol buffers service with a function and a simple
> IPC system that can transfer raw byte messages between two processes.
> My problem is understanding how my RpcChannel::CallMethod should be
> implemented. I was expecting it was as easy to encode the message to
> raw bytes and pick it up in the other process and decode it. However,
> it turned out that the MethodDescriptor is not encoded in the Message
> and it's needed in the other end to route the message to the right
> function. I found no functionality to serialize the MethodDescriptor
> to a byte stream.
>
> Of course I can come up with encodings for the method descriptor,
> either sending the name or the index should work. Then I can just put
> it first in the buffer that's transferred to the other process, but
> that means I lose the nice feature of letting protocol buffers
> handling unicode string encoding, endianness of integers etc. One
> approach would be to create a meta message containing both the method
> descriptor and the encoded message as a variable size byte array in
> one message.
> I somehow think I'm missing the point here, how is
> RpcChannel::CallMethod supposed to work?
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Protocol Buffers can't use in visual studio 2003 ?

[Kenton Varda]

I don't think DLLs or LIBs created with a never version of MSVC will work
with MSVC 2003.  Protobufs use STL heavily in the interface, and I think the
STL classes change from version to version.
On Tue, Sep 16, 2008 at 4:44 AM, Niall <[EMAIL PROTECTED]> wrote:

>
> I get the same runtime error with MSVC 03.
>
> Could some make available the required *.dlls and *.libs which I could
> download and test with my IDE? This should tell whether it's the dll/
> libs that MSVC 2003 makes are the root of the problem or whether it's
> with program that's linking to them? Right?
>
> > you can create a new project under the solution of vsprojects, then
> select
> > the dependencies to lib*, then it will work.
> I'm not certain what you mean there...
>
> Thanks, Niall
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: [PATCH] Add options to change generated file extensions to C++ generator.

[Kenton Varda]

On Mon, Sep 15, 2008 at 6:14 PM, Leandro Lucarella <[EMAIL PROTECTED]> wrote:

> I understand your concern but I don't think looking after the user should
> stay in the way of flexibility. I surely prefere flexibility than people
> looking after me just in case I can't meassure the consequences.


Sorry, my experience is that if I allow people to shoot themselves in the
foot, some people will, and then they will blame me for letting them, and
then I have to devise and maintain additional hacks that allow them to fix
their problem without tons of work.  I'm not saying that I think *you* would
shoot yourself in the foot, but *someone* will.


> And maybe I'm confused, but what's the risk here? Using some precompiled,
> stuff? Because if use some some third party .proto, and I generate it
> using a different extension, I can't think how that could break anything.


Depends.  descriptor.proto, for example, is provided as part of libprotobuf
itself.  If you want to import it, you would not be generating it yourself;
you would be using the header which is installed along with the other public
headers.  With your scheme, you won't be able to use descriptor.proto.  This
also means you won't be able to define custom options in version 2.0.2, as
soon as I get that code submitted.


> Ok, no problem, I'll just stay with my current hacks (mass rename + sed),
> because I don't want to maintain my own fork =)


This sounds fine to me.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Protocol Buffers can't use in visual studio 2003 ?

[Kenton Varda]

Even if only the implementation changes -- specifically, the class's memory
layout -- that will cause a binary incompatibility, especially since STL is
all templates.  It seems the debug vs. release versions of MSVC's STL are
not even binary compatible, so being compatible between two versions of MSVC
seems doubtful.

On Tue, Sep 16, 2008 at 10:40 AM, Niall <[EMAIL PROTECTED]> wrote:

>
> Thanks for the reply. You're probably right... but I'm still not
> convinced it wouldn't work. I couldn't imagine the STL's interface
> changing. The implementation, sure, but I'd be surprised if the
> interface would (There'd need to be a real good reason with all that
> reference paperwork needing changing).
>
> > Protobufs use STL heavily in the interface, and I think
> > the STL classes change from version to version.
> Sure, but if I can compile the DLL's and LIB's anyway I can't see this
> being a problem.
>
> This is just gut feeling, though. I can't find anything online to say
> either way which would happen. Would it be worth trying anyway? If
> possible.
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Protocol Buffers can't use in visual studio 2003 ?

[Kenton Varda]

On Tue, Sep 16, 2008 at 12:26 PM, Niall <[EMAIL PROTECTED]> wrote:

> But, suppose you have the dll, a lib and the required header files.
> You have all the things you need. The functions, their location, how
> they are called. Surely the DLL would worry about itself. It's a
> separate moster altogether to the source code you're trying to
> compile, no? The DLL doesn't have to interface to anything. No
> headers, no nothing. It only worries about itself, right? Just needs
> to know what in the dll to execute, and that's managed by the lib
> file, correct? All you should need are the entry/exit points I'd've
> though. Though my understanding of DLLs is by no means infallible.


If the STL implementations used by the DLL and its client are not
compatible, then passing STL objects between the two will fail.  Microsoft
even says explicitly in their docs that you cannot pass STL objects across
DLL interfaces.  I've ignored this in protocol buffers because adhering to
this rule would require huge changes to the interface (which was originally
designed on Linux which has no such problems), and in practice it works so
long as you compile the DLL with the same STL version as the rest of your
app.  But, this is why we cannot distribute precompiled binaries -- they
won't be compatible between versions of MSVC.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Protocol Buffers can't use in visual studio 2003 ?

[Kenton Varda]

I think you'll have to run it in a debugger and try to figure out the cause
of the crash you're seeing.  I don't have a copy of MSVC '03, but if you
send me the stack trace I could try to guess what's going on.

On Tue, Sep 16, 2008 at 1:03 PM, Niall <[EMAIL PROTECTED]> wrote:

>
> Finally. I see what you mean. Thanks for your help and clarification.
> I wasn't aware of those issues. Thanks again!
>
> Any suggestions on how I could use protobufs with MSVC '03? Without
> the runtime errors when calling SerializeToOstream, that is. Lenjoy,
> in the second post, said:
>
> >you can create a new project under the solution of vsprojects, then
> >select the dependencies to lib*, then it will work.
>
> But I don't know what that means... I tried a couple of things, like
> adding a new project to the solution I used in generating the DLLs,
> libs etc. But with the same result.
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Protocol Buffers can't use in visual studio 2003 ?

[Kenton Varda]

It sounds like output_ is not a valid pointer.  What's the full stack trace?

On Tue, Sep 16, 2008 at 1:25 PM, Niall <[EMAIL PROTECTED]> wrote:

>
> For some reason the debugger didn't even come to mind...
>
> OK. It crashes at the 'bool
> OstreamOutputStream::CopyingOstreamOutputStream::Write( const void*
> buffer, int size )' function call. I think it's in the output_->write
> function, but I'm not sure. If I try to step into output_->write it
> crashes... without stepping into it... Not sure what that means.
> What's the stack trace? Or I suppose how  do I find it? Sounds fairly
> self explanatory.
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Protocol Buffers can't use in visual studio 2003 ?

[Kenton Varda]

It's the list of function calls which lead to the point of the crash.

On Tue, Sep 16, 2008 at 1:48 PM, Niall <[EMAIL PROTECTED]> wrote:

>
> What's the stack trace?
>
> If it's the debugging output this is what I have:
>
> 'protobuf_test.exe': Loaded 'C:\Documents and Settings\Niall\My
> Documents\Visual Studio Projects\protobuf_test\Release
> \protobuf_test.exe', Symbols loaded.
> 'protobuf_test.exe': Loaded 'C:\WINDOWS\system32\ntdll.dll', No
> symbols loaded.
> 'protobuf_test.exe': Loaded 'C:\WINDOWS\system32\kernel32.dll', No
> symbols loaded.
> 'protobuf_test.exe': Loaded 'C:\Documents and Settings\Niall\My
> Documents\Visual Studio Projects\protobuf_test\Release
> \libprotobuf.dll', Symbols loaded.
> 'protobuf_test.exe': Loaded 'C:\WINDOWS\system32\msvcp71.dll', No
> symbols loaded.
> 'protobuf_test.exe': Loaded 'C:\WINDOWS\system32\msvcr71.dll', No
> symbols loaded.
> 'protobuf_test.exe': Loaded 'C:\WINDOWS\system32\shimeng.dll', No
> symbols loaded.
> 'protobuf_test.exe': Unloaded 'C:\WINDOWS\system32\shimeng.dll'
> First-chance exception at 0x7c901010 in protobuf_test.exe: 0xC005:
> Access violation reading location 0x5075.
> Unhandled exception at 0x7c901010 in protobuf_test.exe: 0xC005:
> Access violation reading location 0x5075.
> The program '[3000] protobuf_test.exe: Native' has exited with code 0
> (0x0).
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: complie error

[Kenton Varda]

I believe I've seen people report these errors when using very old versions
of GCC.  What version of GCC / what OS are you using?

On Wed, Sep 17, 2008 at 4:20 AM, dd shan <[EMAIL PROTECTED]> wrote:

>
>  ./configure
> checking for a BSD-compatible install... /usr/bin/install -c
> checking whether build environment is sane... yes
> checking for gawk... gawk
> checking whether make sets $(MAKE)... yes
> checking for gcc... gcc
> checking for C compiler default output file name... a.out
> checking whether the C compiler works... yes
> checking whether we are cross compiling... no
> checking for suffix of executables...
> checking for suffix of object files... o
> checking whether we are using the GNU C compiler... yes
> checking whether gcc accepts -g... yes
> checking for gcc option to accept ANSI C... none needed
> checking for style of include used by make... GNU
> checking dependency style of gcc... gcc3
> checking for g++... g++
> checking whether we are using the GNU C++ compiler... yes
> checking whether g++ accepts -g... yes
> checking dependency style of g++... gcc3
> checking build system type... i686-pc-linux-gnu
> checking host system type... i686-pc-linux-gnu
> checking for a sed that does not truncate output... /bin/sed
> checking for egrep... grep -E
> checking for ld used by gcc... /usr/bin/ld
> checking if the linker (/usr/bin/ld) is GNU ld... yes
> checking for /usr/bin/ld option to reload object files... -r
> checking for BSD-compatible nm... /usr/bin/nm -B
> checking whether ln -s works... yes
> checking how to recognise dependent libraries... pass_all
> checking how to run the C preprocessor... gcc -E
> checking for ANSI C header files... yes
> checking for sys/types.h... yes
> checking for sys/stat.h... yes
> checking for stdlib.h... yes
> checking for string.h... yes
> checking for memory.h... yes
> checking for strings.h... yes
> checking for inttypes.h... yes
> checking for stdint.h... yes
> checking for unistd.h... yes
> checking dlfcn.h usability... yes
> checking dlfcn.h presence... yes
> checking for dlfcn.h... yes
> checking how to run the C++ preprocessor... g++ -E
> checking for g77... g77
> checking whether we are using the GNU Fortran 77 compiler... yes
> checking whether g77 accepts -g... yes
> checking the maximum length of command line arguments... 32768
> checking command to parse /usr/bin/nm -B output from gcc object... ok
> checking for objdir... .libs
> checking for ar... ar
> checking for ranlib... ranlib
> checking for strip... strip
> checking if gcc supports -fno-rtti -fno-exceptions... yes
> checking for gcc option to produce PIC... -fPIC
> checking if gcc PIC flag -fPIC works... yes
> checking if gcc static flag -static works... yes
> checking if gcc supports -c -o file.o... yes
> checking whether the gcc linker (/usr/bin/ld) supports shared
> libraries... yes
> checking whether -lc should be explicitly linked in... no
> checking dynamic linker characteristics... GNU/Linux ld.so
> checking how to hardcode library paths into programs... immediate
> checking whether stripping libraries is possible... yes
> checking if libtool supports shared libraries... yes
> checking whether to build shared libraries... yes
> checking whether to build static libraries... yes
> configure: creating libtool
> appending configuration tag "CXX" to libtool
> checking for ld used by g++... /usr/bin/ld
> checking if the linker (/usr/bin/ld) is GNU ld... yes
> checking whether the g++ linker (/usr/bin/ld) supports shared
> libraries... yes
> checking for g++ option to produce PIC... -fPIC
> checking if g++ PIC flag -fPIC works... yes
> checking if g++ static flag -static works... yes
> checking if g++ supports -c -o file.o... yes
> checking whether the g++ linker (/usr/bin/ld) supports shared
> libraries... yes
> checking dynamic linker characteristics... GNU/Linux ld.so
> checking how to hardcode library paths into programs... immediate
> appending configuration tag "F77" to libtool
> checking if libtool supports shared libraries... yes
> checking whether to build shared libraries... yes
> checking whether to build static libraries... yes
> checking for g77 option to produce PIC... -fPIC
> checking if g77 PIC flag -fPIC works... yes
> checking if g77 static flag -static works... yes
> checking if g77 supports -c -o file.o... yes
> checking whether the g77 linker (/usr/bin/ld) supports shared
> libraries... yes
> checking dynamic linker characteristics... GNU/Linux ld.so
> checking how to hardcode library paths into programs... immediate
> checking for ANSI C header files... (cached) yes
> checking fcntl.h usability... yes
> checking fcntl.h presence... yes
> checking for fcntl.h... yes
> checking for inttypes.h... (cached) yes
> checking limits.h usability... yes
> checking limits.h presence... yes
> checking for limits.h... yes
> checking for stdlib.h... (cached) yes
> checking for unistd.h... (cached) yes
> checking for working memcmp... yes
> checking for working strtod... yes
> chec

Re: Protocol Buffers can't use in visual studio 2003 ?

[Kenton Varda]

Ugh.  More evidence that libprotobuf should be statically-linked, not a DLL.
Niall:  Maybe this explains your problem.  Can you try compiling static libs
only and see if that fixes anything?

On Tue, Sep 16, 2008 at 9:31 PM, mcdowella
<[EMAIL PROTECTED]>wrote:

>
> There are at least two problems with using STL and even ordinary code
> across DLL boundaries with MSVC, both of which can be hard to get
> round. One is that if you link statically (select the non-DLL version
> of the run time libraries) the code inside the DLL will use a
> different storage pool from code outside the DLL, so store allocated
> on one side of the boundary and freed on the other side of the
> boundary will break the storage pool that tries to free it. This will
> also happen if the run time library selected by the code in the DLL is
> not identical to that selected by the program using it. Another
> problem is that much of the STL implementation selected by Microsoft
> does not use 0 as a special pointer value to mark the ends of linked
> lists and such things. It tends to allocate a little chunk of memory
> and use that instead. These marker values are not exported across the
> DLL boundary, so somebody reading an STL structure created on the
> other side of the DLL boundary will not recognise the end list marker
> and will fall of the end and start using rubbish pointers.
>
> See
> http://support.microsoft.com/kb/172396
> http://support.microsoft.com/kb/168958
>
> http://www.deez.info/sengelha/2006/03/03/c-stl-dlls-and-buggy-optimizations/
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Implementing Rpc service in Java -- need help!

[Kenton Varda]

Use the callback:

  Message response =
responsePrototype.newBuilderForType.mergeFrom(data).build();
  done.run(response);

On Thu, Sep 18, 2008 at 12:07 PM, Java Jogger <[EMAIL PROTECTED]> wrote:

>
> I just started looking at implementing RPC service in Java.  In my
> implementation class of RpcChannel interface, one method to be
> implemented defined as:
>
> public void callMethod(Descriptors.MethodDescriptor method,
> RpcController controller, Message request,
>   Message responsePrototype,
> RpcCallback done) {
>
>// my implementation goes here...
>
> }
>
> My understanding is the implementation of the above method is where
> the remote request/response should be hooked in.  The question is: how
> does it return the response message and make it available to the
> caller of this method?
>
> From the C++ example code in Language Guide, the responseProtoType is
> passed by reference (pointer), thus making it possible to carry the
> output of the method.  Since arguments to java method call are always
> passed by value, how does one get hold of the response message after
> "callMethod" returns successfully.
>
> Deeply puzzled and need help.
>
> Jie
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Emulate hash_map using map when necessary.

[Kenton Varda]

Any comments on the code?  If not, I'll go ahead and submit it.

On Thu, Sep 18, 2008 at 1:59 PM, <[EMAIL PROTECTED]> wrote:

> Yes, is working.
>
> ==> Test ok on Tru64 (with some other changed in the code for the
> compiler)
> ==> Test ok on Windows Visual Studio 2005
>
> Issue 5682 can be ignored...
>
>
> http://codereview.appspot.com/5683
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Implementing Rpc service in Java -- need help!

[Kenton Varda]

Yes, currently that's how it works.  C++ and Python work this way too.

On Thu, Sep 18, 2008 at 1:43 PM, Jie Yao <[EMAIL PROTECTED]> wrote:

>
> I see.  The java RPC implementation has to be async via call back, is
> that right?
>
> Jie
>
> On Sep 18, 2:15 pm, "Kenton Varda" <[EMAIL PROTECTED]> wrote:
> > Use the callback:
> >
> >   Message response =
> > responsePrototype.newBuilderForType.mergeFrom(data).build();
> >   done.run(response);
> >
> > On Thu, Sep 18, 2008 at 12:07 PM, Java Jogger <[EMAIL PROTECTED]> wrote:
> >
> > > I just started looking at implementing RPC service in Java.  In my
> > > implementation class of RpcChannel interface, one method to be
> > > implemented defined as:
> >
> > > public void callMethod(Descriptors.MethodDescriptor method,
> > > RpcController controller, Message request,
> > >   Message responsePrototype,
> > > RpcCallback done) {
> >
> > >// my implementation goes here...
> >
> > > }
> >
> > > My understanding is the implementation of the above method is where
> > > the remote request/response should be hooked in.  The question is: how
> > > does it return the response message and make it available to the
> > > caller of this method?
> >
> > > From the C++ example code in Language Guide, the responseProtoType is
> > > passed by reference (pointer), thus making it possible to carry the
> > > output of the method.  Since arguments to java method call are always
> > > passed by value, how does one get hold of the response message after
> > > "callMethod" returns successfully.
> >
> > > Deeply puzzled and need help.
> >
> > > Jie
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Implementing Rpc service in Java -- need help!

[Kenton Varda]

The client needs to check the RpcController for errors.
In retrospect, there probably should have been a second method on
RpcCallback called in case of error.

On Thu, Sep 18, 2008 at 2:39 PM, Jie Yao <[EMAIL PROTECTED]> wrote:

> How do I return the error status to client if
>
> done.run(response)
>
> returns only the expected response under successful scenarios.
>
> Jie
>
> ----- Original Message 
> From: Kenton Varda <[EMAIL PROTECTED]>
> To: Jie Yao <[EMAIL PROTECTED]>
> Cc: Protocol Buffers 
> Sent: Thursday, September 18, 2008 4:29:35 PM
> Subject: Re: Implementing Rpc service in Java -- need help!
>
> Yes, currently that's how it works.  C++ and Python work this way too.
>
> On Thu, Sep 18, 2008 at 1:43 PM, Jie Yao <[EMAIL PROTECTED]> wrote:
>
>>
>> I see.  The java RPC implementation has to be async via call back, is
>> that right?
>>
>> Jie
>>
>> On Sep 18, 2:15 pm, "Kenton Varda" <[EMAIL PROTECTED]> wrote:
>> > Use the callback:
>> >
>> >   Message response =
>> > responsePrototype.newBuilderForType.mergeFrom(data).build();
>> >   done.run(response);
>> >
>> > On Thu, Sep 18, 2008 at 12:07 PM, Java Jogger <[EMAIL PROTECTED]>
>> wrote:
>> >
>> > > I just started looking at implementing RPC service in Java.  In my
>> > > implementation class of RpcChannel interface, one method to be
>> > > implemented defined as:
>> >
>> > > public void callMethod(Descriptors.MethodDescriptor method,
>> > > RpcController controller, Message request,
>> > >   Message responsePrototype,
>> > > RpcCallback done) {
>> >
>> > >// my implementation goes here...
>> >
>> > > }
>> >
>> > > My understanding is the implementation of the above method is where
>> > > the remote request/response should be hooked in.  The question is: how
>> > > does it return the response message and make it available to the
>> > > caller of this method?
>> >
>> > > From the C++ example code in Language Guide, the responseProtoType is
>> > > passed by reference (pointer), thus making it possible to carry the
>> > > output of the method.  Since arguments to java method call are always
>> > > passed by value, how does one get hold of the response message after
>> > > "callMethod" returns successfully.
>> >
>> > > Deeply puzzled and need help.
>> >
>> > > Jie
>> >>
>>
>
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Emulate hash_map using map when necessary.

[Kenton Varda]

submitted.  Thanks for prodding me into fixing this.  :)

On Fri, Sep 19, 2008 at 6:41 AM, <[EMAIL PROTECTED]> wrote:

>  You can submit it, the code is correct.
>
>
>
> Thank you.
>
>
>
>
>
>
>
>
>  --
>
> *De :* Kenton Varda [mailto:[EMAIL PROTECTED]
> *Envoyé :* 18 septembre 2008 17:13
> *À :* [EMAIL PROTECTED]; Choinière, Vincent; protobuf@googlegroups.com
> *Objet :* Re: Emulate hash_map using map when necessary.
>
>
>
> Any comments on the code?  If not, I'll go ahead and submit it.
>
> On Thu, Sep 18, 2008 at 1:59 PM, <[EMAIL PROTECTED]> wrote:
>
> Yes, is working.
>
> ==> Test ok on Tru64 (with some other changed in the code for the
> compiler)
> ==> Test ok on Windows Visual Studio 2005
>
> Issue 5682 can be ignored...
>
>
>
> http://codereview.appspot.com/5683
>
>
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Multiple proto buffers on the same wire

[Kenton Varda]

You'll have to stick some sort of one-byte tag on the front of the message
to identify which type it is.

On Fri, Sep 19, 2008 at 11:05 AM, BeamMeUpScotty <[EMAIL PROTECTED]>wrote:

>
> We're looking at a scenario where there could possibly be many
> different messages on the same wire.  For example, DeviceA and DeviceB
> communicate with a PC.  Both DeviceA and DeviceB use the same
> functional network, and they also use different Proto messages.  The
> question that we're wrestling with is how does the PC differentiate
> between the different Proto implementations?  I've done tests using
> the generated Python sources and looked at the serialized messages.
> (btw, we're VERY concerned with wire format size)  Looks like if I
> have multiple messages in the same proto, there's no way that I can
> tell the difference between them from looking at the wire format.  We
> could nest the messages, but this adds a bytes to the implementation.
> Is there another way?
>
> Scott Pavetti
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Any small sample Class Service code for Java

[Kenton Varda]

On Fri, Sep 19, 2008 at 9:50 AM, Mars <[EMAIL PROTECTED]> wrote:

>
> just want to find some sample code for Class Service:
>
> http://code.google.com/apis/protocolbuffers/docs/reference/python/google.protobuf.service.Service-class.html


That's the Python version.  Shouldn't you be looking at the Java version?

http://code.google.com/apis/protocolbuffers/docs/reference/java/com/google/protobuf/Service.html

for example:
> given .proto .java files below,
> how to use simply use them in some basic Java network program, such as
> ServerSocket or DatagramSocket?


Maybe look at some of the RPC implementations people are working on:

http://code.google.com/p/protobuf/wiki/RPCImplementations


>
>
> =
> # .proto
> message Req1 {
>  required int32 id = 1;
> }
>
> message Res1 {
>  required string name = 1;
> }
>
> service Serv1 {
>  rpc Func1(Req1) returns (Res1) {
>option name = "name1";
>  }
> }
>
> ==
> // .java
> public class MyServ implements Serv1 {
>
>  public EmptyMessage Func1(RPC rpc, Req1 req) throws RpcException {
>//...
>  }
> }
>
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Does SerializeToXxx() guarantee a call to ByteSize()?

[Kenton Varda]

You probably shouldn't rely on this.  Some of the serialization methods will
guarantee this, some won't.  What you can do is call ByteSize() yourself,
and then use SerializeWithCachedSizes(), which requires that ByteSize() was
already called and won't call it again.  You'll have to do a small amount of
extra setup (allocating a ZeroCopyOutputStream and a CodedOutputStream), but
it's pretty straightforward -- you can just copy the code out of message.cc.
Another thing that you could do which is specific to your use case:  Instead
of allocating a byte array to serialize into, just use a string:

string bytes;
pkt.SerializeToString(&bytes);
send(fd, bytes.data(), bytes.size(), 0);

This is better anyway since it won't break if your messages become larger
than SIZE.

On Fri, Sep 19, 2008 at 2:12 PM, Leandro Lucarella <[EMAIL PROTECTED]> wrote:

>
> Hi.
>
> I guess SerializeToXxx() methods internally calls to ByteSize(), so after
> a serialization, I can call GetCachedSize().
>
> My question is, first, is this true? =) And second, can I rely on this?
>
> The use case is this:
>
> pkt.SerializeToArray(buffer, SIZE);
> send(fd, buffer, pkt.GetCachedSize(), 0);
>
> Is this safe or should I use pkt.ByteSize() instead?
>
>
> Thank you.
>
> --
> Leandro Lucarella (luca) | Blog colectivo:
> http://www.mazziblog.com.ar/blog/
>
> 
> GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
>
> 
> Soñé que tenia un caballo
> Que me trataba mejor que vos
> Tenia tan buena onda con ella
> Era mi yegua, mucho mas que vos
>
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Unable to Complie and Run Example

[Kenton Varda]

Use the Makefile.  If you don't have make, then don't worry about it and
compile and run the Java files like you would any Java executable class.
 add_person_java is really just an alias for "java AddPerson".

On Mon, Sep 22, 2008 at 8:16 AM, Adrian <[EMAIL PROTECTED]> wrote:

>
> I was able to install protobuf-2.0.1 using maven and netbeans 6.1
> I successfully generated protobuf-java-2.0.1.jar
>
> I copied protoc into the example directory
> I ran "protoc addressbook.proto --java_out=" from the windows command
> line to generate com/example/tutorial/AddressBookProtos.java
>
> I don't understand how to create the executable files
> add_person_java
> list_people_java
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Proto file format problem

[Kenton Varda]

I think the documentation should be changed; I'll fix that.  However,
negative enum values should probably be discouraged as they will always be
encoded as 10 bytes on the wire.

On Mon, Sep 22, 2008 at 3:08 AM, Chris <[EMAIL PROTECTED]> wrote:

>
> There is a disagreement between the documentation at
> http://code.google.com/apis/protocolbuffers/docs/proto.html#enum
> and the behavior of proto.
>
> The documentation states that
> "Enumerator constants must be in the range [0, 2147483647]."
>
> But the protoc program allows negative values.
>
> I happened to write my Haskell version according the documentation, and
> noticed this when the unittest.proto defined SPARSE constants there were
> negative.  My code rejected this as invalid.
>
> Should the documentation be changed (and my Haskell code) or the protoc
> program?
>
> Thanks,
>  Chris
>
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Does SerializeToXxx() guarantee a call to ByteSize()?

[Kenton Varda]

On Fri, Sep 19, 2008 at 7:39 PM, Leandro Lucarella <[EMAIL PROTECTED]> wrote:

>
> Kenton Varda, el 19 de septiembre a las 16:12 me escribiste:
> > You probably shouldn't rely on this.  Some of the serialization methods
> will
> > guarantee this, some won't.  What you can do is call ByteSize() yourself,
> > and then use SerializeWithCachedSizes(), which requires that ByteSize()
> was
> > already called and won't call it again.  You'll have to do a small amount
> of
> > extra setup (allocating a ZeroCopyOutputStream and a CodedOutputStream),
> but
> > it's pretty straightforward -- you can just copy the code out of
> message.cc.
> > Another thing that you could do which is specific to your use case:
>  Instead
> > of allocating a byte array to serialize into, just use a string:
> >
> > string bytes;
> > pkt.SerializeToString(&bytes);
> > send(fd, bytes.data(), bytes.size(), 0);
>
> Thanks for the tip. I didn't do it this way because I use the same buffer
> for receiving, and I can't use std::string for receiving.
>
> I could use two buffers, but now I wonder how std::string is used when
> serializing. Is the ByteSize() precalculated and space is reserve()ed in
> the string or it's incrementally expanded as it's serialized?


The size is precomputed.  Though, StringOutputStream (a subclass of
ZeroCopyOutputStream) can be used without a precomputed size, in which case
it grows the string as needed.


>
>
> > This is better anyway since it won't break if your messages become larger
> > than SIZE.
>
> This is not a problem to me, all messages are really short and I have
> a physical size limit imposed by the lower level (I'm using TIPC RDM
> sockets) anyway. So I have to drop the message (or abort the program) if
> it's size is longer than this limit, and I have guaranteed that I can't
> possibly receive a message that doesn't fit my buffer.
>
> --
> Leandro Lucarella (luca) | Blog colectivo:
> http://www.mazziblog.com.ar/blog/
>
> 
> GPG Key: 5F5A8D05 (F8CD F9A7 BF00 5431 4145  104C 949E BFB6 5F5A 8D05)
>
> 
> HOMBRE DESNUDO AMENAZA A LOS VECINOS CON UNA "KATANA" DESDE SU BALCON
>-- Crónica TV
>
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Issue dynamically reloading libprotobuf?

[Kenton Varda]

Unfortunately, the protocol buffer library was designed in an environment
where all code is statically-linked, and it probably won't work so well with
dynamically-loaded code.  The problem is that at start-up, descriptors are
built for each compiled-in message type and registered in a singleton
registry.  There is no code that removes the descriptors on shutdown.  In
your case, it looks like libprotobuf.so is staying loaded after your plugin
is shut down.  When it starts back up again, it tries to re-register its
descriptors, but this fails because the descriptors already exist.
You might be able to work around this by statically linking libprotobuf.a
into your .so.  That way, each time you load the library, a new registry
will be created.  You'll still have some memory leaks, though, as there are
a lot of data structures which are not properly destroyed when libprotobuf
is shut down (again, because the design assumed that the code would only be
unloaded when the process exits anyway).

It would probably be possible to rework parts of the library to make it more
friendly to plugins, if someone wants to spend time on this, but it will
take some effort.

On Mon, Sep 22, 2008 at 8:53 PM, <[EMAIL PROTECTED]> wrote:

>
> Hi. I have an plug-in based application under Linux that dynamically
> links in libprotobuf.so. The app's plug-ins also link in the library.
> This works fine when a plug-in is first dynamically loaded via
> dl_open(). However, if the plug-in is removed dl_close() and then re-
> opened, I get the following error:
>
>  libprotobuf ERROR google/protobuf/descriptor.cc:1778] Invalid proto
> descriptor for file "component_ec.proto":
> libprotobuf ERROR google/protobuf/descriptor.cc:1781]
> component_ec.proto: A file with this name is already in the pool.
>
>
> It appears that there is some unwanted state associated with open/
> close. Has anyone seen this type of issue? Any ideas?
>
> Thanks,
> Aaron
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: protobuf-perlxs

[Kenton Varda]

I've added this to the wiki.

On Mon, Sep 22, 2008 at 3:50 PM, Dave Bailey <[EMAIL PROTECTED]> wrote:

>
> Hi,
>
> I've uploaded my Perl/XS code generator for Protocol Buffers:
>
> http://code.google.com/p/protobuf-perlxs
>
> This is a work in progress, of course.  I have gotten this to work on
> my Linux system, but have not tested it on any other architectures.
>
> -dave
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Patch to eliminate warning when building

[Kenton Varda]

I've submitted this patch (or something equivalent, anyway).  Sorry for the
delay.

On Sat, Sep 6, 2008 at 12:08 AM, Mats Kindahl <[EMAIL PROTECTED]> wrote:

> Kenton Varda wrote:
> > I don't think GCC generates this warning with -Wall and -Werror alone.
> >  Maybe this is the result of -pedantic?  (Someone else on your team said
> you
> > use that flag).  I'll be happy to make this change but you might want to
> > consider disabling -pedantic when compiling .pb.cc files.  Otherwise,
> future
> > changes may once again cause problems for you.
>
> Yes, we are using -pedantic as well.
>
> Best wishes,
> Mats Kindahl
>
> >
> > On Thu, Sep 4, 2008 at 10:29 AM, Mats Kindahl <[EMAIL PROTECTED]> wrote:
> >
> >> Hi!
> >>
> >> When building Drizzle, we use -Wall and -Werror, which causes some
> warnings
> >> in
> >> code generated by protobuf to abort the build. The patch below
> eliminates
> >> the
> >> problem by adding the call to the default constructor.
> >>
> >> Just my few cents,
> >> Mats Kindahl
> >>
> >> --
> >> diff -r 70188e371c05 src/google/protobuf/compiler/cpp/cpp_message.cc
> >> --- a/src/google/protobuf/compiler/cpp/cpp_message.cc   Thu Sep 04
> 15:09:20
> >> 2008
> >> +0200
> >> +++ b/src/google/protobuf/compiler/cpp/cpp_message.cc   Thu Sep 04
> 19:04:27
> >> 2008
> >> +0200
> >> @@ -721,7 +721,8 @@ GenerateInitializerList(io::Printer* pri
> >>   }
> >>
> >>   printer->Print(
> >> -"_cached_size_(0)");
> >> +"$base_class$(), _cached_size_(0)",
> >> +"base_class", "::google::protobuf::Message");
> >>
> >>   // Write the initializers for each field.
> >>   for (int i = 0; i < descriptor_->field_count(); i++) {
> >>
> >> > >>
> >>
> >
>
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Question with ASCII format backward compatibility

[Kenton Varda]

Please re-send your question to the appropriate Google-internal mailing
list.  protobuf@googlegroups.com is populated by people who are not Google
employees and is not an appropriate place to discuss internal code or
documentation.  Luckily your message doesn't contain anything sensitive, but
since you're discussing the version 1 implementation of Protocol Buffers,
which is not public, and posting links to internal documents, you should
really be using an internal list.

On Tue, Sep 23, 2008 at 3:20 PM, svenkata <[EMAIL PROTECTED]> wrote:

>
> In one place in production code, we used ToASCII and ParseASCII APIs
> to encode and decode a protocol message.
>
> The previous implementer had a comment on the usage of ASCII:
>
> // The protocol buffer is converted into an
> // ascii version so that a big endian transmitter can talk to a little
> // endian receiver and vice versa. This will change once the protocol
> // buffer implementation is ported to work with a neutral wire format.
>
> First question: Does the protocol buffer implementation support
> neutral format now? In other words, does two machines with different
> endians talk to each other (encode and decode) via protocol buffers.
>
> I added couple of new optional field to the old protocol buffers. I'm
> seeing a problem with backward compatibility.  That is old code is
> complaining that the new fields are unrecognized:
>
> 09-21-2008 06:25:12 XX ygs04f1s2i30 user.err lcm[846]: tagmapper-
> ascii.cc:388] Did not find tag type in message type IfmNDMessage
> 09-21-2008 06:25:12 XX ygs04f1s2i30 user.err lcm[846]: tagmapper-
> ascii.cc:254] input error at line: 7 column: 1: ParseSlot(group_name)
> failed
>
> Here is what I found after close reading:
>
> https://www.corp.google.com/eng/designdocs/infrastructure/protocol-buffer.html
>
> Extending a Protocol
> ...
> Also, make any new elements and groups be optional or repeated, so
> that messages generated by old code can be parsed by new code. As
> noted above, the old code will ignore any unknown elements in the
> message.
> ...
>
> ASCII Format
> ...
> Support is provided for converting from ASCII to binary with the
> ParseASCII method. As this support is slow and does not allow
> evolution of protocol types its use is discouraged for production
> services.
> ...
>
> Second question: What does it mean "does not allow evolution of
> protocol types" in the above doc? Does it mean ToASCII and ParseASCII
> methods are not backward compatible?
>
> Third question: How should I correct this now? The protocol message to
> which I added optional fields (call it FooMessage) is embedded inside
> other message (call it BarMessage) as a required field of type string.
> Because it is a string, the previous author used ToASCII and
> ParseASCII. Now I can't change this string to FooMessage because it is
> a required field in production.
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

License changing to BSD.

[Kenton Varda]

Hi all,

We've decided to change the license for Protocol Buffers from Apache 2.0 to
BSD.  In general we like the Apache 2.0 license better, but unfortunately it
is not compatible with GPLv2.  We decided that allowing GPLv2 projects to
use protocol buffers is more important, so we're switching.  I'll be
changing all the license headers in SVN tomorrow, probably, and version
2.0.2 should be released under the new license shortly thereafter.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: repeated fields

[Kenton Varda]

Playerlist also has methods:
const RepeatedPtrField& player() const;
RepeatedPtrField* mutable_player();

RepeatedPtrField is documented here:

http://code.google.com/apis/protocolbuffers/docs/reference/cpp/google.protobuf.repeated_field.html

It has methods which allow you to add an existing object without copying
(look under "Advanced memory management).  Of course, if you use these, then
the Player objects will become property of the Playerlist.  You will have to
be careful to take ownership back from the list later by repeatedly calling
ReleaseLast().

Also note that every protocol message has a CopyFrom() method which can be
used to copy an entire message object, so you would not have to call
set_id(), set_x(), etc. manually.

On Wed, Sep 24, 2008 at 8:01 AM, <[EMAIL PROTECTED]> wrote:

>
> Hello,
>
> I have a question concerning the repeated fields. I'll start with an
> example:
>
> message Player {
>  required int32 id = 1;
>  required int32 x =2;
>  required int32 y=3;
> }
>
> message Playerlist{
>  repeated Player player= 1;
> }
>
> I have a server and get Player's from different clients and hold them
> in a map (this map is updated all the time). After a certain time I
> want to get all Players, create a Playerlist and send it to all
> clients. The Playerlist however has just a method
>
> Player* add_player();
>
> I will have to copy all the Player's I've already created :
>
> Player* newP = playerlist.add_player();
> newP->set_id( playerFromMap->id() );
>
> Isn't there a better way to do this?
>
> Regards
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Major update submitted; 2.0.2 release soon

[Kenton Varda]

Hi all,
I just submitted revision 50 and 51 to SVN, including these changes:

  General
  * License changed from Apache 2.0 to New BSD.
  * It is now possible to define custom "options", which are basically
annotations which may be placed on definitions in a .proto file.
For example, you might define a field option called "foo" like so:
  import "google/protobuf/descriptor.proto"
  extend google.protobuf.FieldOptions {
optional string foo = 12345;
  }
Then you annotate a field using the "foo" option:
  message MyMessage {
optional int32 some_field = 1 [(foo) = "bar"]
  }
The value of this option is then visible via the message's
Descriptor:
  const FieldDescriptor* field =
MyMessage::descriptor()->FindFieldByName("some_field");
  assert(field->options().GetExtension(foo) == "bar");
This feature has been implemented and tested in C++ and Java.
Other languages may or may not need to do extra work to support
custom options, depending on how they construct descriptors.

  C++
  * Fixed some GCC warnings that only occur when using -pedantic.
  * Improved static initialization code, making ordering more
predictable among other things.
  * TextFormat will no longer accept messages which contain multiple
instances of a singular field.  Previously, the latter instance
would overwrite the former.
  * Now works on systems that don't have hash_map.

  Python
  * Strings now use the "unicode" type rather than the "str" type.
String fields may still be assigned ASCII "str" values; they will
automatically be converted.
  * Adding a property to an object representing a repeated field now
raises an exception.  For example:
  # No longer works (and never should have).
  message.some_repeated_field.foo = 1

This will probably become version 2.0.2.  Please try it out and let me know
if you have any problems.  I'll probably build a release candidate tomorrow
or Friday and a full release next week.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: xlC suffers the same can't-decide-which-template-to-use as Symbian

[Kenton Varda]

Thanks.  Can you send this to me via codereview.appspot.com?

On Wed, Sep 24, 2008 at 4:25 PM, Travis Pouarz <[EMAIL PROTECTED]> wrote:

> based on 2.0.1 release since I
> don't see any Subversion repos of trunk


What do you mean?

http://protobuf.googlecode.com/svn/trunk/

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Major update submitted; 2.0.2 release soon

[Kenton Varda]

I'll be updating the documentation to explain this better soon.  It's a bit
complicated.  Check out unittest_custom_options.proto for an example.  Let
me try to explain briefly...

First of all, note that UninterpretedOption is actually only ever used
within protoc, between the parsing phase and the cross-linking phase.
 During cross-linking, the UninterpretedOptions are converted to regular
extensions.  Users *never* use UninterpretedOption.  So, you could consider
UninterpretedOption to be an implementation detail of protoc and ignore it.
 Since you've written your own parser, you might want to use
UninterpretedOption as part of a similar design, but you might also want to
do something completely different; as long as the values eventually end up
as regular extensions, it's up to you.

OK, on to the syntax...

Let's take the example of a field option.  You see that the FieldOptions
message in descriptor.proto now allows extensions.  You might declare an
extension like:

  extend google.protobuf.FieldOptions {
optional string foo = 51234;
  }

Now when you declare a field, you can specify that the above extension
should be set in that field's descriptor with syntax like:

  message MyMessage {
optional int32 bar = 1 [(foo) = "baz"];
  }

The parentheses here say that the name within them is naming an extension,
not a regular field.  The name might even be qualified.  For example, if
"foo" had been declared in another package, e.g.:

  package pkg;
  extend google.protobuf.FieldOptions {
optional string foo = 51234;
  }

And now you wanted to use "foo" from some other package, you'd have to do
something like:

  package other;
  message MyMessage {
optional int32 bar = 1 [(pkg.foo) = "baz"];
  }

However, the qualification is not necessary if "foo" is in the same package.
 Just as with type names, extension names are resolved relative to the
current scope.

I think something that is confusing you is syntax like this:

  message MyMessage {
optional int32 bar = 1 [(foo).bar = "baz"];
  }

Here, "foo" is naming an extension whose type is a message, and bar is a
field within that message.  E.g., foo might be defined as:

  message FooType {
optional string bar = 1;
optional int32 baz = 2;
  }
  extend google.protobuf.FieldOptions {
optional FooType foo = 51234;
  }

It gets really complicated when FooType itself might have extensions.  Then
we might see something like "(foo).(bar) = 1".  This means that "bar" is an
extension of FooType.  I don't expect this to be used much, but it's
supported for consistency.

On Thu, Sep 25, 2008 at 9:18 AM, Chris <[EMAIL PROTECTED]> wrote:

> I can support the new options in Haskell right after I finish the unknown
> field loading/storing/writing.
> But work is busy right now, so I won't estimate how long I will take.
>
> I have more than a few questions about the new options below:
>
> Kenton Varda wrote:
>
>>  * It is now possible to define custom "options", which are basically
>>annotations which may be placed on definitions in a .proto file.
>>For example, you might define a field option called "foo" like so:
>>  import "google/protobuf/descriptor.proto"
>>  extend google.protobuf.FieldOptions {
>>optional string foo = 12345;
>>  }
>>Then you annotate a field using the "foo" option:
>>  message MyMessage {
>>optional int32 some_field = 1 [(foo) = "bar"]
>>  }
>>The value of this option is then visible via the message's
>>Descriptor:
>>  const FieldDescriptor* field =
>>MyMessage::descriptor()->FindFieldByName("some_field");
>>  assert(field->options().GetExtension(foo) == "bar");
>>This feature has been implemented and tested in C++ and Java.
>>Other languages may or may not need to do extra work to support
>>custom options, depending on how they construct descriptors.
>>
> I have just done the svn checkout to look at the new descriptor.proto file.
>
> In Haskell I generate code from descriptor.proto and parse the lexical
> tokens into
> precisely these message structures.  Your new descriptor.proto looks just
> as perfect
> to parse into as the old one, so I expect no trouble adding support for
> custom options.
>
> I see the meat of the addition is encoded as "UninterpretedOption":
>
>>
>> // A message representing a option the parser does not recognize. This
>> only
>> // appears in options protos created by the compiler::Parser class.
>> // DescriptorPool resolves these when building Descriptor objects.
>>

Re: License changing to BSD.

[Kenton Varda]

I also cannot speak for Google as a whole, or even the open source
department.  However, my personal opinion is that I would like users of
Protocol Buffers to have the freedom to use any license they want.  It would
be pretty disappointing to me if some project was unable to use Protocol
Buffers simply because the license was not compatible.

On Thu, Sep 25, 2008 at 2:33 AM, Ande Turner <[EMAIL PROTECTED]> wrote:

> A majority of people are using GPLv3 for newer projects.  Those with GPLv2
> can switch to GPLv3 easily in most cases.
>
> IF it was left as Apache it is GPLv3 compatible anyways.
>
> Why did you decide against GPLv3?
>
> 2008/9/24 Kenton Varda <[EMAIL PROTECTED]>
>
>> Hi all,
>>
>> We've decided to change the license for Protocol Buffers from Apache 2.0
>> to BSD.  In general we like the Apache 2.0 license better, but unfortunately
>> it is not compatible with GPLv2.  We decided that allowing GPLv2 projects to
>> use protocol buffers is more important, so we're switching.  I'll be
>> changing all the license headers in SVN tomorrow, probably, and version
>> 2.0.2 should be released under the new license shortly thereafter.
>>
>>
>>
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: xlC suffers the same can't-decide-which-template-to-use as Symbian

[Kenton Varda]

It seems like something went wrong with the upload.  When I look at the
side-by-side diffs it just says "error: old chunk mismatch".  Do you have
any idea what might have gone wrong?

On Wed, Sep 24, 2008 at 6:31 PM, Travis Pouarz <[EMAIL PROTECTED]> wrote:

>
>
>
> On Sep 24, 7:17 pm, "Kenton Varda" <[EMAIL PROTECTED]> wrote:
> > Thanks.  Can you send this to me via codereview.appspot.com?
>
> http://codereview.appspot.com/6051
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Two test suite failures on AIX 5.3 / xlC 7.0

[Kenton Varda]

On Wed, Sep 24, 2008 at 4:37 PM, Travis Pouarz <[EMAIL PROTECTED]> wrote:

> This first one reveals a disagreement about whether a directory name
> should be followed by a slash:
>
> [ RUN  ] CommandLineInterfaceTest.OutputDirectoryIsFileError
> google/protobuf/compiler/command_line_interface_unittest.cc:411:
> Failure
> Value of: printer.failed()
>  Actual: true
> Expected: false
> google/protobuf/compiler/command_line_interface_unittest.cc:328:
> Failure
> Value of: expected_substring
>  Actual: "foo.proto/: Not a directory"
> Expected: a substring of error_text_
> Which is: "output.test.foo.proto: Not a directory
> "
> [  FAILED  ] CommandLineInterfaceTest.OutputDirectoryIsFileError


The test creates a file called "foo.proto" and then attempts to access
"foo.proto/", which should fail because the trailing slash implies that a
directory is expected.  I guess your platform just ignores the trailing
slash.  This isn't a big deal, since it just means protoc's errors might be
a bit less helpful than they should be.  I suppose you could #ifdef out the
test on your platform with a comment saying it doesn't work correctly due to
an OS bug but we don't care that much.


> Here's another failure, though the nature of the failure (and what the
> malfunction might affect) isn't so obvious:
>
> [ RUN  ] DiskSourceTreeTest.MapDirectory
> google/protobuf/compiler/importer_unittest.cc:362: Failure
> Value of: input == NULL
>  Actual: false
> Expected: true
> [  FAILED  ] DiskSourceTreeTest.MapDirectory


Not enough info.  Can you change line 362 from:
  EXPECT_TRUE(input == NULL);
to:
  EXPECT_TRUE(input == NULL) << "Expected file not found: " << filename;
then run again?  This should tell us exactly which file is being found when
it shouldn't be.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: xlC suffers the same can't-decide-which-template-to-use as Symbian

[Kenton Varda]

I still only see the first patch set.

On Thu, Sep 25, 2008 at 4:29 PM, Travis Pouarz <[EMAIL PROTECTED]> wrote:

>
>
> > >http://codereview.appspot.com/6051
>
> On Sep 25, 1:12 pm, "Kenton Varda" <[EMAIL PROTECTED]> wrote:
> > It seems like something went wrong with the upload.  When I look at the
> > side-by-side diffs it just says "error: old chunk mismatch".  Do you have
> > any idea what might have gone wrong?
>
> Strange.  I suspect some line-ending corruption as I worked around a
> firewall issue.  I found a different path and uploaded a new change
> set that looks like it works now.
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: xlC suffers the same can't-decide-which-template-to-use as Symbian

[Kenton Varda]

Apparently codereview.appspot.com allows over-zealous browser caching.  I
had to refresh.  I see it now.  Will review tonight or tomorrow.

On Thu, Sep 25, 2008 at 5:37 PM, Travis <[EMAIL PROTECTED]> wrote:

>
> On Sep 25, 2008, at 6:51 PM, Kenton Varda wrote:
>
>  I still only see the first patch set.
>>
>
>
> You don't see this?
> "Patch Set 2 : Same changes, but might fix problem where side-by-side diffs
> didn't work"
>
> I'm looking at http://codereview.appspot.com/6051 and I see it.  And if I
> look at the side-by-side diffs, they look fine (I did see the error that you
> saw with my "Patch Set 1").
>
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: xlC suffers the same can't-decide-which-template-to-use as Symbian

[Kenton Varda]

Code looks good to me (minus two typos that I'll fix before submitting).
 Which just leaves the legal annoyance...  Have you signed the contributor
license agreement?
http://code.google.com/legal/individual-cla-v1.0.html
http://code.google.com/legal/corporate-cla-v1.0.html

On Thu, Sep 25, 2008 at 7:40 PM, Kenton Varda <[EMAIL PROTECTED]> wrote:

> Apparently codereview.appspot.com allows over-zealous browser caching.  I
> had to refresh.  I see it now.  Will review tonight or tomorrow.
>
>
> On Thu, Sep 25, 2008 at 5:37 PM, Travis <[EMAIL PROTECTED]> wrote:
>
>>
>> On Sep 25, 2008, at 6:51 PM, Kenton Varda wrote:
>>
>>  I still only see the first patch set.
>>>
>>
>>
>> You don't see this?
>> "Patch Set 2 : Same changes, but might fix problem where side-by-side
>> diffs didn't work"
>>
>> I'm looking at http://codereview.appspot.com/6051 and I see it.  And if I
>> look at the side-by-side diffs, they look fine (I did see the error that you
>> saw with my "Patch Set 1").
>>
>>
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: xlC suffers the same can't-decide-which-template-to-use as Symbian

[Kenton Varda]

OK, let me know when it's in.  Thanks!
BTW, if your changes haven't been submitted to upstream gtest, I'll make
sure they get there.

On Fri, Sep 26, 2008 at 6:53 AM, Travis <[EMAIL PROTECTED]> wrote:

>
>  Which just leaves the legal annoyance...  Have you signed the contributor
>> license agreement?
>>
>
> No, not yet.  I'm pursuing getting that handled now.  There are several
> cogs I have to get to turn to make that happen, but I've started turning the
> crank.
>
> -Travis
>
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Things to do with extensible options....

[Kenton Varda]

Interesting ideas.  Before we get too creative, though, let me say a word of
caution:  if an option affects the generated code in all languages (e.g. by
changing the wire format), then it can't really be a "custom" option, since
it will have to be compiled into protoc anyway.  I'm very wary about adding
such features, because while they're often very useful in certain cases,
together they make the system more bloated, possibly harming more than
helping overall (since all users get the additional bloat but only some
actually use the feature).
With that said, here's some more ideas:

* Cache keys:  If you have a service which takes a protocol buffer request
and returns a protocol buffer response with no side-effects, you could
implement a caching layer over it.  Cache keys could be generated by simply
serializing the message and hashing it.  However, your request messages
might contain some fields which should be ignored when computing a cache
key, e.g. options which only affect post-cache operations.  You could use a
custom option to mark which fields should be stripped before computing cache
keys.

* Auto interface generation:  Someone mentioned before that they'd like to
auto-generate a user interface for inputting or printing a protocol message
in a friendly way.  For this, it would be useful to have a custom string
option called "description" which is a human-readable description of the
field.

On Thu, Sep 25, 2008 at 10:24 AM, Chris <[EMAIL PROTECTED]> wrote:

>
> Since there extensible options are almost released, I though I would
> start a thread to discuss what would be interesting to support.
>
> The current options affect the code generation trade offs for a given
> language:
>  optional string java_package = 1;
>  optional string java_outer_classname = 8;
>  optional bool java_multiple_files = 10 [default=false];
>  optional OptimizeMode optimize_for = 9 [default=CODE_SIZE];
>  optional CType ctype = 1;
>
> Or are not to be used in current .proto files:
>  optional bool message_set_wire_format = 1 [default=false];
>  optional string experimental_map_key = 9;
>
> My brainstorming at the moment produces the following language-agnostic
> suggestions.  Note that I am not suggesting the names for any of these
> proposed options ( these bikesheds are unpainted ) :
>
> * FileOptions
>
>  ** A "version" option that holds a string.  This could be inserted
> into the generated code so that programs can inquire and report what
> version of the .proto file was used.  ( Since protocol-buffers is
> designed to support changes in formats over time I am somewhat surprised
> this is not already present. )
>
>  ** Many of the MessageOptions or EnumOptions below could be set at the
> FileOptions level to affect all such messages or enums by default.
>
> * MessageOptions
>
>  ** an option to control whether this message (by default) uses the
> length-prefix message format or the start/end tag group format on the
> wire.  This would make people who prefer streaming writes over
> known-length writes happier.
>
>  ** an option to control whether, when read from the wire format, this
> message should allow unknown fields or always throw an error.  This
> might be handy to avoid much boilerplate checking for stray unknown
> fields.  One could even make the message object slimmer if it does not
> not a pointer/reference to some unknown field data structure.
>
>  ** an option to have the message skip unknown fields on the wire
> without either storing them or throwing an error.  This might be handy
> to define "cut-down" versions of messages that skip over unwanted parts
> to save memory and speed up processing when loading (really querying) a
> large binary file.
>
> * EnumOptions
>
>  ** An option to control the wire format, so it could be set to
> sfixed32 or sint32 by default.  This would be an annoying change to
> support, so I probably should not have mentioned it.  It makes large
> positive and all negative values more compact on the wire.
>
>  ** An option to set the default value.  The current default-default
> value is always the first enum value in the .proto file.
>
> * FieldOptions
>
>  ** Note that the message options and enum option above may be
> considered to set the default behavior for the a message, but that a
> corresponding field option could be defined to override this for a given
> use of the message or enum.  ( This is already possible for setting the
> default enum value for the field !)
>
>  ** One could mark a repeated field to have a more efficient wire
> encoding: Write the field-id+wire-type tag only once followed by the
> uint32 encoded count of items (non-zero!) and then each item without any
> more of the tags.  This would enable much more compact storage of
> numeric arrays.
>
>  ** One could mark a repeated field to never have zero length.  This
> might be handy to avoid much boilerplate checking for empty fields.
>
>  ** One could mark an optional field to always write a value when
> se

Re: repeated fields

[Kenton Varda]

You need to write a length before writing each message, so that when you
read them on the receiving end you can separate the messages before parsing
them.

On Sat, Sep 27, 2008 at 10:17 AM, <[EMAIL PROTECTED]> wrote:

>
> Hello again,
>
> I have another problem concerning repeated fields:
>
> The same example as above, every client sends continuously a Player
> message to the server.
> The server continuously generates a Playerlist message taking the
> newest Player's and sends one to every client.
>
> The problem is, when I parse a Playerlist on the client side, and
> there are more than one Playerlists in the buffer (probably all from
> the same player) I get one Playerlist with multiple players in it.
> I now realized that ParseFrom does actually merge messages and
> repeated fields are appended. This is not desired in my case, I just
> need the last one.
> Is there a way to disable the appending of repeated fields when
> parsing a buffer or is my idea for distributing the players anyway a
> bad solution ?
>
> Regards,
> Rico
>
> On Sep 24, 7:04 pm, [EMAIL PROTECTED] wrote:
> > Hello Kenton,
> >
> > Thanks a lot! That is exactly what I was looking for!
> >
> > Regards,
> > Rico
> >
> > On 24 Sep., 18:56, "Kenton Varda" <[EMAIL PROTECTED]> wrote:
> >
> > > Playerlist also has methods:
> > > const RepeatedPtrField& player() const;
> > > RepeatedPtrField* mutable_player();
> >
> > > RepeatedPtrField is documented here:
> >
> > >http://code.google.com/apis/protocolbuffers/docs/reference/cpp/google.
> ..
> >
> > > It has methods which allow you to add an existing object without
> copying
> > > (look under "Advanced memory management).  Of course, if you use these,
> then
> > > the Player objects will become property of the Playerlist.  You will
> have to
> > > be careful to take ownership back from the list later by repeatedly
> calling
> > > ReleaseLast().
> >
> > > Also note that every protocol message has a CopyFrom() method which can
> be
> > > used to copy an entire message object, so you would not have to call
> > > set_id(), set_x(), etc. manually.
> >
> > > On Wed, Sep 24, 2008 at 8:01 AM, <[EMAIL PROTECTED]> wrote:
> >
> > > > Hello,
> >
> > > > I have a question concerning the repeated fields. I'll start with an
> > > > example:
> >
> > > > message Player {
> > > >  required int32 id = 1;
> > > >  required int32 x =2;
> > > >  required int32 y=3;
> > > > }
> >
> > > > message Playerlist{
> > > >  repeated Player player= 1;
> > > > }
> >
> > > > I have a server and get Player's from different clients and hold them
> > > > in a map (this map is updated all the time). After a certain time I
> > > > want to get all Players, create a Playerlist and send it to all
> > > > clients. The Playerlist however has just a method
> >
> > > > Player* add_player();
> >
> > > > I will have to copy all the Player's I've already created :
> >
> > > > Player* newP = playerlist.add_player();
> > > > newP->set_id( playerFromMap->id() );
> >
> > > > Isn't there a better way to do this?
> >
> > > > Regards
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: protobuf-c compilation problem

[Kenton Varda]

[cc Dave Benson, author of the C bindings]

On Mon, Sep 29, 2008 at 2:05 AM, Doron Tal <[EMAIL PROTECTED]> wrote:

>
> Hi,
>
> I'm new to protobuf, and I want to use the new c binding. However,
> compilation does not pass.
>
> Thanks for your help,
> Doron
>
> [doron-lx] protobuf-c-0.4 > make
> Making all in src
> make[1]: Entering directory `/home1/doron.tal/work/protobuf-c-0.4/src'
> Making all in .
> make[2]: Entering directory `/home1/doron.tal/work/protobuf-c-0.4/src'
> if /bin/sh ../libtool --tag=CC --mode=compile gcc -DPACKAGE_NAME=\"\" -
> DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -
> DPACKAGE_BUGREPORT=\"\" -DPACKAGE=\"protobuf-c\" -DVERSION=\"0.4\" -
> DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -
> DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -
> DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -
> DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -I. -I. -g -O2 -MT protobuf-c.lo
> -MD -MP -MF ".deps/protobuf-c.Tpo" -c -o protobuf-c.lo `test -f
> 'google/protobuf-c/protobuf-c.c' || echo './'`google/protobuf-c/
> protobuf-c.c; \
>then mv -f ".deps/protobuf-c.Tpo" ".deps/protobuf-c.Plo"; else
> rm -f ".deps/protobuf-c.Tpo"; exit 1; fi
> mkdir .libs
>  gcc -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\"
> -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE=\"protobuf-c
> \" -DVERSION=\"0.4\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -
> DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -
> DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -
> DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -I. -I. -g -O2 -MT
> protobuf-c.lo -MD -MP -MF .deps/protobuf-c.Tpo -c google/protobuf-c/
> protobuf-c.c  -fPIC -DPIC -o .libs/protobuf-c.o
>  gcc -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\"
> -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE=\"protobuf-c
> \" -DVERSION=\"0.4\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -
> DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -
> DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -
> DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -I. -I. -g -O2 -MT
> protobuf-c.lo -MD -MP -MF .deps/protobuf-c.Tpo -c google/protobuf-c/
> protobuf-c.c -o protobuf-c.o >/dev/null 2>&1
> /bin/sh ../libtool --tag=CC --mode=link gcc  -g -O2   -o libprotobuf-
> c.la -rpath /usr/local/lib  protobuf-c.lo
> gcc -shared  .libs/protobuf-c.o   -Wl,-soname -Wl,libprotobuf-c.so.0 -
> o .libs/libprotobuf-c.so.0.0.0
> (cd .libs && rm -f libprotobuf-c.so.0 && ln -s libprotobuf-c.so.0.0.0
> libprotobuf-c.so.0)
> (cd .libs && rm -f libprotobuf-c.so && ln -s libprotobuf-c.so.0.0.0
> libprotobuf-c.so)
> ar cru .libs/libprotobuf-c.a  protobuf-c.o
> ranlib .libs/libprotobuf-c.a
> creating libprotobuf-c.la
> (cd .libs && rm -f libprotobuf-c.la && ln -s ../libprotobuf-c.la
> libprotobuf-c.la)
> if g++ -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=
> \"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DPACKAGE=
> \"protobuf-c\" -DVERSION=\"0.4\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1
> -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -
> DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -
> DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -I. -I. -g -O2
> -MT c_service.o -MD -MP -MF ".deps/c_service.Tpo" -c -o c_service.o
> `test -f 'google/protobuf/compiler/c/c_service.cc' || echo './'`google/
> protobuf/compiler/c/c_service.cc; \
>then mv -f ".deps/c_service.Tpo" ".deps/c_service.Po"; else rm
> -f ".deps/c_service.Tpo"; exit 1; fi
> In file included from google/protobuf/compiler/c/c_service.cc:23:
> ./google/protobuf/compiler/c/c_service.h:28:42: error: google/protobuf/
> stubs/common.h: No such file or directory
> ./google/protobuf/compiler/c/c_service.h:29:40: error: google/protobuf/
> descriptor.h: No such file or directory
> In file included from google/protobuf/compiler/c/c_service.cc:24:
> ./google/protobuf/compiler/c/c_helpers.h:29:40: error: google/protobuf/
> io/printer.h: No such file or directory
> ./google/protobuf/compiler/c/c_service.h:45: error: expected ',' or
> '...' before '*' token
> ./google/protobuf/compiler/c/c_service.h:46: error: ISO C++ forbids
> declaration of 'ServiceDescriptor' with no type
> ./google/protobuf/compiler/c/c_service.h:62: error: ISO C++ forbids
> declaration of 'ServiceDescriptor' with no type
> ./google/protobuf/compiler/c/c_service.h:62: error: expected ';'
> before '*' token
> ./google/protobuf/compiler/c/c_service.h:63: error: ISO C++ forbids
> declaration of 'map' with no type
> ./google/protobuf/compiler/c/c_service.h:63: error: expected ';'
> before '<' token
> ./google/protobuf/compiler/c/c_service.h:65: error: ISO C++ forbids
> declaration of 'GOOGLE_DISALLOW_EVIL_CONSTRUCTORS' with no type
> ./google/protobuf/compiler/c/c_helpers.h:49: error: 'string' does not
> name a type
> ./google/protobuf/compiler/c/c_helpers.h:50: error: 'string' does not
> name a type
> ./google/protobuf/compiler/c/c_hel

2.0.2 release candidate

[Kenton Varda]

Hi all,

I've put up a release candidate for v2.0.2 here:
http://groups.google.com/group/protobuf/files

It is also tagged in SVN:
http://protobuf.googlecode.com/svn/tags/release-2.0.2rc1/

Please test it if you have a moment.  Assuming no one finds any problems,
I'll do the official release in a couple days.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

MSVC build switched to static linking

[Kenton Varda]

Hi all,
Just before building the 2.0.2 release candidate, I submitted a change to
make static linking the default for the protobuf libraries under MSVC.  This
seems necessary to avoid some of the problems inherent in DLLs which have
been discussed here, such as the fact that each DLL has a separate heap.
 You can still build DLLs by simply changing the libprotobuf and libprotoc
projects to be DLL projects -- instructions are included in
vsprojects/readme.txt.

Let me know if anyone objects to this.  I don't use MSVC much so it's hard
for me to know what is best for MSVC users.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: License of the output (the flex/bison problem)

[Kenton Varda]

The code generated by protoc requires linking against the protobuf
libraries, so the Protocol Buffers license terms apply to you either way.  I
can talk to our licensing people and get this clarified if you want, but
given this, does it still matter?  BTW, v2.0.2 (to be released later this
week) will be under BSD instead of Apache 2.
On Tue, Sep 30, 2008 at 9:39 AM, Travis P <[EMAIL PROTECTED]> wrote:

>
> Hi Kenton,
>
> Protocol Buffers presents an unusual, but not unique/novel, situation with
> respect to licensing.
>
> Projects that use PB will often not actually include PB code in their
> software (and often may not be re-distributing PB code in source or compiled
> form), but rather, will be using the output of PB, which is code.  What is
> the license of that output-code?  That's the type of question that makes
> lawyers nervous.
>
> This is similar to the bison situation.  Bison is GPL v2.  But obviously
> not all programs that use Bison output are GPL v2.  Bison output includes
> explicit license notice:
> _
> /* Skeleton parser for Yacc-like parsing with Bison,
>   Copyright (C) 1984, 1989, 1990, 2000, 2001, 2002, 2003, 2004, 2005 Free
> Software Foundation, Inc.
>
>   This program is free software; you can redistribute it and/or modify
>   it under the terms of the GNU General Public License as published by
>   the Free Software Foundation; either version 2, or (at your option)
>   any later version.
>
>   .. */
>
> /* As a special exception, when this file is copied by Bison into a
>   Bison output file, you may use that output file without restriction.
>   This special exception was added by the Free Software Foundation
>   in version 1.24 of Bison.  */
> _
>
> Can there be explicit clarification on that point?  Maybe a notice on the
> output to the effect:
>
> // Generated by the protocol buffer compiler.
> // Any copyright ownership is retained by the owner of the input to the
> protocol buffer compiler.
>
> I don't know if that's the best wording (IANAL), but I think that gives you
> the flavor of what I'm thinking.
>
> But then, flex output (where flex is also presumably GPL) doesn't include
> any explicit notice of licensing at all.  I seem to recall that bison was
> 'special' because some code was copied as a verbatim chunk.  I don't know if
> PB includes any such chunks.  Either way, being explicit about it may smooth
> the path in many cases.
>
> -Travis
>
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Support VC6?

[Kenton Varda]

I don't have any plans to support VC6, but if someone does the port and
submits a patch I'll be happy to apply it (assuming it doesn't harm other
platforms).

On Mon, Sep 29, 2008 at 8:37 PM, Angus <[EMAIL PROTECTED]> wrote:

>
> I'm working on several VC6 client server projects and want to migrate
> our RPC protocol to  Protocol Buffers.  I have the dll but can't
> compile the generated code in VC6.  Any plan on supporting VC6, at
> least on the generated code part?
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: License of the output (the flex/bison problem)

[Kenton Varda]

Our lawyer's opinion seems to be that the generated code is owned by the
owner of the input file by default, even without a special note.  However,
I've added a note to COPYING.txt anyway to make it completely clear.
http://protobuf.googlecode.com/svn/trunk/COPYING.txt

On Tue, Sep 30, 2008 at 10:25 AM, Travis P <[EMAIL PROTECTED]> wrote:

>
>  The code generated by protoc requires linking against the protobuf
>> libraries, so the Protocol Buffers license terms apply to you either way.
>>
>
> Ah, right.  Thanks for reminding me of that.
>
>  I can talk to our licensing people and get this clarified if you want, but
>> given this, does it still matter?
>>
>
> Alas, yes, even so, it'd probably be good to have clarity in this area.
>  :-/
> It'll make PB easier to use.  :-)
>
>  BTW, v2.0.2 (to be released later this week) will be under BSD instead of
>> Apache 2.
>>
>
> Yep, I noticed that you'd already changed COPYING.txt.
>
> -Travis
>
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: [PATCH] Clean up autogen.sh script

[Kenton Varda]

The reason I specified automake-1.9 explicitly is because our corp
workstations actually have the regular "automake" command linked to
automake-1.4, which is not good enough.  automake-1.9 is installed on our
workstations, but is not the default.  Can we somehow auto-detect this
situation and do something about it?

On Tue, Sep 30, 2008 at 11:40 AM, Jeff Bailey <[EMAIL PROTECTED]> wrote:

>
> The autogen.sh script has all sorts of hardcoded magic in it,
> including the version of the autotools, and the fact that we're
> requiring "foriegn" mode.  With this patch, this will attempt to use
> whatever the system autotools is, and also makes the foreign mode
> explicit.
>
> diff --git a/Makefile.am b/Makefile.am
> index cd90b6c..52f2caf 100644
> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -2,6 +2,8 @@
>
>  ACLOCAL_AMFLAGS = -I m4
>
> +AUTOMAKE_OPTIONS = foreign
> +
>  SUBDIRS = src
>
>  EXTRA_DIST
> = \
> diff --git a/autogen.sh b/autogen.sh
> index 34a4d2e..959279b 100755
> --- a/autogen.sh
> +++ b/autogen.sh
> @@ -15,13 +15,7 @@ fi
>
>  set -ex
>
> -rm -rf autom4te.cache
> -
> -aclocal-1.9 --force -I m4
> -libtoolize -c -f
> -autoheader -f -W all
> -automake-1.9 --foreign -a -c -f -W all
> -autoconf -f -W all,no-obsolete
> +autoreconf -Wall -f -i
>
>  rm -rf autom4te.cache config.h.in~
>  exit 0
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: [PATCH] Clean up autogen.sh script

[Kenton Varda]

After some offline discussion I've gone ahead and submitted this.

On Tue, Sep 30, 2008 at 11:40 AM, Jeff Bailey <[EMAIL PROTECTED]> wrote:

>
> The autogen.sh script has all sorts of hardcoded magic in it,
> including the version of the autotools, and the fact that we're
> requiring "foriegn" mode.  With this patch, this will attempt to use
> whatever the system autotools is, and also makes the foreign mode
> explicit.
>
> diff --git a/Makefile.am b/Makefile.am
> index cd90b6c..52f2caf 100644
> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -2,6 +2,8 @@
>
>  ACLOCAL_AMFLAGS = -I m4
>
> +AUTOMAKE_OPTIONS = foreign
> +
>  SUBDIRS = src
>
>  EXTRA_DIST
> = \
> diff --git a/autogen.sh b/autogen.sh
> index 34a4d2e..959279b 100755
> --- a/autogen.sh
> +++ b/autogen.sh
> @@ -15,13 +15,7 @@ fi
>
>  set -ex
>
> -rm -rf autom4te.cache
> -
> -aclocal-1.9 --force -I m4
> -libtoolize -c -f
> -autoheader -f -W all
> -automake-1.9 --foreign -a -c -f -W all
> -autoconf -f -W all,no-obsolete
> +autoreconf -Wall -f -i
>
>  rm -rf autom4te.cache config.h.in~
>  exit 0
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Clean up autogen.sh script

[Kenton Varda]

Done.

On Tue, Sep 30, 2008 at 2:51 PM, Jeff Bailey <[EMAIL PROTECTED]> wrote:

>
> Heya Kenton,
>
> I played with the warning option a touch more, and if we do:
>
> autoreconf -f -i -Wall,no-obsolete
>
> it seems to give it without all the noise from old constructs in the
> libtool and automake M4 macros.  Checked with automake-1.9/
> autoconf2.59 and automake1.10/autoconf2.61
>
> On Sep 30, 1:56 pm, "Kenton Varda" <[EMAIL PROTECTED]> wrote:
> > After some offline discussion I've gone ahead and submitted this.
> >
> > On Tue, Sep 30, 2008 at 11:40 AM, Jeff Bailey <[EMAIL PROTECTED]>
> wrote:
> >
> > > The autogen.sh script has all sorts of hardcoded magic in it,
> > > including the version of the autotools, and the fact that we're
> > > requiring "foriegn" mode.  With this patch, this will attempt to use
> > > whatever the system autotools is, and also makes the foreign mode
> > > explicit.
> >
> > > diff --git a/Makefile.am b/Makefile.am
> > > index cd90b6c..52f2caf 100644
> > > --- a/Makefile.am
> > > +++ b/Makefile.am
> > > @@ -2,6 +2,8 @@
> >
> > >  ACLOCAL_AMFLAGS = -I m4
> >
> > > +AUTOMAKE_OPTIONS = foreign
> > > +
> > >  SUBDIRS = src
> >
> > >  EXTRA_DIST
> > > = \
> > > diff --git a/autogen.sh b/autogen.sh
> > > index 34a4d2e..959279b 100755
> > > --- a/autogen.sh
> > > +++ b/autogen.sh
> > > @@ -15,13 +15,7 @@ fi
> >
> > >  set -ex
> >
> > > -rm -rf autom4te.cache
> > > -
> > > -aclocal-1.9 --force -I m4
> > > -libtoolize -c -f
> > > -autoheader -f -W all
> > > -automake-1.9 --foreign -a -c -f -W all
> > > -autoconf -f -W all,no-obsolete
> > > +autoreconf -Wall -f -i
> >
> > >  rm -rf autom4te.cache config.h.in~
> > >  exit 0
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: OCD quoting of M4 constructs

[Kenton Varda]

Submitted.
Note:  Google groups word-wraps input, which can break patches.

On Tue, Sep 30, 2008 at 2:58 PM, Jeff Bailey <[EMAIL PROTECTED]> wrote:

>
> The following patch fixes one obsolete construct in configure.ac and
> neurotically adds quoting.  Because I <3 m4.
>
> Tested with automake-1.9/autoconf2.59 and automake-1.10/autoconf2.61
>
> index 203a2ff..4543e47 100644
> --- a/configure.ac
> +++ b/configure.ac
> @@ -7,10 +7,10 @@ AC_PREREQ(2.59)
>  # * java/pom.xml
>  # * python/setup.py
>  # * src/google/protobuf/stubs/common.h
> -AC_INIT(protobuf, 2.0.2-SNAPSHOT, protobuf@googlegroups.com)
> +AC_INIT([protobuf],[2.0.2-SNAPSHOT],[EMAIL PROTECTED])
>
>  AC_CONFIG_SRCDIR(src/google/protobuf/message.cc)
> -AM_CONFIG_HEADER(config.h)
> +AC_CONFIG_HEADERS([config.h])
>  AM_INIT_AUTOMAKE
>
>  # Checks for programs.
> @@ -31,4 +31,5 @@ AC_CHECK_FUNCS([ftruncate memset mkdir strchr
> strerror strtol]
>  ACX_PTHREAD
>  AC_CXX_STL_HASH
>
> -AC_OUTPUT( Makefile src/Makefile )
> +AC_CONFIG_FILES([Makefile src/Makefile ])
> +AC_OUTPUT
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Things to do with extensible options....

[Kenton Varda]

To clarify, the two ideas I pointed out are just examples of the kind of
things people might want to do with custom options within their own
applications.  They certainly do not belong in the core protobuf library and
they certainly shouldn't be defined or used in applications that don't care
about them.

On Wed, Oct 1, 2008 at 7:07 AM, daveb <[EMAIL PROTECTED]> wrote:

>
> I'd be wary about getting into too custom of applications here.
> People should make their own configuration files for GUI generator and
> caches, in my opinion; they're too complex and special-purpose and so
> on.  For example, GUIs require more layout information than "render as
> slider" or something-- usually the layout groups information
> aesthetically etc.  For caches, you need TTL type metadata usually
> anyways, so the cache implementation can easily provide hooks to
> custom compute the cache-key (which may be case-insenstive or
> approximate -- it's more tricky in general than a list of keys).
>
> What protobuf does is to map between a wire protocol and code in some
> language.  The .proto file should contain predominantly stuff to do
> with that.
>
> * I would really like the repeated field idea (since I have an app
> that serializes vectors as repeated doubles, ugh).  I appreciate that
> every LB will require an update to handle it (since it changes the
> wire protocol), which is rather unpleasant.
>
> * There are a few C options I can see adding...
>  * "inline" -- specify that a submessage field is be included inline
> in the message, instead of a pointer to the message.
>  * "package_c_prefix" -- use this string instead of converting the
> package name to the prefix by changing "." to "__"; for deeply nested
> package names, this convention is too verbose.
>
>
>
> On Sep 26, 11:31 am, "Kenton Varda" <[EMAIL PROTECTED]> wrote:
> > Interesting ideas.  Before we get too creative, though, let me say a word
> of
> > caution:  if an option affects the generated code in all languages (e.g.
> by
> > changing the wire format), then it can't really be a "custom" option,
> since
> > it will have to be compiled into protoc anyway.  I'm very wary about
> adding
> > such features, because while they're often very useful in certain cases,
> > together they make the system more bloated, possibly harming more than
> > helping overall (since all users get the additional bloat but only some
> > actually use the feature).
> > With that said, here's some more ideas:
> >
> > * Cache keys:  If you have a service which takes a protocol buffer
> request
> > and returns a protocol buffer response with no side-effects, you could
> > implement a caching layer over it.  Cache keys could be generated by
> simply
> > serializing the message and hashing it.  However, your request messages
> > might contain some fields which should be ignored when computing a cache
> > key, e.g. options which only affect post-cache operations.  You could use
> a
> > custom option to mark which fields should be stripped before computing
> cache
> > keys.
> >
> > * Auto interface generation:  Someone mentioned before that they'd like
> to
> > auto-generate a user interface for inputting or printing a protocol
> message
> > in a friendly way.  For this, it would be useful to have a custom string
> > option called "description" which is a human-readable description of the
> > field.
> >
> > On Thu, Sep 25, 2008 at 10:24 AM, Chris <[EMAIL PROTECTED]> wrote:
> >
> > > Since there extensible options are almost released, I though I would
> > > start a thread to discuss what would be interesting to support.
> >
> > > The current options affect the code generation trade offs for a given
> > > language:
> > >  optional string java_package = 1;
> > >  optional string java_outer_classname = 8;
> > >  optional bool java_multiple_files = 10 [default=false];
> > >  optional OptimizeMode optimize_for = 9 [default=CODE_SIZE];
> > >  optional CType ctype = 1;
> >
> > > Or are not to be used in current .proto files:
> > >  optional bool message_set_wire_format = 1 [default=false];
> > >  optional string experimental_map_key = 9;
> >
> > > My brainstorming at the moment produces the following language-agnostic
> > > suggestions.  Note that I am not suggesting the names for any of these
> > > proposed options ( these bikesheds are unpainted ) :
> >
> > > * FileOptions
> >
> > >  ** A &

Re: Comparing 2 PBs?

[Kenton Varda]

In which language?  In Java you can just use .equals().  In C++, you could
serialize the two and compare the bytes.  Alternatively, you could write
some code that iterates over the fields via reflection and compares them.

On Wed, Oct 1, 2008 at 9:13 AM, <[EMAIL PROTECTED]> wrote:

>
> What's the "best" (simplest, fastest) way to compare two protocol-
> buffer objects?  (in compiled_for=speed mode)
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Advice on use with early JRE

[Kenton Varda]

I'm not really familiar with Java 1.1 so I'm not sure I can provide much
advice here.
There are lots of things you could potentially strip out, depending on your
needs.  You might want to decide whether your app would rather use
optimize_for = SPEED or optimize_for = CODE_SIZE in general.  Optimizing for
speed means that protoc generates type-specific parsers, whereas optimizing
for size means a lot of stuff is done via reflection.  If you optimized for
speed, you get rid of the introspection methods in the Message interface and
could rip out quite a bit of code in AbstractMessage.java and
GeneratedMessage.java.  Maybe you could even get rid of descriptors.  If
your app does not use extensions, you could rip that out.  You could also
choose to ignore unknown fields rather than storing them in an
UnknownFieldSet, thus eliminating the need for that class.  There's really
quite a bit of stuff that you could potentially remove while still having a
useful system.

On Wed, Oct 1, 2008 at 8:43 AM, <[EMAIL PROTECTED]> wrote:

>
> Hi,
>
> I am currently in the R&D phase of developing interactive television
> applications for a large media company.  The interactive television
> technology being used is formerly known as OCAP, and now referred to
> as Tru2Way.  This technology is currently being adopted by the
> majority of the consumer electronic industry and is planned to be
> included in television sets and STB's (set top boxes) in the future.
> The basics of the technology are that an embedded computer with
> limited memory (etc) and JVM will be built into the TV sets and
> STB's.  Television programs which are broadcast using MPEG-2 format
> contain the ability to transmit additional data including Java
> application code to be broadcast along with the television program.
> The Tru2Way enabled TV's and STB's will then interpret the Java code,
> and allow for applications to be pushed down to users.
>
> The application we are developing will need to receive continuous data
> in order to update the application with the latest information and
> display that information to the TV screen.  We ruled out XML because
> it is too fat and inefficient for the limited embedded computers being
> used.  I then stumbled on Protocol Buffers, and this looks like a
> perfect alternative except for one issue.  The JVM that runs on the
> Tru2Way enabled devices are 1.1.18 JVM's, and therefore do not have
> support for generics, enums, etc.  On the receiving end (the TV and
> STB) the only functionality our application will need is to read the
> data coming in to display to the user.
>
> I was hoping someone more familiar with the source code can shed some
> light on how difficult it may be if I were to bring the code back to
> being compatible with pre-1.2 JVM's.  From first glance at the code,
> it appears the changes won't be too dramatic, but I just wanted to
> make sure I am not missing anything major.  Also, are there any
> additional options anyone can think of for this situation?  Also, what
> source files are needed if all I am interested in is receiving
> messages?  I assume I might not need the Rpc* files, but did not know
> of any additional files I can strip out.
>
> Thanks
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: protocol buffers for javascript

[Kenton Varda]

Done.

On Thu, Oct 2, 2008 at 10:55 AM, yas <[EMAIL PROTECTED]> wrote:

>
> Hi project owners,
>
> I've just started a project for a protocol buffers encoder/decoder
> written in JavaScript.
>
> http://code.google.com/p/protobuf-js/
>
> To tell the truth, I don't think it's very useful. but I'm glad if you
> add it to the OtherLanguages page (http://code.google.com/p/protobuf/
> wiki/OtherLanguages
> ).
>
> Thanks,
> ANDO Yasushi
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: 2.0.2 release candidate

[Kenton Varda]

Sorry, you're just a little bit too late.  I just put up the new release.
On Fri, Oct 3, 2008 at 11:59 AM, Jon Skeet <[EMAIL PROTECTED]> wrote:

>
> On Sep 30, 1:41 am, "Kenton Varda" <[EMAIL PROTECTED]> wrote:
> > I've put up a release candidate for v2.0.2 here:
> http://groups.google.com/group/protobuf/files
> >
> > It is also tagged in SVN:
> http://protobuf.googlecode.com/svn/tags/release-2.0.2rc1/
> >
> > Please test it if you have a moment.  Assuming no one finds any problems,
> > I'll do the official release in a couple days.
>
> I hope I'm not to late for this. I have a small change request.
> Currently the UninterpretedOption.NamePart message has a name_part
> field. Unfortunately in C# that ends up being a type called NamePart
> with a property called NamePart, which is forbidden. Clearly I'll need
> to put something in the C# generator to cope with this (e.g. by
> renaming it to NamePart_) but it would be nice to avoid it for the
> core messages if we possibly could :)
>
> I don't have any particular preference as to how the change is
> accomplished - and if I'm too late, it's relatively unimportant - but
> I thought I'd ask now rather than regret it later!
>
> Jon
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

2.0.2 release is up

[Kenton Varda]

http://code.google.com/p/protobuf/downloads/list

I forgot to strip the win32 binary in the first upload, making for a
ginormous download.  Hence the -a.  :/

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Library versioning question

[Kenton Varda]

Argh, that was a mistake.  They should use the same version.  And 2.0.1
should really have been 1:0:0.
I'm not very good at this maintenance thing...

Unfortunately, due to the nature of C++, it's unlikely that any two versions
of the library will be binary-compatible.

On Mon, Oct 6, 2008 at 11:08 AM, Iustin Pop <[EMAIL PROTECTED]> wrote:

>
> Hi,
>
> I see that version 2.0.2 has started versioning the libprotobuf library
> (with version = 2:0:0), but the libprotc library is still at version
> 0:0:0.
>
> Is there a reason for this discrepancy? And is the version stable now
> for libprotobuf?
>
> (I ask as I'm working on the debian packages and this change requires
> package name changes...)
>
> thanks,
> iustin
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Major update submitted; 2.0.2 release soon

[Kenton Varda]

On Mon, Oct 6, 2008 at 10:36 AM, Chris <[EMAIL PROTECTED]> wrote:

> Kenton Varda wrote:
>
>> I'll be updating the documentation to explain this better soon.  It's a
>> bit complicated.  Check out unittest_custom_options.proto for an example.
>>  Let me try to explain briefly...
>>
>>
> The unittest_custom_options.proto is useful.  It certainly helps show the
> constraints at work.
>
> The design rationale I infer is that normal field names, unlike extensions,
> will never need to be qualified so there is no need for parenthesis around
> these names.   This is promoted to a hard rule that normal field names are
> never allowed to be in parenthesis.  Thus the parenthesis are present if and
> only if the thing named is an extension field.


Mostly right.  The way I think of it, what the parentheses do is change the
name lookup semantics of the field.  An unparenthesized identifier names a
field defined within the scope of the message type, whereas a parenthesized
expression names something relative to the *current* scope.

These theoretical details aren't terribly important, though.


> Since every single existing normal field name of an option message is NOT a
> message/group type, the first name part being unparenthesized means that
> there will be no '.' and no additional name parts.


Currently, that's true.  However, if someone added a message- or group-type
field to one of the options protos, the current code would be able to handle
it automatically.  The syntax would be like:

  optional int32 foo = 1 [some_message_option.some_field = 123];


> I have updated my Lexer and Parser (in my development version) to recognize
> the new proto file syntax and fill in the UninterpretedOption fields.  This
> has been tested on unittest_custom_options.proto.  No further processing of
> the new options is done yet: no name resolution, no conversion to extension
> field value.  This will eventually get done.


OK.  Note that users should never be playing with UninterpretedOptions,
though.  So, until the actual conversion to extensions happens, I would call
the feature incomplete.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: 2.0.2 release is up

[Kenton Varda]

Well, the code is correct C++, so it's hard to tell what might convince the
compiler to accept it.
Does it work if you change this:

  typename DescriptorT::OptionsType* options =
  tables_->AllocateMessage();

To this:

  typedef typename DescriptorT::OptionsType OptionsType;
  OptionsType* options = tables_->AllocateMessage();

?

On Sun, Oct 5, 2008 at 7:52 AM, edan <[EMAIL PROTECTED]> wrote:

>
> 2.0.2 build bombs for me:
>
> google/protobuf/descriptor.cc: In member function `void
>   google::protobuf::DescriptorBuilder::AllocateOptionsImpl(const
> std::string&,
>   const std::string&, DescriptorT::OptionsType&, DescriptorT*)':
> google/protobuf/descriptor.cc:2225: syntax error before `>' token
> make[2]: *** [descriptor.lo] Error 1
>
> Granted I am unfortunately using a pretty ancient gcc:
> gcc (GCC) 3.2.3 20030502 (Red Hat Linux 3.2.3-52)
>
> But protobuf-2.0.1 worked great.
> Any way you can preserve this compatibility you had until now?  We are
> moving to a newer EL soon, but not soon enough such that this breakage
> inconveniences me quite a bit right now.
>
> BTW, I am thrilled with protocol buffers so far!  Great work.
>
> Thanks
> --edan
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: python setup.py test failing

[Kenton Varda]

Petar, can you look into this?

On Sun, Oct 5, 2008 at 2:47 AM, <[EMAIL PROTECTED]> wrote:

>
> Hi,
>
> I'm trying to install and test out protobuf-2.0.2.  I read the
> instructions in python/README.txt and ran
> $ python setup.py test
>
> And I get
>
> ==
> FAIL: testReadScalars
> (google.protobuf.internal.decoder_test.DecoderTest)
> --
> Traceback (most recent call last):
>  File "/home/frank/projects/protobuf-2.0.2/python/google/protobuf/
> internal/decoder_test.py", line 156, in testReadScalars
>self.ReadScalarTestHelper(*args)
>  File "/home/frank/projects/protobuf-2.0.2/python/google/protobuf/
> internal/decoder_test.py", line 113, in ReadScalarTestHelper
>'Type of reslt %s not the expected one %s' % (type(result),
> type(expected_result
> AssertionError
>
>
> After some digging into the code, I find that it's failing in
> decoder_test.py when testing the decoder.Decoder.ReadSFixed32()
> function.  The expected value is long(-1), but the actual returned
> value was int(-1).  The test passes if I change line 128 in /google/
> protobuf/internal/decoder_test.py from
>['sfixed32', decoder.Decoder.ReadSFixed32, long(-1),
> to
>['sfixed32', decoder.Decoder.ReadSFixed32, -1,
>
> Logically this changes seems reasonable, but I can't tell for sure.
> Is this a bug in the download, or is this a platform dependent
> problem?
>
> I'm running Python2.5 on a x86_64 machine.  This also fails with
> python2.4.
>
> Thanks,
> Frank
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Unittest failure in python decoder tests

[Kenton Varda]

[+petar]
(Same problem as the other thread.)

On Sun, Oct 5, 2008 at 2:41 AM, Iustin Pop <[EMAIL PROTECTED]> wrote:

>
> Hi there,
>
> Revision 50 changed the following line in
> python/google/protobuf/internal/decoder_test.py:
>
> 122: ['sfixed32', decoder.Decoder.ReadSFixed32, -1,
>  'ReadLittleEndian32', 0x],
>
> (in testReadScalars). The change is going from the above "-1" to
> "long(-1)". While this passes ok on i386, it fails on amd64:
>
> i386: result is , expected is 
> amd64: result is , expected is 
>
> I don't understand exactly what's going on, but I think the problem is
> that an i386 system cannot represent that constant as int, but amd64 can
> (not sure why...):
> $ python32 -c 'print type(0x)'
> 
> $ python64 -c 'print type(0x)'
> 
>
> Of course, just reverting the 'long(-1)' change makes the test fail on
> i386.
>
> So, is this a genuine failure in the code, or is it just a bad unittest
> that I can workaround by changing 0x to long(0x)?
>
> I'm thinking of applying this patch in order to make the tests pass on
> both platforms:
>
> $ svn diff
> Index: python/google/protobuf/internal/decoder_test.py
> ===
> --- python/google/protobuf/internal/decoder_test.py (revision 64)
> +++ python/google/protobuf/internal/decoder_test.py (working copy)
> @@ -120,7 +120,7 @@
> ['fixed64', decoder.Decoder.ReadFixed64, 0x,
> 'ReadLittleEndian64', 0x],
> ['sfixed32', decoder.Decoder.ReadSFixed32, long(-1),
> - 'ReadLittleEndian32', 0x],
> + 'ReadLittleEndian32', long(0x)],
> ['sfixed64', decoder.Decoder.ReadSFixed64, long(-1),
>  'ReadLittleEndian64', 0x],
> ['float', decoder.Decoder.ReadFloat, 0.0,
>
> regards,
> iustin
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Patch for python example program

[Kenton Varda]

Done and submitted.  Thanks.

On Sun, Oct 5, 2008 at 3:42 AM, Iustin Pop <[EMAIL PROTECTED]> wrote:

>
> Currently, the python example program list_people.py has a slightly
> different output than the others, so you can't use it to test that all
> languages give the same output.
>
> The following patch removes extra spaces before the phone number:
>
> Index: list_people.py
> ===
> --- list_people.py  (revision 64)
> +++ list_people.py  (working copy)
> @@ -15,11 +15,11 @@
>
> for phone_number in person.phone:
>   if phone_number.type == addressbook_pb2.Person.MOBILE:
> -print "  Mobile phone #: ",
> +print "  Mobile phone #:",
>   elif phone_number.type == addressbook_pb2.Person.HOME:
> -print "  Home phone #: ",
> +print "  Home phone #:",
>   elif phone_number.type == addressbook_pb2.Person.WORK:
> -print "  Work phone #: ",
> +print "  Work phone #:",
>   print phone_number.number
>
>  # Main procedure:  Reads the entire address book from a file and prints
> all
>
>
> regards,
> iustin
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Can a text formated "Flat file" be parsed to PB message given the format of message in .proto file?

[Kenton Varda]

You can use TextFormat for this.  Say your file is going to have a set of
FooMessages in it.  Then, define a message like:

  message FooSet {
repeated FooMessage foo = 1;
  }

Now, your file can look like this:

  foo {
... contents of the first FooMessage ...
  }
  foo {
... the second FooMessage ...
  }
  ...

Now parse the entire file as a single FooSet.

BTW, protoc can do text<->binary conversion from the command line:

  protoc foo.proto --encode=FooSet < text_data > binary_data

Hope that helps.

On Sun, Oct 5, 2008 at 11:43 PM, Akhilesh <[EMAIL PROTECTED]> wrote:

>
> More over, I could use TextFormat class for reading a single message
> in TEXT format. But there is a restriction here, If I have multiple
> messages in the same file, then it's not possible to create multiple
> messages while parser is reading from input stream.
>
> Please correct me if I am wrong.
>
> Finally Aim is: If I have *multiple* messages written in *text* in any
> file, I want to open a stream to read this file. And now I should be
> able to simply wrap this stream in suitable PB streams and simple call
> any function that will go on reading the stream and provide me an
> array of PB Messages in Binary format as a result.
>
> So is there anything like that? Or I'll have to write my code above PB
> api?
>
> Thanks in advance.
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Default values - escaping

[Kenton Varda]

We use C-style escaping in both the .proto language and TextFormat.  The
only difference from C string literals is that the outer quotes may be
either single or double quotes (as long as they match).

On Tue, Oct 7, 2008 at 5:05 AM, Marc Gravell <[EMAIL PROTECTED]> wrote:

>
> It doesn't seem explicit in the language guide - what escaping should
> be applied to default strings that contain quotes?
>
> Cheers,
>
> Marc
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Solaris 10 Compile problem

[Kenton Varda]

Hi Chad,

First, please be aware that protobuf@googlegroups.com is a public mailing
list with non-Googlers on it.  If you need to communicate anything
confidential or Google-specific, please e-mail the appropriate internal
mailing list.
Anyway, I'm afraid you're going to have to help out with this, as I do not
have a Solaris machine, and "bus error" is way too generic to tell me
anything useful.  Can you run the tests in a debugger and give me a stack
trace?

On Tue, Oct 7, 2008 at 2:51 PM, Chad Harvey <[EMAIL PROTECTED]> wrote:

>
> Howdy -
>
> I'm an SA (from doubleclick) working with some former doubleclick and other
> google folx trying to get some things set up to xfer between legacy dclk and
> google.
>
> They requested that I install protobuf on a couple of hosts, but I'm having
> some problem with the make check, and was hoping you could help.
>
> If there's somewhere else I should ask about this (like a guts queue),
> please let me know.
>
> This is on a Solaris 10, sparc arch, T2000, with gcc/g++ 3.4.6.
>
> The only errors I got on the compile were from ld at the end:
>
> g++ -D_REENTRANT -pthreads -Wall -Wwrite-strings -Woverloaded-virtual
> -Wno-sign-compare -g -O2 -D_REENTRANT -pthreads -o .libs/protobuf-test
> common_unittest.o strutil_unittest.o descriptor_database_unittest.o
> descriptor_unittest.o dynamic_message_unittest.o extension_set_unittest.o
> generated_message_reflection_unittest.o message_unittest.o
> reflection_ops_unittest.o repeated_field_unittest.o text_format_unittest.o
> unknown_field_set_unittest.o wire_format_unittest.o coded_stream_unittest.o
> printer_unittest.o tokenizer_unittest.o zero_copy_stream_unittest.o
> command_line_interface_unittest.o importer_unittest.o parser_unittest.o
> cpp_bootstrap_unittest.o cpp_unittest.o test_util.o googletest.o file.o
> gtest.o gtest-death-test.o gtest-filepath.o gtest-port.o gtest_main.o
> unittest.pb.o unittest_import.pb.o unittest_mset.pb.o
> unittest_optimize_for.pb.o unittest_embed_optimize_for.pb.o
> unittest_custom_options.pb.o cpp_test_bad_identifiers.pb.o
> ./.libs/libprotobuf.so
> -L/sol10/SOURCES/S10/gcc-3.4.6/objdir/sparc-sun-solaris2.10/libstdc++-v3/src
> -L/sol10/SOURCES/S10/gcc-3.4.6/objdir/sparc-sun-solaris2.10/libstdc++-v3/src/.libs
> -L/usr/local/lib -L/usr/local/ssl/lib -L/usr/openwin/lib
> -L/sol10/SOURCES/S10/gcc-3.4.6/objdir/gcc ./.libs/libprotoc.so
> /root/protobuf-2.0.2/src/.libs/libprotobuf.so -lpthread
> /usr/local/lib/libstdc++.so -Wl,-R -Wl,/usr/local/lib
> ld: warning: file /root/protobuf-2.0.2/src/.libs/libprotobuf.so: linked to
> ./.libs/libprotobuf.so: attempted multiple inclusion of file
> ld: warning: file /usr/local/lib/libstdc++.so: attempted multiple inclusion
> of file
> creating protobuf-test
> make[3]: Leaving directory `/root/protobuf-2.0.2/src'
> make[2]: Leaving directory `/root/protobuf-2.0.2/src'
> make[2]: Entering directory `/root/protobuf-2.0.2'
> make[2]: Leaving directory `/root/protobuf-2.0.2'
> make[1]: Leaving directory `/root/protobuf-2.0.2'
>
>
> The 'make check' fails with:
>
> Making check in src
> make  check-am
> make  check-TESTS
> Running main() from gtest_main.cc
> [==] Running 557 tests from 85 test cases.
> [--] Global test environment set-up.
> [--] 1 test from GeneratedDescriptorTest
> [ RUN  ] GeneratedDescriptorTest.IdenticalDescriptors
> [   OK ] GeneratedDescriptorTest.IdenticalDescriptors
> [--] 22 tests from GeneratedMessageTest
> [ RUN  ] GeneratedMessageTest.Defaults
> [   OK ] GeneratedMessageTest.Defaults
> [ RUN  ] GeneratedMessageTest.Accessors
> [   OK ] GeneratedMessageTest.Accessors
> [ RUN  ] GeneratedMessageTest.MutableStringDefault
> [   OK ] GeneratedMessageTest.MutableStringDefault
> [ RUN  ] GeneratedMessageTest.Clear
> [   OK ] GeneratedMessageTest.Clear
> [ RUN  ] GeneratedMessageTest.EmbeddedNullsInBytesCharStar
> [   OK ] GeneratedMessageTest.EmbeddedNullsInBytesCharStar
> [ RUN  ] GeneratedMessageTest.ClearOneField
> [   OK ] GeneratedMessageTest.ClearOneField
> [ RUN  ] GeneratedMessageTest.CopyFrom
> [   OK ] GeneratedMessageTest.CopyFrom
> [ RUN  ] GeneratedMessageTest.CopyConstructor
> [   OK ] GeneratedMessageTest.CopyConstructor
> [ RUN  ] GeneratedMessageTest.CopyAssignmentOperator
> [   OK ] GeneratedMessageTest.CopyAssignmentOperator
> [ RUN  ] GeneratedMessageTest.UpcastCopyFrom
> [   OK ] GeneratedMessageTest.UpcastCopyFrom
> [ RUN  ] GeneratedMessageTest.DynamicMessageCopyFrom
> bash: line 4:  9616 Bus Error   (core dumped) ${dir}$tst
> FAIL: protobuf-test
> ==
> 1 of 1 tests failed
> Please report to protobuf@googlegroups.com
> ==
> *** Error code 1
> The following command caused the error:
> failed=0; all=0; xfail=0; xpass=0; skip=0; \
> srcdir=.; export srcdir; \
> list='protobuf-test'; \
> if test -n "$list"; then \

Re: 2.0.2 release is up

[Kenton Varda]

OK, based on sephacryl's message I created this patch, but I suspect edan's
problem is actually different.  Can you both try it out and tell me if it
works?  edan, if this doesn't solve the problem for you, can you try making
some changes yourself and figure out what works?
Index: src/google/protobuf/descriptor.cc
===
--- src/google/protobuf/descriptor.cc   (revision 65)
+++ src/google/protobuf/descriptor.cc   (working copy)
@@ -1693,6 +1693,9 @@
   template void AllocateOptions(
   const typename DescriptorT::OptionsType& orig_options,
   DescriptorT* descriptor);
+  // Specialization for FileOptions.
+  void AllocateOptions(const FileOptions& orig_options,
+   FileDescriptor* descriptor);

   // Implementation for AllocateOptions(). Don't call this directly.
   template void AllocateOptionsImpl(
@@ -2208,9 +2211,8 @@
 }

 // We specialize for FileDescriptor.
-template<> void DescriptorBuilder::AllocateOptions(
-const FileDescriptor::OptionsType& orig_options,
-FileDescriptor* descriptor) {
+void DescriptorBuilder::AllocateOptions(const FileOptions& orig_options,
+FileDescriptor* descriptor) {
   // We add the dummy token so that LookupSymbol does the right thing.
   AllocateOptionsImpl(descriptor->package() + ".dummy", descriptor->name(),
   orig_options, descriptor);


On Tue, Oct 7, 2008 at 10:05 PM, <[EMAIL PROTECTED]> wrote:

>
> What works is adding to the definition of class DescriptorBuilder to
> line1696,
>
>  void AllocateOptions(
>  const typename FileDescriptor::OptionsType& orig_options,
>  FileDescriptor* descriptor);
>
> And changing the implementation, removing the template specialization,
> at line 2213 to:
>
> void DescriptorBuilder::AllocateOptions(
>const FileDescriptor::OptionsType& orig_options,
>FileDescriptor* descriptor) {
>  // We add the dummy token so that LookupSymbol does the right thing.
>  AllocateOptionsImpl(descriptor->package() + ".dummy", descriptor-
> >name(),
>  orig_options, descriptor);
> }
>
> This is not incorrect either since it implements function overloading,
> without the messy template specialization.
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Import broken?

[Kenton Varda]

protobuf_BuildDesc_RequestHeader_2eproto() should be declared in
RequestHeader.pb.h.  Is it not?
What exact flags are you passing to protoc?

On Thu, Oct 9, 2008 at 8:29 AM, <[EMAIL PROTECTED]> wrote:

>
> I'm in the process of evaluating PB for a distributed cross language
> project I'm on. I've created a little 'hello world' app to test it's
> capabilities. I have created 2 simple .proto files,
> RequestHeader.proto and Rerender.proto. They compile and run as
> expected. However, if I import RequestHeader into Rerender with the
> following line:
>
> import "RequestHeader.proto";
>
> I get the following compile error:
>
> Error C2039: 'protobuf_BuildDesc_RequestHeader_2eproto' : is not a
> member of '`global namespace''
>
> The method protobuf_BuildDesc_RequestHeader_2eproto() doesn't exist
> anywhere in the code except where it errors out. Below is the
> offending method in Rerender.pb.cc that contains the error:
>
> void
>
> protobuf_BuildDesc_e_3a_2fVital_203_2e0_2fCPPProtobufTest_2fCPPProtobufTest_2fRerender_2eproto()
> {
>  static bool already_here = false;
>  if (already_here) return;
>  already_here = true;
>  GOOGLE_PROTOBUF_VERIFY_VERSION;
>  ::google::protobuf::DescriptorPool* pool =
>::google::protobuf::DescriptorPool::internal_generated_pool();
>
>  ::protobuf_BuildDesc_RequestHeader_2eproto(); // <-- ERROR
>  pool->InternalBuildGeneratedFile(
>"\n;e:/Vital 3.0/CPPProtobufTest/CPPProtob"
>"ufTest/Rerender.proto\032\023RequestHeader.pro"
>"to\"\030\n\010Rerender\022\014\n\004Data\030\001 \002(\t", 108,
>
>
> &protobuf_BuildDesc_e_3a_2fVital_203_2e0_2fCPPProtobufTest_2fCPPProtobufTest_2fRerender_2eproto_AssignGlobalDescriptors);
> }
>
> I would assume that the ability to import one proto file into another
> is something that is exercised quite often, so I'm at a loss as to why
> the resulting code from protoc won't compile. I'm using protobuf 2.0.2
> built from source using VS 2008.
>
> Any assistance on how to get this working would be greatly
> appreciated. Thanks.
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Patch to remove trailing comma from C++ generated enum output

[Kenton Varda]

This is fixed in 2.0.2.  I didn't use the exact patch that Jay provided but
I made an equivalent change.

On Thu, Oct 9, 2008 at 11:34 AM, mordred <[EMAIL PROTECTED]> wrote:

>
> Any word on this? It doesn't seem to be in 2.0.2. This is causing us
> to commit patched generated files in Drizzle rather than being able to
> have just the .proto files.
>
> Thanks!
>
> On Aug 26, 6:23 pm, "Kenton Varda" <[EMAIL PROTECTED]> wrote:
> > Thanks.  I'll apply this after 2.0.1 release, which has been pending for
> > some time blocked on some stupid issue that's out of my hands but should
> > supposedly be resolved any time now.
> >
> > On Tue, Aug 26, 2008 at 8:42 AM, jaypipes <[EMAIL PROTECTED]> wrote:
> >
> > > Hi!
> >
> > > The C++ code generator has a small bug that produces code that
> > > produces warnings on some compilers when the -pedantic flag is
> > > enabled.  In particular, when an enum definition ends in a trailing
> > > comma, a warning is thrown.  Here is a patch to fix the small bug.
> > > Thanks,
> >
> > > Jay Pipes
> > > MySQL/Sun
> >
> > > diff -u for /src/google/protobuf/compiler/cpp/cpp_enum.cc:
> >
> > > --- cpp_enum.cc 2008-08-26 10:57:59.0 -0400
> > > +++ cpp_enum.cc.bak 2008-08-26 10:58:16.0 -0400
> > > @@ -57,7 +57,14 @@
> > > vars["prefix"] = (descriptor_->containing_type() == NULL) ?
> > >   "" : classname_ + "_";
> >
> > > -printer->Print(vars, "$prefix$$name$ = $number$,\n");
> > > +const char* enum_output;
> > > +// A trailing comma is a pedantic warning on some C++ compilers
> > > +// and so we ensure that no trailing slash is present
> > > +if (i != (descriptor_->value_count() - 1))
> > > +  enum_output = "$prefix$$name$ = $number$, \n";
> > > +else
> > > +  enum_output = "$prefix$$name$ = $number$\n";
> > > +printer->Print(vars, enum_output);
> >
> > > if (descriptor_->value(i)->number() < min_value->number()) {
> > >   min_value = descriptor_->value(i);
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Import broken?

[Kenton Varda]

On Thu, Oct 9, 2008 at 12:52 PM, <[EMAIL PROTECTED]> wrote:

> protoc.exe --cpp_out= "[Fully qualified path name]"


Ah, ok, you've identified a bug.  protoc is supposed to reject any input
file which is not located within the import path.  The import path is
specified using --proto_path and defaults to the current directory.  The
problem is that protoc detects whether an input file's path is absolute
based on whether it starts with a slash, but on Windows absolute paths don't
start with a slash.  So, protoc failed to report the incorrect input path.

To fix this, you can either do:

  protoc --proto_path="[full path of current directory]" --cpp_out=. "[full
path of input file]"

Or do:

  protoc --cpp_out=. "[file name without path]"

I'll make sure the bug gets fixed.  Thanks.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Options in 2.0.2 broken?

[Kenton Varda]

Looks like a bug to me.  Thanks for catching it.

On Fri, Oct 10, 2008 at 10:19 AM, Jon Skeet <[EMAIL PROTECTED]> wrote:

>
> On Oct 10, 5:33 pm, Jon Skeet <[EMAIL PROTECTED]> wrote:
> > Anyone fancy pointing out my stupid mistake?
>
> Got it!
>
> Not only does options.proto need to import google/protobuf/
> descriptor.proto, but so does test.proto. In other words, the
> dependencies aren't treated as being transitive.
>
> Is this by design? If so, the comment for the error reporting is
> incorrect, and the error should probably explain the likely fix :)
>
> Either way, I can at least make progress with the C# generator now :)
>
> Jon
>
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Extension question

[Kenton Varda]

Extensions work analogously to the generated accessors you'd see for a
normal field of the same type.  So, for a message-typed field, there is no
"set" accessor.  Instead, there is a "mutable" accessor.  So, you want to
do:
  Request request;
  Renderer* renderer = request.MutableExtension(Renderer::Command);

This way, the ownership relation between the objects is clear.

The C++ generated code guide describes exactly what accessors are generated
for any particular field type:

http://code.google.com/apis/protocolbuffers/docs/reference/cpp-generated.html

Unfortunately, the compiler produces pretty confusing errors when extensions
are misused.  Such is the nature of C++ templates.  Sorry about that.

On an unrelated note, I notice your naming convention differs from the one
suggested in the protocol buffer style guide.  This is, of course, up to
you, but you might find the system plays nicer if you follow the guide:

  http://code.google.com/apis/protocolbuffers/docs/style.html

On Fri, Oct 10, 2008 at 8:26 AM, <[EMAIL PROTECTED]> wrote:

>
> I have a question about extensions. I have the following 2 .proto
> files:
>
> //Request.proto
> message Request
> {
>required string Guid = 1;
>required string Name = 2;
>extensions 100 to 199;
> }
>
> // Rerender.proto
> import "Request.proto";
> message Rerender
> {
>extend Request
>{
>optional Rerender Command = 100;
>}
>
>required string Data = 1;
> }
>
> When I try to do the following:
> Request request;
> Rerender rerender;
> request.SetExtension(Rerender::Command, rerender);
>
> I get the following:
> error C2039: 'Set' : is not a member of
> 'google::protobuf::internal::MessageTypeTraits'
>
> The offending method from Request.pb.h:
>
> template 
>  inline void SetExtension(
>  const ::google::protobuf::internal::ExtensionIdentifier<
>Request, _proto_TypeTraits>& id,
>  typename _proto_TypeTraits::ConstType value) {
>_proto_TypeTraits::Set(id.number(), value, &_extensions_);
>  }
>
> Note that if I use a Scalar type as the datatype of my extension, it
> compiles fine. The problem appears to be limited to using message
> types as an extension.
>
> Any assistance on how to get this to work would be much appreciated!!
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Import broken?

[Kenton Varda]

Strange, I don't know why you'd be getting permission denied errors, but I'm
glad you figured out something that works.

On Fri, Oct 10, 2008 at 7:38 AM, <[EMAIL PROTECTED]> wrote:

>
> Thanks!!
>
> I also noticed that I get a permission denied error if I specify
> either a valid path or "." for the --proto_path argument on the
> command line. What I did to get it to work was to cd to the directory
> the file is in, and then simply pass the file name to --cpp_out
> without passing a --proto_path argument. The resulting code is now
> compiles :)
>
> Below is how got .proto files compiling with protoc as a custom build
> step in VS:
> protobuild.cmd [inputs]
>
> protobuild.cmd is the following:
> @echo off
> cd "%~p1"
> protoc.exe --cpp_out= "%~n1%~x1"
>
> The reason for the .cmd file is that the [inputs] macro from VS
> provides the .proto file to the command line using a relative path.
> The cmd file takes the relative path and expands it out to a fully
> qualified path, filename and extension and passes it to protoc.
>
>
> On Oct 9, 5:00 pm, "Kenton Varda" <[EMAIL PROTECTED]> wrote:
> > On Thu, Oct 9, 2008 at 12:52 PM, <[EMAIL PROTECTED]> wrote:
> > > protoc.exe --cpp_out= "[Fully qualified path name]"
> >
> > Ah, ok, you've identified a bug.  protoc is supposed to reject any input
> > file which is not located within the import path.  The import path is
> > specified using --proto_path and defaults to the current directory.  The
> > problem is that protoc detects whether an input file's path is absolute
> > based on whether it starts with a slash, but on Windows absolute paths
> don't
> > start with a slash.  So, protoc failed to report the incorrect input
> path.
> >
> > To fix this, you can either do:
> >
> >   protoc --proto_path="[full path of current directory]" --cpp_out=.
> "[full
> > path of input file]"
> >
> > Or do:
> >
> >   protoc --cpp_out=. "[file name without path]"
> >
> > I'll make sure the bug gets fixed.  Thanks.
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: How can I compile the protobuf with stlport?

[Kenton Varda]

I'm not familiar with stlport.  One thing you could try doing is editing
config.h manually and removing all the HASH_* #defines.  Then the code will
fall back to using map instead of hash_map, which should work since map has
a standard interface.

On Fri, Oct 10, 2008 at 2:58 AM, <[EMAIL PROTECTED]> wrote:

>
> I'm using stlport, and i want to use protobuf, but I see some
> compiling errors about the hash_map.
>
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Merge two messages into one file

[Kenton Varda]

Sorry, I'm not sure I understand the question.  What, exactly, is the main
problem you're having?

On Fri, Oct 10, 2008 at 3:11 AM, [EMAIL PROTECTED]
<[EMAIL PROTECTED]>wrote:

>
> I have some kind of requirement that I did not find answer from the
> document.  May someone help me.
>
> I have different message objects which are inherited from one parent.
> I need to serialize the objects into one file.
> Some kind of solutions like:
> e.g.
>  message Style {
>optional int32 color = 3;
>optional int32 size = 4;
> }
>
> message Text {
>optional Style style = 3;
>optional string context = 4;
>  }
>
> message Image {
>optional Style style = 3;
>
>optional int32 width = 4;
>optional int32 height = 5;
>optional bytes data = 6;
>  }
>
> My Group object includes Text objects and Image objects.  But the
> position is not fixed.
> e.g.Text.Image.Image...Text
> and could be: Image.Text...Image...Text
>
> Now I need to serialize the objects into one file, and seek for the
> solution using protocol buffer.
> Implement this in Java is quite easy, because I can use
> DataInputStream and serialize the objects.
> Thanks for any hints.
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: invalid tag (zero)

[Kenton Varda]

Are you sure that the data you are sending to the parser is exactly the same
data that was generated by the serializer?  Remember that protocol buffers
are not self-delimiting, so you need to make sure that you limit the input
to the exact number of bytes that were produced when serializing.
If the data is exactly the same, then this is a bug.  If you can create a
small program or pair of programs that demonstrate the problem, I would be
happy to debug it.

On Mon, Oct 13, 2008 at 10:09 AM, Dominik Steenken <[EMAIL PROTECTED]> wrote:

>
> Hi everyone,
>
>  we are currrently implementing a server/client system, the server
> being implemented in c++, the client in java. During our last rounds
> of tests, we encountered a problem that had to do with the sending of
> (not so) long messages. on the (receiving) java side, we get the
> following exception:
>  Exception in augnet.client.aim.connection.Receiver, Parse error:
> com.google.protobuf.InvalidProtocolBufferException: Protocol message
> contained an invalid tag (zero).
> com.google.protobuf.InvalidProtocolBufferException: Protocol message
> contained an invalid tag (zero).
>at
>
> com.google.protobuf.InvalidProtocolBufferException.invalidTag(InvalidProtocolBufferException.java:
> 52)
>at
> com.google.protobuf.CodedInputStream.readTag(CodedInputStream.java:
> 67)
>at com.google.protobuf.FieldSet.mergeFrom(FieldSet.java:397)
>at com.google.protobuf.AbstractMessage
> $Builder.mergeFrom(AbstractMessage.java:248)
>at com.google.protobuf.GeneratedMessage
> $Builder.mergeFrom(GeneratedMessage.java:1)
>at
> com.google.protobuf.CodedInputStream.readMessage(CodedInputStream.java:
> 227)
>at com.google.protobuf.FieldSet.mergeFieldFrom(FieldSet.java:482)
>at com.google.protobuf.FieldSet.mergeFrom(FieldSet.java:402)
>at com.google.protobuf.AbstractMessage
> $Builder.mergeFrom(AbstractMessage.java:248)
>at com.google.protobuf.GeneratedMessage
> $Builder.mergeFrom(GeneratedMessage.java:1)
>at
> com.google.protobuf.CodedInputStream.readMessage(CodedInputStream.java:
> 227)
>at com.google.protobuf.FieldSet.mergeFieldFrom(FieldSet.java:482)
>at com.google.protobuf.FieldSet.mergeFrom(FieldSet.java:402)
>at com.google.protobuf.AbstractMessage
> $Builder.mergeFrom(AbstractMessage.java:248)
>at com.google.protobuf.AbstractMessage
> $Builder.mergeFrom(AbstractMessage.java:240)
>at com.google.protobuf.AbstractMessage
> $Builder.mergeFrom(AbstractMessage.java:298)
>at augnet.client.aim.messages.MessageProtos
> $AugNetMessage.parseFrom(MessageProtos.java:6289)
>at augnet.client.aim.connection.Receiver.run(Receiver.java:47)
>
> while the (sending) c++ side encounters no errors. When we scale down
> the message, no error occurs. Is this a bug in protobuf or are we
> doing something wrong?
>
> Best regards,
>  Dominik
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Merge two messages into one file

[Kenton Varda]

My apologies for not having added this to the FAQ page yet.  Too many little
things to do...  :(

On Sun, Oct 12, 2008 at 10:18 AM, Chris <[EMAIL PROTECTED]> wrote:

>
> This is a FAQ.
>
> [EMAIL PROTECTED] wrote:
> > Hi, Kenton,
> >
> > The main problem is how can I serialize two different objects into one
> > stream.
> >
> > e.g.  If I serialize the Text object into one file, how can I append
> > another object Image into the file.
> >  Also the methods how to deserialize?   Thanks.
> >
> The normal API for serializing messages produces a series of numbered
> fields on the wire.  This is not a delimited format, so if you
> concatenate two such encodings you get nonsense unless they are the
> exact same type (in which case deserializing them produces the the same
> result as the two messages being merged into one).
>
> There have been _many_ threads on the this list that talk about how to
> have delimited messages.  The main need is to write the length of the
> message before the message itself and use this when writing your
> decoder.  There are several equivalent but incompatible ways to do
> this.  Some people prefer to write a field tag to the wire as well,
> which means the result looks like a proper protocol-buffer on the wire,
> but for some meta-message that may or may not have been explicitly defined.
>
> Mainly I have not heard that the main Google protobuf API provids a
> simple agreed upon way to write and read back a delimited message.  The
> Haskell project (which is mine) and a C# project both have ways of doing
> this.
>
> Hopefully this FAQ will eventually lead to a simple delimited API being
> added to all the implementations.
>
> --
> Chris
>
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: 2.0.2 release is up

[Kenton Varda]

On Mon, Oct 13, 2008 at 2:26 AM, edan <[EMAIL PROTECTED]> wrote:

> I quick google didn't turn up much for me on what that actually means
> - does it mean anything to y'all?


Yes, this was an overzealous warning present in one version of GCC which was
promptly toned down in subsequent versions.  You can ignore it.


> In any case, using gcc 4.1.2, "make" and "make check" (any reason you
> didn't use the more standard "make test"?) succeeded for me, so I
> guess I will have to just wait to update to protobuf-2.0.2 until I can
> move myself to the newer gcc.


I'll try to look into this more when I get a chance.  You should be fine
sticking with 2.0.1, though, as long as you weren't hoping to use the new
"custom options" feature.

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: How can I compile the protobuf with stlport?

[Kenton Varda]

Does stlport's hash_map work like SGI's hash_map, or like MSVC's hash_map,
or does it do its own thing?  A brief look at the docs suggest that it is
most similar to SGI's.  In that case, you should only need to change the
#ifdefs in src/google/protobuf/stubs/hash.h so that your compile takes the
non-MSVC branch.  That is, change line 101 from:
  #elif defined(_MSC_VER)

to:

  #elif 0

Let me know if this works.

On Mon, Oct 13, 2008 at 3:08 AM, <[EMAIL PROTECTED]> wrote:

>
> thanks. it's okay, but i am thinking how to use stlport's hash_map
> with protobuf.
> how can i do?
>
> On 10月11日, 上午4时24分, "Kenton Varda" <[EMAIL PROTECTED]> wrote:
> > I'm not familiar with stlport.  One thing you could try doing is editing
> > config.h manually and removing all the HASH_* #defines.  Then the code
> will
> > fall back to using map instead of hash_map, which should work since map
> has
> > a standard interface.
> >
> > On Fri, Oct 10, 2008 at 2:58 AM, <[EMAIL PROTECTED]> wrote:
> >
> > > I'm using stlport, and i want to use protobuf, but I see some
> > > compiling errors about the hash_map.
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Unknown Message Type

[Kenton Varda]

You can create a message like this:

message Container {
  // One of the following will be filled in.
  optional Ping ping = 1;
  optional StatusUpdate status_update = 2;
  optional FindWorker find_worker = 3;
}

Now send Containers over the wire instead of instances of the individual
message types.

On Tue, Oct 14, 2008 at 11:37 AM, <[EMAIL PROTECTED]> wrote:

>
> Hi,
>
> I was wondering if it was possible to extract the contents of a
> serialized string without knowing ahead of time what kind of message
> it is?
>
> For example: I'm using tcp to send data back and fourth.  On one side
> I create a message and serialize the data, and on the receiving side I
> 'ParseFromString' and check the information.  However, I have
> different types of messages being sent so the receiving side has no
> idea what it is actually getting.
>
> example .proto file [messages actually contain field, taken out for
> simplicity] which creates 'dtm_pb2' after compiling
>
> package dtm;
> message ping {
> }
> message statusUpdate {
> }
> message findWorker {
> }
>
> so in my code on the receiving end I have:
> msg = dtm_pb2()
> msg.ParseFromString(message)
>
> this gives an error.
>
> Is there any way to initialize the msg variable and extract the
> information??
> If I knew what kind of message it was I could always use:
> msg = dtm_pb2.statusUpdate()
> msg.ParseFromString(message)
> and this would work fine, but this is no a viable solution for me.
>
> thanks,
>
> Nima
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Problems applying new options to descriptor.proto

[Kenton Varda]

Yeah, I think special-casing descriptor.proto might be the only way.  :(
On Tue, Oct 14, 2008 at 12:28 PM, Jon Skeet <[EMAIL PROTECTED]> wrote:

>
> One of the major benefits of descriptor.proto, as I saw it, was that
> we could get away from the hard-coded extra options as there are for
> Java.
>
> So, I've been trying to define my C# options as extensions to
> FileOptions etc. However, I need to apply those options to
> descriptor.proto itself, in order to get it to generate in the right
> namespace etc. That leads to a recursive problem: csharp_options.proto
> needs to import descriptor.proto in order to declare what it's
> extending, and descriptor.proto needs to import csharp_options.proto
> in order to apply those options.
>
> Can anyone think of a way round this? At the moment I suspect I'm just
> going to have to hard-code the namespace and class name into the
> generator when it sees descriptor.proto, but that's really ugly...
>
> Jon
>
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Popping values from repeated fields

[Kenton Varda]

In which language?

On Wed, Oct 15, 2008 at 1:50 AM, Andy <[EMAIL PROTECTED]> wrote:

>
> Hi,
> We can push values onto repeated fields. Is there a way of popping
> values from the same?
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Popping values from repeated fields

[Kenton Varda]

Currently, no, you can only clear the whole list and start over.  I would be
amenable to a patch, though, if people feel this would be a useful addition.

On Wed, Oct 15, 2008 at 8:36 PM, Andrew Wachira <[EMAIL PROTECTED]> wrote:

> In the Java language.
>
> On 10/15/08, Kenton Varda <[EMAIL PROTECTED]> wrote:
> > In which language?
> >
> > On Wed, Oct 15, 2008 at 1:50 AM, Andy <[EMAIL PROTECTED]> wrote:
> >
> >>
> >> Hi,
> >> We can push values onto repeated fields. Is there a way of popping
> >> values from the same?
> >> > >>
> >>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: How should I Input the path of compiling .proto files?

[Kenton Varda]

You need to specify a --proto_path, like:
protoc --proto_path=C:\Projects\Apps_main\C++\Common --cpp_out=.
C:\Projects\Apps_main\C++\Common\test.proto

There is a bug in protoc which causes it to print the wrong error here -- it
should have reported that your input file was not in the proto_path.  That
will be fixed in the next version.

On Thu, Oct 16, 2008 at 9:41 AM, <[EMAIL PROTECTED]> wrote:

>
> I have the same problem.  Tried 2.0.2 release, the problem still
> exist.
>
> Here the error:
>
> protoc.exe  --cpp_out=. C:\Projects\Apps_main\C++\Common\test.proto
>
> C:/Projects/Apps_main/C++/Common/test.pb.h: while trying to create
> directory ./C:: Invalid argument
> C:/Projects/Apps_main/C++/Common/test.pb.cc: while trying to create
> directory ./C:: Invalid argument
>
> On Sep 2, 2:22 pm, "Kenton Varda" <[EMAIL PROTECTED]> wrote:
> > On Tue, Sep 2, 2008 at 10:06 AM, <[EMAIL PROTECTED]> wrote:
> > > the console display:  " Unknow generator option: d"
> >
> > This is a bug which will be fixed in 2.0.1 which should be released any
> day
> > now.  You can get the release candidate here:
> >
> > http://groups.google.com/group/protobuf/files
> >
> >
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Protocol Buffer as XCode Framework

[Kenton Varda]

I'm not sure how mac frameworks work, but it looks like your compiler is not
linking against the protobuf library.

On Thu, Oct 16, 2008 at 9:41 AM, Carlos Gamboa <[EMAIL PROTECTED]> wrote:

>
> Hello. I am trying to add the protocol buffer to my application (for
> iphone) and i have been struggling for about 2 days to figure out how
> to do it. I am new to XCode too by the way. I tried creating a
> framework like Brian Olson suggested on this post:
>
> http://groups.google.com/group/protobuf/browse_thread/thread/65e781a5e8d6fc06#
> , but i do not know all the things i need to include this framework.
> Here are the steps im following.
> 1. Compile the protobuf project
> 2. On my project, right click on Frameworks and click 'add->existing
> frameworks' and then locate the 'macosx/build/release/
> protobuf.framework' folder. The framework is located at the documents
> folder (documents/macosx)
> 3. I created my .pb.h and .pb.c with the 'protoc' program and added
> them to my project.
> 4. I try to compile, but the following 18 errors appear like this:
>  "google::protobuf::UnknownFieldSet::UnknownFieldSet()", referenced
> from:
>   myNamespace::myProto::myProto() in myProto.pb.o
> ...
>
> I really dont care if i add it as a framework or as direct code, but i
> have not been able to include the protocol buffer as part of my
> project. Can anyone please help me?
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: using RPC on asynchronous server

[Kenton Varda]

Many (most?) servers at Google use asynchronous protobuf RPC.  I'm not sure
how to answer your question, though.  Can you be more specific?

On Tue, Oct 14, 2008 at 7:16 AM, muqu <[EMAIL PROTECTED]> wrote:

>
> Anybody can shine some light on their experience on using the PB RPC
> on an asynchronous server, like whether RPC is well-designed for it or
> any potential pitfalls? I find the for async server, matching a
> response to the original request is kind of a pain.
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Patch to quiet compiler warnings from generated service code

[Kenton Varda]

I've made this change in SVN.  Thanks.

On Thu, Oct 2, 2008 at 12:40 PM, Caleb <[EMAIL PROTECTED]> wrote:

>
> When run with high warning levels, gcc complains that the 'request'
> and 'response' arguments are unused for all of the stub service
> functions generated by the protoc compiler.  Here's the 2-line patch
> to fix this.
>
> --- src/google/protobuf/compiler/cpp/cpp_service.cc.orig
> 2008-10-02 15:36:56.0 -0400
> +++ src/google/protobuf/compiler/cpp/cpp_service.cc 2008-10-02
> 15:37:07.0 -0400
> @@ -219,8 +219,8 @@
>
> printer->Print(sub_vars,
>   "void $classname$::$name$(::google::protobuf::RpcController*
> controller,\n"
> -  " const $input_type$* request,\n"
> -  " $output_type$* response,\n"
> +  " const $input_type$*,\n"
> +  " $output_type$*,\n"
>   " ::google::protobuf::Closure* done)
> {\n"
>   "  controller->SetFailed(\"Method $name$() not implemented.\");
> \n"
>   "  done->Run();\n"
>
>
>
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: protocol buffers not repeating items marked repeated

[Kenton Varda]

I don't see any obvious problems.  Try printing the whole message just
before serializing and just after parsing to see if they are both as you
expect.

On Sat, Oct 18, 2008 at 11:24 PM, comsatcat <[EMAIL PROTECTED]> wrote:

>
> I have a simple client server in python... I have the following
> protobuf definition:
>
> message NodeListData {
>enum State {
>FALSE = 0;
>TRUE = 1;
>}
>required string node = 1;
>required string folder = 2;
>required State pendingcsr = 3;
> }
>
> message NodeList {
>repeated NodeListData items = 1;
> }
>
> message Packet {
>enum Type {
>ACK = 0;
>KEEP_ALIVE = 1;
>AUTHORIZE = 2;
>ADD_ENDPOINT = 3;
>GET_ENDPOINT_STATS = 4;
>ENDPOINT_STATS = 5;
>REMOVE_ENDPOINT = 6;
>NODE_LIST = 7;
>}
>required Type type = 1;
>optional Ack ack = 2;
>optional KeepAlive keepalive = 3;
>optional Authorize authorize = 4;
>optional AddEndpoint addendpoint = 5;
>optional EndpointStats endpointstats = 6;
>optional RemoveEndpoint removeendpoint = 7;
>optional NodeList nodelist = 8;
> }
>
> *note some fields have been removed for shortening this message*
>
> My problem is with the NodeList type...
>
> I've verified in my server that I am doing a
> packet.nodelist.items.add(), then proceeding to set the required
> values of the returned NodeItemList.
>
> I then send the packet over the wire.
>
> I receive the packet from the client and do a:
>
> for i in packet.nodelist.items:
>print str(i.node)
>
> and it only comes back with 1 result, no matter how many I build in
> the server (I've tried up to 3)...
>
> Any ideas?
>
> I'm doing this exactly the same way with a different packet type and
> it works fine... am I just missing something small?
>
> Thanks,
> Ben
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---

Re: Cross compiling

[Kenton Varda]

The code generated by protoc is identical on all platforms, so you can use
the same protoc that you compiled for Linux.  You only have to cross-compile
the libprotobuf library.  Cross-compiling should work exactly like it does
with any other autoconf-based package.

On Mon, Oct 20, 2008 at 6:58 AM, <[EMAIL PROTECTED]> wrote:

>
> Hi, I am building and using protocol buffers 2.0.1 on a Linux host and
> it works fine.
>
> Now I am moving to a target hardware with an ARM9 cpu, also running
> Linux  (and later to an ARM7 with no os) and would like to cross
> compile to these platforms. I have tried to find information on how to
> do this but have not found any information on the subject.
>
> Has anyone tested this? Can anybody help me out?
>
> Thanks for your help!
>
> /Magnus
> >
>

--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---