Re: zero_copy_stream_unittest.cc doesn't compile with msvc 7.1/2003
OK, I'll make the #if check the version number in the next release. On Tue, Jul 7, 2009 at 12:15 AM, Cosmin Cremarenco < cosmin.cremare...@gmail.com> wrote: > > It is 1310 > > On Jul 6, 7:49 pm, Kenton Varda wrote: > > What's the _MSC_VER for that version? > > On Mon, Jul 6, 2009 at 10:37 AM, Cosmin Cremarenco < > > > > > > > > cosmin.cremare...@gmail.com> wrote: > > > > > It is Visual Studio 2003 - Version 7.1.6030. > > > I think the .1 minor comes from the installation of sp1 over 7.0 > > > > > Cosmin > > > > > On Jul 6, 7:33 pm, Kenton Varda wrote: > > > > This is with MSVC 7.1? Which version of Visual Studio does that > > > correspond > > > > to? I can never keep these straight. > > > > > > On Mon, Jul 6, 2009 at 7:27 AM, Cosmin Cremarenco < > > > > > > cosmin.cremare...@gmail.com> wrote: > > > > > > > Hi > > > > > > > When compiling zero_copy_stream_unittest.cc from the last version > > > > > 2.1.0 msvc complains about the > > > > > type "_invalid_parameter_handler" (I don't think it is supported in > > > > > 7.1): > > > > > Compiling... > > > > > zero_copy_stream_unittest.cc > > > > > \tmp\protobuf-2.1.0-vc71\src\google\protobuf\io > > > > > \zero_copy_stream_unittest.cc(475) : error C2146: syntax error : > > > > > missing ';' before identifier 'old_handler_' > > > > > \tmp\protobuf-2.1.0-vc71\src\google\protobuf\io > > > > > \zero_copy_stream_unittest.cc(475) : error C2501: > > > > > 'google::protobuf::io::`anonymous- > > > > > namespace'::MsvcDebugDisabler::_invalid_parameter_handler' : > missing > > > > > storage-class or type specifiers > > > > > \tmp\protobuf-2.1.0-vc71\src\google\protobuf\io > > > > > \zero_copy_stream_unittest.cc(475) : error C2501: > > > > > 'google::protobuf::io::`anonymous- > > > > > namespace'::MsvcDebugDisabler::old_handler_' : missing > storage-class > > > > > or type specifiers > > > > > > > Changing the wrapping conditional compilation macro from "#if > > > > > _MSC_VER" to "#if 0" fixes it. > > > > > > > Best regards, > > > > > > > Cosmin > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Protocol buffers extensions getting lost on the way?
The javadoc for ExtensionRegistry discusses the reasons why we didn't take
the auto-populated approach as was done in C++. In fact, I sort of wish we
had done something like ExtensionRegistry in C++ as well, as I get a lot of
messages from people who are confused why their program behaves differently
when they simply link in an extra library without actually using it. Also,
technically the C++ standard does not require a translation unit to be
initialized until it is first accessed (similar to Java), but it just
happens that most compilers do all initialization at startup time.
On Wed, Jul 8, 2009 at 9:38 AM, ProzakLord wrote:
>
> Since I figured it out I might as well put the solution here.
>
> The problem was on the java side.
>
> The call to parse has to be modified so that it looks in the following
> way(there may be a better option out there but this does it for me).
>
> ExtensionsRegistry registry = ExtensionsRegistry.newInstance();
>
> registry.add(Application.operation1);
> registry.add(Application.operation2);
>
> PbFrameworkMessage request = PbFrameworkMessage.parseFrom
> (networkBuffer,registry);
>
> Thats it.
>
> Now the bonus points are is there a way in java for the
> ExtensionsRegistry to autopopulate like in C++? is it already so?
>
> Kind regards.
>
> On 8 jul, 16:14, ProzakLord wrote:
> > Hi all,
> >
> > I was wondering if anyone had run across the same issue.
> >
> > The issue is the following. I have a bunch of protocl buffers that I
> > use for the communnication between client and server, the client is in
> > C++ the server in java. It was all running fine until I made the
> > following refactoring in the code:
> >
> > I went from having something along the lines of
> >
> > message PbApplicationMessage {
> > optional PbApplicationOP1Request operation1 = 1;
> > optional PbApplicationOP2Request operation2 = 2;
> >
> > }
> >
> > to more generic solution using extensions
> > Framework proto file
> >
> > message PbFrameworkMessage {
> > required PbApplication Message appMessage = 1;
> >
> > }
> >
> > message PbApplicationMesasge{
> >extensions 1 to 18999;
> >
> > }
> >
> > and in my application proto file
> > #import "framework.proto"
> >
> > extend PbApplicationMessage {
> > optional PbApplicationOP1Request operation1 = 1;
> > oprional PbApplicationOP2Request operation2 = 2;
> >
> > }
> >
> > I refactored all the code for what could be required. I compile the
> > protocol buffers protofile
> > so that both generated files are available on the same package for
> > both client and server.
> >
> > The framework.proto is compile only once to be part of a library that
> > is statically linked to the client version.
> >
> > Now the problem that I have is teh following.
> > I serialize in C++ the Framework Message after I have put the
> > extension in it by using the
> >
> > request->mutable_appmessge().MutableExtenstion(operation1)
> >
> > Just before calling the request->SerializeToString(string) I check
> > with
> > request->appmessage().HasExtension(operation1) and that returns true.
> >
> > Until here all is fine. I then dump that to the wire
> > and when on the java side I pick up the protocolbuffer I do the
> > following
> >
> > PbFrameworkMessage request = PbFrameworkMessage.parseFrom
> > (networkBuffer);
> > I then do a request.getAppMessage().hasExtension
> > (Application.operation1) which returns false
> >
> > The network code that puts things to and from the wire has been
> > running without the extensions sending portocol buffers around with
> > absolutely no problems.
> >
> > Am I missing something obvious that I ought to do before the serialize
> > to String on the client, whic is required by the enxtensions and is
> > not in the nromal version?
> >
> > I am running protobuf 2.0.0beta on both sides.
> >
> > Any help or pointer will be much appreciated.
> >
> > Kind regards.
> >
>
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Re: Maximizing fixed size buffer without losing the whole message
Sorry, there is no magic solution here. The best suggestions I have are either: 1) Use SerializeToString() instead, which will always allocate a string large enough to hold the message. 2) Build your message, then call ByteSize(), and if it is too large, remove some fields and try again. Once it is big enough, call SerializeWithCachedSizesToArray() -- since you already called ByteSize(), the sizes are already cached. This way, if the message fits on the first attempt, you perform just as well as you did before, because SerializeToArray() calls ByteSize() followed by SerializeWithCachedSizesToArray(). On Wed, Jul 8, 2009 at 4:25 AM, edan wrote: > Hi- > We have a fixed-size buffer into which we are serializing a message (a > pretty large and convoluted message, with many optional sub-messages and > repeated fields, etc). > We have found that when the message size gets bigger than the buffer size, > SerializeToArray just fails to write anything - we lose the whole message. > We'd like to be able to write as much message as we can, and just give up > on some of the optional fields, or, more likely, to stop adding elements to > repeated fields, etc. > What is the best way to do this efficiently? It seems that calling > ByteSize() all the time will cost us in performance. Is there a way to > efficiently compute the space needing during our message-building phase? Or > is there a way to serialize as we go using some of the append methods, and > then just stop appending when we reach the limit? What I don't understand > is how to avoid re-appending the same parts of the message that we already > serialized? Let's also assume that most of the places we are likely to "run > out of space" is while we are adding more and more elements to repeated > fields. > Thanks for any help. > --edan > > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: Protocol buffers extensions getting lost on the way?
Since I figured it out I might as well put the solution here.
The problem was on the java side.
The call to parse has to be modified so that it looks in the following
way(there may be a better option out there but this does it for me).
ExtensionsRegistry registry = ExtensionsRegistry.newInstance();
registry.add(Application.operation1);
registry.add(Application.operation2);
PbFrameworkMessage request = PbFrameworkMessage.parseFrom
(networkBuffer,registry);
Thats it.
Now the bonus points are is there a way in java for the
ExtensionsRegistry to autopopulate like in C++? is it already so?
Kind regards.
On 8 jul, 16:14, ProzakLord wrote:
> Hi all,
>
> I was wondering if anyone had run across the same issue.
>
> The issue is the following. I have a bunch of protocl buffers that I
> use for the communnication between client and server, the client is in
> C++ the server in java. It was all running fine until I made the
> following refactoring in the code:
>
> I went from having something along the lines of
>
> message PbApplicationMessage {
> optional PbApplicationOP1Request operation1 = 1;
> optional PbApplicationOP2Request operation2 = 2;
>
> }
>
> to more generic solution using extensions
> Framework proto file
>
> message PbFrameworkMessage {
> required PbApplication Message appMessage = 1;
>
> }
>
> message PbApplicationMesasge{
> extensions 1 to 18999;
>
> }
>
> and in my application proto file
> #import "framework.proto"
>
> extend PbApplicationMessage {
> optional PbApplicationOP1Request operation1 = 1;
> oprional PbApplicationOP2Request operation2 = 2;
>
> }
>
> I refactored all the code for what could be required. I compile the
> protocol buffers protofile
> so that both generated files are available on the same package for
> both client and server.
>
> The framework.proto is compile only once to be part of a library that
> is statically linked to the client version.
>
> Now the problem that I have is teh following.
> I serialize in C++ the Framework Message after I have put the
> extension in it by using the
>
> request->mutable_appmessge().MutableExtenstion(operation1)
>
> Just before calling the request->SerializeToString(string) I check
> with
> request->appmessage().HasExtension(operation1) and that returns true.
>
> Until here all is fine. I then dump that to the wire
> and when on the java side I pick up the protocolbuffer I do the
> following
>
> PbFrameworkMessage request = PbFrameworkMessage.parseFrom
> (networkBuffer);
> I then do a request.getAppMessage().hasExtension
> (Application.operation1) which returns false
>
> The network code that puts things to and from the wire has been
> running without the extensions sending portocol buffers around with
> absolutely no problems.
>
> Am I missing something obvious that I ought to do before the serialize
> to String on the client, whic is required by the enxtensions and is
> not in the nromal version?
>
> I am running protobuf 2.0.0beta on both sides.
>
> Any help or pointer will be much appreciated.
>
> Kind regards.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Protocol buffers extensions getting lost on the way?
Hi all,
I was wondering if anyone had run across the same issue.
The issue is the following. I have a bunch of protocl buffers that I
use for the communnication between client and server, the client is in
C++ the server in java. It was all running fine until I made the
following refactoring in the code:
I went from having something along the lines of
message PbApplicationMessage {
optional PbApplicationOP1Request operation1 = 1;
optional PbApplicationOP2Request operation2 = 2;
}
to more generic solution using extensions
Framework proto file
message PbFrameworkMessage {
required PbApplication Message appMessage = 1;
}
message PbApplicationMesasge{
extensions 1 to 18999;
}
and in my application proto file
#import "framework.proto"
extend PbApplicationMessage {
optional PbApplicationOP1Request operation1 = 1;
oprional PbApplicationOP2Request operation2 = 2;
}
I refactored all the code for what could be required. I compile the
protocol buffers protofile
so that both generated files are available on the same package for
both client and server.
The framework.proto is compile only once to be part of a library that
is statically linked to the client version.
Now the problem that I have is teh following.
I serialize in C++ the Framework Message after I have put the
extension in it by using the
request->mutable_appmessge().MutableExtenstion(operation1)
Just before calling the request->SerializeToString(string) I check
with
request->appmessage().HasExtension(operation1) and that returns true.
Until here all is fine. I then dump that to the wire
and when on the java side I pick up the protocolbuffer I do the
following
PbFrameworkMessage request = PbFrameworkMessage.parseFrom
(networkBuffer);
I then do a request.getAppMessage().hasExtension
(Application.operation1) which returns false
The network code that puts things to and from the wire has been
running without the extensions sending portocol buffers around with
absolutely no problems.
Am I missing something obvious that I ought to do before the serialize
to String on the client, whic is required by the enxtensions and is
not in the nromal version?
I am running protobuf 2.0.0beta on both sides.
Any help or pointer will be much appreciated.
Kind regards.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Protocol Buffers" group.
To post to this group, send email to protobuf@googlegroups.com
To unsubscribe from this group, send email to
protobuf+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/protobuf?hl=en
-~--~~~~--~~--~--~---
Maximizing fixed size buffer without losing the whole message
Hi- We have a fixed-size buffer into which we are serializing a message (a pretty large and convoluted message, with many optional sub-messages and repeated fields, etc). We have found that when the message size gets bigger than the buffer size, SerializeToArray just fails to write anything - we lose the whole message. We'd like to be able to write as much message as we can, and just give up on some of the optional fields, or, more likely, to stop adding elements to repeated fields, etc. What is the best way to do this efficiently? It seems that calling ByteSize() all the time will cost us in performance. Is there a way to efficiently compute the space needing during our message-building phase? Or is there a way to serialize as we go using some of the append methods, and then just stop appending when we reach the limit? What I don't understand is how to avoid re-appending the same parts of the message that we already serialized? Let's also assume that most of the places we are likely to "run out of space" is while we are adding more and more elements to repeated fields. Thanks for any help. --edan --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
