Re: "Lite" mode is in SVN -- mostly
On Sat, Aug 1, 2009 at 4:42 AM, Jon Skeet wrote: > Out of interest Kenton, does this make the bootstrapping code simpler? > I'd imagine that can be built with just a "lite" version. It would be > nice to get rid of some of the nastiness that's involved in C# just to > get the PB-specific types to work :) In theory, descriptor.proto could become "lite", and this might simplify bootstrapping. In practice, though, we actually use "heavy" features on the types in descriptor.proto, which is part of the reason why bootstrapping is so complicated. So, no, I don't think that would work. > > > Jon > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: "Lite" mode is in SVN -- mostly
On Jul 29, 3:50 am, Kenton Varda wrote: > I just committed the new "lite mode" refactoring for C++ and Java to SVN. > To use the new feature, add this line to your .proto file: > option optimize_for = LITE_RUNTIME; > This will cause protoc to generate code which is just as fast as with > optimize_for = SPEED, but lacks descriptors, reflection, UnknownFieldSet, > and features which are tied to them. This means the code can be linked > against a much smaller subset of the protocol buffers runtime library and > will not spend time at start-up building descriptors. Just in case anyone's wondering, I'm hoping to at least *start* work on porting this to C# early next week. (I have two days dedicated to 20% time next week, but I also want to do some Wave work.) Out of interest Kenton, does this make the bootstrapping code simpler? I'd imagine that can be built with just a "lite" version. It would be nice to get rid of some of the nastiness that's involved in C# just to get the PB-specific types to work :) Jon --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: "Lite" mode is in SVN -- mostly
On Wed, Jul 29, 2009 at 3:41 PM, ijuma wrote: > Is there a way to disable this behaviour? As you say, it doesn't seem > particularly useful for sophisticated JVMs and unless I am missing > something, the queue is currently unbounded so it could end up > retaining quite a bit of memory for some usage patterns. There's no way to disable it at present, but I would be fine with adding a way. In fact I think this should probably be off by default. > > > Best, > Ismael > > > --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: "Lite" mode is in SVN -- mostly
On Jul 29, 7:15 pm, Kenton Varda wrote: > I think that touched java_message.cc in the compiler -- grep for > "QuickQueue". Thanks Kenton. > It's an entirely transparent change -- you do not have to > change your code at all to take advantage of it. Yes, I understand. I was just curious about the implementation and the motivation for the change. The answer for the first is that it uses a thread-local queue and you gave the answer for the second question. > I'm told it improved > performance on Dalvik (Android's JVM), which has a relatively weak garbage > collector. I doubt it helps any on the official JVM. Is there a way to disable this behaviour? As you say, it doesn't seem particularly useful for sophisticated JVMs and unless I am missing something, the queue is currently unbounded so it could end up retaining quite a bit of memory for some usage patterns. Best, Ismael --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: "Lite" mode is in SVN -- mostly
On Tue, Jul 28, 2009 at 7:50 PM, Kenton Varda wrote: > C++, as measured by the "size" command on Linux: > libprotobuf.so: 948k > libprotobuf-lite.so: 148k > libprotobuf-lite.so with extension_set.cc removed: 91k > I just shaved another 23k off the lite library by eliminating use of strutil. So now it's 125k or 68k without extensions. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: "Lite" mode is in SVN -- mostly
On Wed, Jul 29, 2009 at 2:14 AM, ijuma wrote: > On Jul 29, 3:50 am, Kenton Varda wrote: > > I just committed the new "lite mode" refactoring for C++ and Java to SVN. > > Thanks, that sounds useful. I noticed another change in the > Changes.txt: > > "Put Builder objects on a freelist after build() is called, so they > may be reused later." > > Since the diff is quite huge, I was not able to find where this change > was done. If you could point out where I should look, that would be > appreciated. I think that touched java_message.cc in the compiler -- grep for "QuickQueue". It's an entirely transparent change -- you do not have to change your code at all to take advantage of it. I'm told it improved performance on Dalvik (Android's JVM), which has a relatively weak garbage collector. I doubt it helps any on the official JVM. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: "Lite" mode is in SVN -- mostly
I'll try to get to this by the end of the week. On Tue, Jul 28, 2009 at 9:50 PM, Kenton Varda wrote: > I just committed the new "lite mode" refactoring for C++ and Java to SVN. > To use the new feature, add this line to your .proto file: > option optimize_for = LITE_RUNTIME; > This will cause protoc to generate code which is just as fast as with > optimize_for = SPEED, but lacks descriptors, reflection, UnknownFieldSet, > and features which are tied to them. This means the code can be linked > against a much smaller subset of the protocol buffers runtime library and > will not spend time at start-up building descriptors. > Currently, the C++ Makefile will compile libprotobuf and libprotobuf-lite as > independent, stand-alone libraries, even though the latter is a subset of > the former. Is this what we want? I'm not sure. If we make libprotobuf > depend on libprotobuf-lite, then users who want the full library will have > to specify *both* when linking their own apps, unless we provide a > "protobuf-config" script or something like that. Complicating things > further, if you app does not use extensions, then extension_set.cc can be > removed from the lite library to make it even smaller, but having three > separate libraries just seems excessive! > For Java, I have not yet updated the Maven POM to separate the libraries. > I'm not sure how -- Greg, can you work on this? "Lite" messages need only > the following classes: > AbstractMessageLite.java > ByteString.java > CodedInputStream.java > CodedOutputStream.java > ExtensionRegistryLite.java > FieldSet.java > GeneratedMessageLite.java > InvalidProtocolBufferException.java > Internal.java > MessageLite.java > UninitializedMessageException.java > WireFormat.java > Numbers > C++, as measured by the "size" command on Linux: > libprotobuf.so: 948k > libprotobuf-lite.so: 148k > libprotobuf-lite.so with extension_set.cc removed: 91k > Java, measured by totalling the sizes of the .class files: > full library: 904k > lite library: 120k > I suspect we can further reduce some of these numbers with some more work. > For instance, an extension-less Java lite library might be as much as 50k > smaller, though will require a bit more work than simply yanking some > classes. -- Greg Kick Data Liberation & Feedburner Sent from Chicago, IL, United States --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
Re: "Lite" mode is in SVN -- mostly
Hi Kenton, On Jul 29, 3:50 am, Kenton Varda wrote: > I just committed the new "lite mode" refactoring for C++ and Java to SVN. Thanks, that sounds useful. I noticed another change in the Changes.txt: "Put Builder objects on a freelist after build() is called, so they may be reused later." Since the diff is quite huge, I was not able to find where this change was done. If you could point out where I should look, that would be appreciated. Best, Ismael --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
"Lite" mode is in SVN -- mostly
I just committed the new "lite mode" refactoring for C++ and Java to SVN. To use the new feature, add this line to your .proto file: option optimize_for = LITE_RUNTIME; This will cause protoc to generate code which is just as fast as with optimize_for = SPEED, but lacks descriptors, reflection, UnknownFieldSet, and features which are tied to them. This means the code can be linked against a much smaller subset of the protocol buffers runtime library and will not spend time at start-up building descriptors. Currently, the C++ Makefile will compile libprotobuf and libprotobuf-lite as independent, stand-alone libraries, even though the latter is a subset of the former. Is this what we want? I'm not sure. If we make libprotobuf depend on libprotobuf-lite, then users who want the full library will have to specify *both* when linking their own apps, unless we provide a "protobuf-config" script or something like that. Complicating things further, if you app does not use extensions, then extension_set.cc can be removed from the lite library to make it even smaller, but having three separate libraries just seems excessive! For Java, I have not yet updated the Maven POM to separate the libraries. I'm not sure how -- Greg, can you work on this? "Lite" messages need only the following classes: AbstractMessageLite.java ByteString.java CodedInputStream.java CodedOutputStream.java ExtensionRegistryLite.java FieldSet.java GeneratedMessageLite.java InvalidProtocolBufferException.java Internal.java MessageLite.java UninitializedMessageException.java WireFormat.java *Numbers* C++, as measured by the "size" command on Linux: libprotobuf.so: 948k libprotobuf-lite.so: 148k libprotobuf-lite.so with extension_set.cc removed: 91k Java, measured by totalling the sizes of the .class files: full library: 904k lite library: 120k I suspect we can further reduce some of these numbers with some more work. For instance, an extension-less Java lite library might be as much as 50k smaller, though will require a bit more work than simply yanking some classes. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/protobuf?hl=en -~--~~~~--~~--~--~---
