Re: Compressed protobufs?

2008-10-21 Thread GDR
Cool. I think this makes sense as I could use GZIPInputStream with mergeFrom() and parseFrom(). On Oct 21, 4:20 pm, "Kenton Varda" <[EMAIL PROTECTED]> wrote: > There is no built-in support for this, but there's nothing stopping you from > applying whatever compression you want to the serialized d

Re: Compressed protobufs?

2008-10-21 Thread Marc Gravell
It /might/ work (especially if there are a lot of repeated strings) - but from my own tests, "compressing" pb-serialized data actually made it bigger (always a good sign for a dense format). Quite possibly Google have seen better results, but it doesn't work for me... Marc --~--~-~--~

Re: Data structures using protocol buffers

2008-10-21 Thread Jeremy Leader
Keep in mind that protobufs describe serialized data, and there's no concept of an object reference like Java uses. In your example, if A and B are neighbors, then in your proto, the data representing A contains the data representing B, and the data representing B contains the data representi

Failed Test

2008-10-21 Thread tedspecht
I just ran into some test errors before installing the python tools and after installing the protocol buffer libraries. The environment I am using is a Quad-Core AMD Opteron(tm) Processor 8356 with 132104376 kB of memory, 2.6.18-6-amd64. I have tried tried python2.5 and python2.4. The errors are

Data structures using protocol buffers

2008-10-21 Thread GDR
Hi, I'm wondering how would one go about implementing self-referential data structures? As an exercise, I tried to implement a PB version of the adjacency list representation of a graph. I'm having a hard time getting it work. Any suggestions? Thanks! --- graph.proto --- package graph; optio

Re: importing across dlls

2008-10-21 Thread jshuberg
Excellent, thanks!! On Oct 21, 3:19 pm, "Kenton Varda" <[EMAIL PROTECTED]> wrote: > On Tue, Oct 21, 2008 at 12:13 PM, <[EMAIL PROTECTED]> wrote: > > Would simply changing FileGenerator::GenerateHeader to output the > > dllexport_decl in the forward declaration of BuildDescriptors function > > wor

Re: Compressed protobufs?

2008-10-21 Thread Kenton Varda
There is no built-in support for this, but there's nothing stopping you from applying whatever compression you want to the serialized data. On Tue, Oct 21, 2008 at 1:08 PM, GDR <[EMAIL PROTECTED]> wrote: > > I noticed the protocol buffers store strings in their uncompressed > form. Is there any p

Re: importing across dlls

2008-10-21 Thread Kenton Varda
On Tue, Oct 21, 2008 at 12:13 PM, <[EMAIL PROTECTED]> wrote: > Would simply changing FileGenerator::GenerateHeader to output the > dllexport_decl in the forward declaration of BuildDescriptors function > work? Yes, that is the fix. --~--~-~--~~~---~--~~ You recei

Compressed protobufs?

2008-10-21 Thread GDR
I noticed the protocol buffers store strings in their uncompressed form. Is there any provision to produce and consume protobufs in compressed form? This would be specially helpful if a lot of fields were repeated strings. --~--~-~--~~~---~--~~ You received this mes

Re: decrease libprotobuf size?

2008-10-21 Thread idht4n
On Oct 21, 10:31 am, "Kenton Varda" <[EMAIL PROTECTED]> wrote: > Yeah, protocol buffers were designed for servers where we often have absurd > binary sizes already and don't mind that much.  I'm certainly open to the > idea of creating a stripped-down implementation -- though this isn't > someth

Re: importing across dlls

2008-10-21 Thread jshuberg
I'm using jskeet's dotnet-protobuf implementation, because we need C# support. I believe it's based on 2.0.1. Would simply changing FileGenerator::GenerateHeader to output the dllexport_decl in the forward declaration of BuildDescriptors function work? On Oct 21, 1:27 pm, "Kenton Varda" <[EMAIL

importing across dlls

2008-10-21 Thread jshuberg
I have a protoc file named UniqueID.proto that is compiled into and exported from Communication.dll. I have a protoc file named Base.proto that is compiled into and exported from Base.dll. Base.proto imports and uses UniqueID.proto. Base.dll imports the UniqueID class. Importing and exporting is

Re: importing across dlls

2008-10-21 Thread Kenton Varda
I believe this was fixed in 2.0.2. What version are you using? On Tue, Oct 21, 2008 at 11:21 AM, <[EMAIL PROTECTED]> wrote: > > I have a protoc file named UniqueID.proto that is compiled into and > exported from Communication.dll. > I have a protoc file named Base.proto that is compiled into and

Re: Flush operation on ZeroCopyOutputStream

2008-10-21 Thread Kenton Varda
Yeah, Frank's approach is the one I'd use. I did not add a Flush() because I was worried that its meaning would be completely ambiguous. For example, when you flush a stream which is writing to disk, should it do an fsync() to make sure that the data actually reaches the physical device before it

Re: Why copy read stream and zero-copy read streams are not compatible?

2008-10-21 Thread Kenton Varda
You could always have your class implement ZeroCopyInputStream and *contain* an implementation of CopyingInputStream, like: class MyStream : public ZeroCopyInputStream { public: MyStream(); virtual ~MyStream(); CopyingInputStream* AsCopyingInputStream() { return ©ing_stream_

Re: cross compile errors

2008-10-21 Thread Kenton Varda
For the first case, the problem is that the build is trying to run protoc as part of the build process, and it's trying to use the cross-compiled protoc. I think you can fix this by editing src/Makefile.am at line 201 to change this: unittest_proto_middleman: protoc$(EXEEXT) $(protoc_inputs) ./$

Re: decrease libprotobuf size?

2008-10-21 Thread Kenton Varda
Yeah, protocol buffers were designed for servers where we often have absurd binary sizes already and don't mind that much. I'm certainly open to the idea of creating a stripped-down implementation -- though this isn't something I'd have time to work on myself. It might also help to look for uses o

Re: MSVC build switched to static linking

2008-10-21 Thread Kenton Varda
You're right. Thanks for catching that. I have updated the file in SVN. On Tue, Oct 21, 2008 at 4:31 AM, <[EMAIL PROTECTED]> wrote: > > in protocolBuffers\vsprojects\readme.txt, the "Compiling and > Installing" section seems to still talk about DLLs being built by > default? > > > > On Sep 30,

decrease libprotobuf size?

2008-10-21 Thread idht4n
Are there configuration options to reduce the size of libprotobuf? I'm getting a 1,023,544 byte library stripped for ppc. This is for an embedded system with limited memory. I think we're only using a fraction of the functionality of protocol buffers. (parseFromArray is the only explicit call pl

cross compile errors

2008-10-21 Thread idht4n
I'm trying to cross compile for a ppc platform from x86 Linux. I've used the same procedure as I used to successfully cross compile other tools (not to say that the procedure is actually right) but I get some errors. Here's what I tried: setenv CC /opt/cross_compiler/ppc/gcc-4.1.0-glibc-2.3.6/po

Re: MSVC build switched to static linking

2008-10-21 Thread ojwlists
in protocolBuffers\vsprojects\readme.txt, the "Compiling and Installing" section seems to still talk about DLLs being built by default? On Sep 30, 1:45 am, "Kenton Varda" <[EMAIL PROTECTED]> wrote: > Hi all, > Just before building the 2.0.2 release candidate, I submitted a change to > make stat

Re: Patch for zero_copy_stream_impl.cc

2008-10-21 Thread Frank Chu
Here's the patch, attached. On Mon, Oct 20, 2008 at 2:39 PM, Kenton Varda <[EMAIL PROTECTED]> wrote: > Oh, and thanks for catching these! > > > On Mon, Oct 20, 2008 at 2:38 PM, Kenton Varda <[EMAIL PROTECTED]> wrote: > >> Would you care to add some tests to the unit test to check for these bugs?

Re: Patch for zero_copy_stream_impl.cc

2008-10-21 Thread fpmchu
Here's the patch including the patch + unittests. wait I can't send attachment through groups??? sigh I will send it to you via email. Frank On Oct 20, 2:39 pm, "Kenton Varda" <[EMAIL PROTECTED]> wrote: > Oh, and thanks for catching these! > > On Mon, Oct 20, 2008 at 2:38 PM, Kenton Varda