Re: [protobuf] Why not allow null values in Java for scalar types?

2010-12-22 Thread Kenton Varda
There are problems with this: 1) Boxing and unboxing primitives is relatively expensive, compared to just passing them as primitives. If performance matters to you at all (and for many protobuf users, it does), you probably don't want this. 2) If you accept messages from untrusted sources, your

Re: [protobuf] ambiguous Builder errors when compiling a Maven project

2010-12-22 Thread Kenton Varda
I've never seen this error. It doesn't make much sense -- protobuf generated code should not contain any imports. Are you modifying the generated code in some way? On Sun, Dec 19, 2010 at 11:47 PM, Xiao Ling bardm...@gmail.com wrote: Hi All, I have a generated protobuf java file which

Re: [protobuf] Reading and writing multiple messages to a file

2010-12-22 Thread Kenton Varda
You can only have one CodedInputStream wrapping a ZeroCopyInputStream at a time, otherwise they will confuse each other with buffering conflicts. CodedInputStreams are cheap to construct, so you can just make one on the stack in a block, like: { CodedInputStream

Re: [protobuf] ambiguous Builder errors when compiling a Maven project

2010-12-22 Thread Kenton Varda
(that is reponsible for compiling both scala and java code) gives the error messages. Thanks, --Xiao On Wed, Dec 22, 2010 at 4:13 PM, Kenton Varda ken...@google.com wrote: I've never seen this error. It doesn't make much sense -- protobuf generated code should not contain any imports. Are you

Re: [protobuf] Re: Truncated varint decoding a message just built

2010-12-28 Thread Kenton Varda
side doesn't complain, but the decoder does. A bug on my part, but suboptimal behavior for the library. On Dec 22, 6:09 pm, Kenton Varda ken...@google.com wrote: Can you provide a small, self-contained example program demonstrating this problem? On Wed, Dec 22, 2010 at 4:52

Re: [protobuf] Stronger introspection

2010-12-28 Thread Kenton Varda
You can use the type's descriptor to inspect the fields it contains and what all their types are. MyType.DESCRIPTOR is the descriptor for MyType. Once you've figured out what field you want to read/write, use getattr() / setattr(). Note that the C++ and Java interfaces for this are somewhat

Re: [protobuf] Re: Converting JSON Text Into PB Bytes Type

2010-12-28 Thread Kenton Varda
(Encoded: + Base64.encodeBase64String(clearText.getBytes())); } catch (Exception e) { e.printStackTrace(); } } } I'm using this: http://code.google.com/p/protobuf-java-format/ for JSON -- PB conversion On Dec 23, 3:48 am, Kenton

Re: [protobuf] how can we deal with multi-lingual string? (c++)

2010-12-28 Thread Kenton Varda
The dirty little secret of std::wstring is that it does not actually deal with non-ASCII characters on all platforms. On some platforms wchar_t is 8-bit just like char! You should avoid using wstring and wchar_t for this reason; define your own types that are exactly what you need. For protocol

Re: [protobuf] C# client to Java based server

2010-12-28 Thread Kenton Varda
Protocol buffers itself has no built-in RPC implementation. You have to find an RPC implementation that supports whatever languages you are interested in, or write your own. It's not too hard to write a simple RPC implementation given protocol buffers as a base. Sending protobufs over HTTP is a

Re: [protobuf] Transfer protocol buffer object to POJO java object

2010-12-28 Thread Kenton Varda
I don't know of any existing tools for this. You could write code that does this via reflection (protobuf reflection on the protobuf object, and basic java reflection on the POJO). Or, you could write a protoc plugin which generates the code you need, though that will be a lot more complicated.

Re: [protobuf] Re: Transfer protocol buffer object to POJO java object

2010-12-29 Thread Kenton Varda
to get the camel field name out? I don't want to get the original field name name : last_name, first_name and convert it again to camel field name every time that I want to do getter and setter on java pojo (very inefficient). Thanks, On Dec 28, 5:16 pm, Kenton Varda ken...@google.com wrote

Re: [protobuf] Linker issues: undefined reference to `a_message__get_packed_size'

2011-01-04 Thread Kenton Varda
The author of protobuf-c might not read this list. You might want to e-mail him directly. But I would suggest looking at the contents of tut.pb-c.c to see whether it actually defines a_message__pack. Maybe the problem will be obvious. On Tue, Dec 28, 2010 at 3:08 PM, Suhail Doshi

Re: [protobuf] Re: protobuffer suport composite object in stack not in heap?

2011-01-04 Thread Kenton Varda
On Tue, Jan 4, 2011 at 5:25 PM, Igor Gatis igorga...@gmail.com wrote: A while ago, a colleague had a memory leak reusing a PB message which contained a repeated field. If I'm not mistaken the problem was that pb_message::Clear() calls vectorsomething::clear() and string::clear() which does

Re: [protobuf] Using the --proto_path option

2011-01-11 Thread Kenton Varda
On Tue, Jan 11, 2011 at 1:30 PM, Collin blackfr...@gmail.com wrote: I am trying to understand how --proto_path works. Let's say in my .proto file have the following import lines: import descriptor.proto; import google/protobuf/descriptor.proto; Delete the first of these two imports, and

Re: [protobuf] Re: Truncated varint decoding a message just built

2011-01-11 Thread Kenton Varda
I see, I think the encoding problem was fixed in 2.3.0. The type checker is still wrong, though. On Thu, Jan 6, 2011 at 3:17 PM, BigBaaadBob bigbaaad...@gmail.com wrote: 2.2.0a-0.1ubuntu1 On Dec 28 2010, 2:55 pm, Kenton Varda ken...@google.com wrote: Strange. It looks like there is a bug

Re: [protobuf] Problem with java generation when using imports

2011-01-11 Thread Kenton Varda
Can you provide a small, self-contained example? On Mon, Jan 10, 2011 at 7:13 AM, Meghana meghana.vishwan...@gmail.comwrote: Hi, I am using imports in my proto files. So, B has an import of A. When I generate the java code for B, all references to A are specified as A.A i.e. A(dot)A. It

Re: [protobuf] LIBPROTOBUF_EXPORT Missing Headers

2011-01-11 Thread Kenton Varda
OK. What is the problem? On Mon, Jan 10, 2011 at 11:46 AM, Collin blackfr...@gmail.com wrote: The following headers have LIBPROTOBUF_EXPORT in them but are installed with `make install.` google/protobuf/stubs/strutil.h google/protobuf/stubs/substitute.h -- You received this message

Re: [protobuf] Using a ByteBuffer instead of a ByteString?

2011-01-11 Thread Kenton Varda
On Mon, Jan 10, 2011 at 9:45 PM, Nicolae Mihalache xproma...@gmail.comwrote: Hello, I recently started to use GPB, great software! :) But I have noticed in java that it is impossible to create a message containing a bytes fields without copying some buffers around. For example if I have a

Re: [protobuf] Re: 2.4.0 release candidate #1

2011-01-11 Thread Kenton Varda
On Wed, Jan 5, 2011 at 10:33 PM, Scratch scratch@gmail.com wrote: Thanks! That's right what I needed. And one more question - will sub-builders be introduced for lite package? Sorry, but probably not. The whole point of the lite package is to be as small as possible, and the

[protobuf] Re: Problem with java generation when using imports

2011-01-12 Thread Kenton Varda
kmobject_; public boolean hasKmobject() { return hasKmobject; } public KmObject.KmObject getKmobject() { return kmobject_; } I was expecting it to be KmObjectProtos.KmObject. On Jan 12, 2:26 am, Kenton Varda ken...@google.com wrote: Can you provide a small, self-contained example? On Mon

Re: [protobuf] Zig Zag Encoding (vs two's complement encoding)

2011-01-18 Thread Kenton Varda
The reason ZigZag encoding evolved as it did is because it can be applied as a separate layer on top of varint encoding. Originally, ZigZag was not officially part of the protobuf spec, but just something some people did manually to encode negative numbers efficiently. Eventually we made it

Re: [protobuf] best way to create message from prototype + data

2011-01-18 Thread Kenton Varda
Yep, that's the best thing to do right now. We're thinking of introducing a separate Parser interface, which would allow you to do: message = prototype.getParserForType().parse(data); On Tue, Jan 18, 2011 at 7:34 AM, koert koertkuip...@gmail.com wrote: A very common situation for me is when

[protobuf] Re: 2.4.0 release candidate #1

2011-01-18 Thread Kenton Varda
I tested the release on MSVC 2005 and FreeBSD. Both worked fine. I also tested this on my home machine running stock Ubuntu 10.10, and I got the same error we've been seeing on our corp workstations. Thus it seems this problem is not an issue with our corp setup as I had initially thought.

Re: [protobuf] Building Protobuffers under linux

2011-01-25 Thread Kenton Varda
I think if -lprotobuf wasn't found, the linker would complain about that first, rather than complain about undefined references -- unless you have some other libprotobuf in your library path which is being taken instead. Maybe an older version? On Wed, Jan 19, 2011 at 10:29 AM, Jalitt

Re: [protobuf] Problem with accents in python while unpacking a message

2011-01-25 Thread Kenton Varda
When using non-ASCII characters, make sure you are using the unicode type, not str, to avoid any ambiguity in character encoding. Also, I vaguely remember a bug fix related to this that is in version 2.4.0, so you might try that. On Thu, Jan 20, 2011 at 10:39 AM, Louhike louh...@gmail.com wrote:

Re: [protobuf] Re: protobuf not handling special characters between Java server and C++ client

2011-01-25 Thread Kenton Varda
On Tue, Jan 25, 2011 at 8:57 PM, Hitesh Jethwani hjethwa...@gmail.comwrote: if on the stream writer, I add something like: writer.write(new String(msg.getBytes(), UTF8).getBytes()) instead of simply writer.write(msg.getBytes()), I see the characters as expected on the C++ client. However

[protobuf] New protobuf feature proposal: Generated classes for streaming / visitors

2011-02-01 Thread Kenton Varda
Hello open source protobuf users, *Background* Probably the biggest deficiency in the open source protocol buffers libraries today is a lack of built-in support for handling streams of messages. True, it's not too hard for users to support it manually, by prefixing each message with its size as

Re: [protobuf] Repeated Fields Encoding

2011-02-01 Thread Kenton Varda
The encoding is documented in detail here: http://code.google.com/apis/protocolbuffers/docs/encoding.html http://code.google.com/apis/protocolbuffers/docs/encoding.htmlThe short answer is, yes, repeated fields are literally encoded as repeated individual values, unless you use packed encoding.

Re: [protobuf] Services not compiling

2011-02-01 Thread Kenton Varda
The *_generic_services options now default to false. You must explicitly set them to true, e.g.: option cc_generic_services = true; option java_generic_services = true; option py_generic_services = true; We decided to make these off-by-default because we want to encourage RPC

[protobuf] Re: New protobuf feature proposal: Generated classes for streaming / visitors

2011-02-01 Thread Kenton Varda
that wouldn't play well with ownership-passing. On Tue, Feb 1, 2011 at 10:45 AM, Kenton Varda ken...@google.com wrote: Hello open source protobuf users, *Background* Probably the biggest deficiency in the open source protocol buffers libraries today is a lack of built-in support for handling streams

Re: [protobuf] Re: inheritance.. well sort of... and FieldDescriptors

2011-02-01 Thread Kenton Varda
You can't use the same field descriptors for the four classes. But, note that one thing you *can* do is define a base type that just contains the shared fields, and then parse any of the other types as this type in order to access the common fields. Since the field numbers match, they are

Re: [protobuf] Re: New protobuf feature proposal: Generated classes for streaming / visitors

2011-02-04 Thread Kenton Varda
On Wed, Feb 2, 2011 at 10:13 AM, Henner Zeller henner.zel...@googlemail.com wrote: I guess the naming is confusing in the example. The Visit is per field-name; but since the typed is named the same as the field in this example, it is confusing. Yes, sorry. Better example: message

Re: [protobuf] 2.4.0a released.

2011-02-04 Thread Kenton Varda
On Wed, Feb 2, 2011 at 9:59 PM, David Yu david.yu@gmail.com wrote: Where the jars at? (Or do we have to build the jars from source?) Have to build from source. (Though we're going to try to get the Maven repo updated. Long story.) I'm trying to benchmark 2.4.0a with its java perf

Re: [protobuf] Re: New protobuf feature proposal: Generated classes for streaming / visitors

2011-02-05 Thread Kenton Varda
() visit_baz() then? It's good to have some consistency. Frank On Sat, Feb 5, 2011 at 08:06, Kenton Varda ken...@google.com wrote: On Wed, Feb 2, 2011 at 10:13 AM, Henner Zeller henner.zel...@googlemail.com wrote: I guess the naming is confusing in the example. The Visit is per field

Re: [protobuf] header update in building protobuf 2.4.0a on cygwin

2011-02-06 Thread Kenton Varda
What Cygwin version? What GCC version? We do test releases on Cygwin so I'm guessing you're using an older version. The error looks like a problem that old GCCs had (though we may want to work around it). On Sat, Feb 5, 2011 at 3:59 PM, jimo555 jimo...@gmail.com wrote: Just wanted to pass

Re: [protobuf] header update in building protobuf 2.4.0a on cygwin

2011-02-07 Thread Kenton Varda
shell scripts in the examples dir. It took me a while to figure this out because I don't usually use cygwin for java work. Thanks Jim Olson On Sun, Feb 6, 2011 at 3:33 PM, Kenton Varda ken...@google.com wrote: What Cygwin version? What GCC version? We do test releases on Cygwin so I'm

Re: [protobuf] Circular package import problem

2011-02-07 Thread Kenton Varda
With protoc (the standard compiler), this should produce a simple error. Since Ken said he got StackOverFlow exception, it sounds like he's using some other compiler (not written in C++). On Mon, Feb 7, 2011 at 10:26 AM, Pherl Liu liuj...@google.com wrote: You probably need to break up your

Re: [protobuf] New protobuf feature proposal: Generated classes for streaming / visitors

2011-02-08 Thread Kenton Varda
On Tue, Feb 8, 2011 at 11:23 AM, Evan Jones ev...@mit.edu wrote: Sorry, just an example of why you might want a different protocol. If I've streamed 10e9 messages to disk, I don't want this stream to break if there is some weird corruption in the middle, so I want some protocol that can

Re: [protobuf] compound message allows colon

2011-02-22 Thread Kenton Varda
It's a feature. This has always been allowed. No one really knows why. On Fri, Feb 18, 2011 at 9:53 AM, naderp paul.na...@gmail.com wrote: Hi, Given the following .proto definition: message person { required string name = 1; required int32 id = 2; } I noticed the following

Re: [protobuf] Suspected bug in java 2.4.0a Sub-builders

2011-02-22 Thread Kenton Varda
The first issue is the intended behavior. It matches C++, where calling mutable_inner() instantiates the inner message. There needs to be a way to mark a sub-message present without actually setting any of its friends. The second issue looks genuine, although it seems to me that it would cause

Re: [protobuf] RpcChannel and RpcController Implementation

2011-02-22 Thread Kenton Varda
For AppEngine you'll need to accept all incoming requests over HTTP. You probably don't want to use the RPC interfaces -- just use the regular servlet interface and use encoded protobufs in the request and response bodies. On Wed, Feb 16, 2011 at 10:39 PM, Amit amit.s...@gmail.com wrote: Hi

Re: [protobuf] Re: Why does ARMv5 thinks their are 2 implementations for these 4 functions in repeated_field.CC?

2011-02-22 Thread Kenton Varda
A lot of C++ compilers do not implement the (ridiculously complicated) standard correctly, especially when it comes to complex templates. If you can provide a patch which fixes your problem, we could consider adopting it, but otherwise the only advice I can give is: use GCC, Clang, or MSVC. On

Re: [protobuf] Re: .proto file with enumeration only

2011-02-22 Thread Kenton Varda
Unix and Windows EOLs should work, but old-Mac EOLs (which were just '\r's with no '\n') probably will not work. I thought that Mac stopped using that style with OSX, though, so I'm not sure why you'd be seeing this problem 10 years later. On Wed, Feb 16, 2011 at 11:16 AM, dan

Re: [protobuf] Typed Array Support

2011-02-22 Thread Kenton Varda
Well, first we'd need a Javascript protobuf implementation. There are a couple third-party ones, but Google doesn't provide one at this time. The usual way to deal with Javascript is to encode your message in JSON, e.g. using the code provided by http://code.google.com/p/protobuf-java-format/.

Re: [protobuf] Error when build universal binary on Mac OS Tiger 10.4 sdk

2011-02-22 Thread Kenton Varda
From the error message, it seems like GCC doesn't like being passed /Volumes/Data/XCode/SDKs/MacOSX10.4u.sdk in CXXFLAGS. Are you sure this is needed at all? If so, maybe this should be passed as LIBS or LDFLAGS instead? On Tue, Feb 22, 2011 at 3:50 AM, pick_tim nguyenquycu...@gmail.com wrote:

Re: [protobuf] Number of Bits Written to Memory

2011-02-22 Thread Kenton Varda
Protobufs provide their own encoding. You can't control exactly how the bytes are written, unless you write your own encoder, which would defeat the purpose. On Thu, Feb 3, 2011 at 8:58 AM, Tanya turtle...@gmail.com wrote: Hi, we are looking at replacing a XML/JAXB project with something

Re: [protobuf] Extending the compiler and programatically importing options

2011-02-22 Thread Kenton Varda
Nope, there's no way to do this, sorry. On Tue, Feb 15, 2011 at 4:49 AM, Arkaitz Jimenez arkai...@gmail.com wrote: Hi, I've my own compiler extension that adds some code over the standard code generated by cppgenerator. Now, I want to add a custom option, I know I can do that by adding the

Re: [protobuf] Feature request: ability to suppress logging of message fields

2011-02-22 Thread Kenton Varda
You can always write custom code that formats messages any way you want for your logs. The TextFormat class is based purely on public interfaces, so you can copy it and modify it. You can define a custom option for flagging fields that you want removed from the logs and use that custom option in

Re: [protobuf] Creating a FileDescriptorSet of all .proto files used in a certain executable

2011-02-22 Thread Kenton Varda
As you note, you can't iterate over all compiled-in protos. However, if you can collect a set of root protos that you care about, it's easy to traverse down the tree to collect all their dependencies. In general, trying to enumerate everything that is linked in tends to lead to trouble. The

Re: [protobuf] Linker Errors and stdcall vs cdecl calling conventions (VC++ 2008)

2011-03-24 Thread Kenton Varda
We never actually chose to use cdecl; it was simply the default used by MSVC. This whole idea that the programmer should have to choose between calling conventions is a bit strange and inconvenient. The LIBPROTOBUF_EXPORT annotations are annoying enough; I hope we don't have to litter our code

Re: [protobuf] Re: New protobuf feature proposal: Generated classes for streaming / visitors

2011-04-03 Thread Kenton Varda
of the same problems. thanks -jeff On Feb 8, 2:34 pm, Kenton Varda ken...@google.com wrote: On Tue, Feb 8, 2011 at 5:47 AM, Evan Jones ev...@mit.edu wrote: The Visitor class has two standard implementations: Writer and Filler. MyStream::Writer writes the visited fields

Re: [protobuf] undefined reference to GzipInputStream

2011-05-18 Thread Kenton Varda
GzipInputStream is only compiled into the library if zlib is available, since it depends on zlib. On Unix you should be able to fix this by installing libz.so, on MSVC you may need to modify the project files a bit. On Wed, May 18, 2011 at 12:18 AM, Shin for.shin1...@gmail.com wrote: I'm

Re: [protobuf] Licensing, again

2011-06-21 Thread Kenton Varda
I'm not sure I understand the question. As the FAQ says, we hold no patents on this code. What is your concern, exactly? If we dual-licensed the code under BSD and Apache, would that help? On Tue, Jun 21, 2011 at 2:32 AM, John Haine j.l.hain...@googlemail.comwrote: First, apologies if this

Re: [protobuf] Re: Licensing, again

2011-06-22 Thread Kenton Varda
that you move to a dual-licence regime but perhaps the FAQ could be amended if indeed it is the case that there are no patents, pending or issued, that cover PB? Thanks, and apologies again for being pedantic! John. On Jun 21, 6:21 pm, Kenton Varda ken...@google.com wrote: I'm not sure I

<    6   7   8   9   10   11