Re: [protobuf] Re: Java UTF-8 encoding/decoding: possible performance improvements

2010-05-17 Thread Evan Jones
need to test it to be sure. This is sort of just a fun experiment for me at this point, so who knows when I may get around to actually finishing this. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group

Re: [protobuf] Re: Java UTF-8 encoding/decoding: possible performance improvements

2010-05-07 Thread Evan Jones
it doesn't overallocate quite as much (in exchange my code does many copies). Conclusion: there is a legitimate reason for this code to be faster than the JDK's code. But it still may not be worth including this patch in the main line protocol buffer code. Evan -- Evan Jones http

Re: [protobuf] Re: Java UTF-8 encoding/decoding: possible performance improvements

2010-05-03 Thread Evan Jones
On May 3, 2010, at 21:11 , Evan Jones wrote: Yes, I actually changed ByteString, since ByteString.copyFromUtf8 is how protocol buffers get UTF-8 encoded strings at this point. Although now that I think about it, I think it might be possible to enable this only for SPEED messages

[protobuf] Re: Java UTF-8 encoding/decoding: possible performance improvements

2010-04-28 Thread Evan Jones
Evan Jones wrote: Problem 2: Using the NIO encoders/decoders can be faster than String.getBytes, but only if it is used = 4 times. If used only once, it is worse. The same is approximately true about decoding. Lame results: http://evanjones.ca/software/java-string-encoding.html I'm

Re: [protobuf] serialization size from 2.0.x to 2.3.x, also message design best practices

2010-04-27 Thread Evan Jones
buffers and strings. I'll try to spend some time revisiting this at some point soon. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com

[protobuf] valgrind possibly lost warnings?

2010-03-31 Thread Evan Jones
/protorpc/protorpcchannel_test) ==17815==by 0x804A8EF: (within /home/evanj/hstore/hstore-async/build/protorpc/protorpcchannel_test) ==17815==by 0x80C3C08: __libc_csu_init (in /home/evanj/hstore/hstore-async/build/protorpc/protorpcchannel_test) -- Evan Jones http://evanjones.ca/ -- You

Re: [protobuf] valgrind possibly lost warnings?

2010-03-31 Thread Evan Jones
valgrind. I'm assuming that won't cause any weird failures, but if there might be some problem that I haven't considered, please let me know. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post

Re: [protobuf] Best method for adding to messages..

2010-03-22 Thread Evan Jones
On Mar 22, 2010, at 14:00 , Olan wrote: const DIDS::Station station_iter = stationlog.station(k); You probably want stationlog.mutable_station(k); See: http://code.google.com/apis/protocolbuffers/docs/reference/cpp-generated.html#fields Hope this helps, Evan -- Evan Jones http

Re: [protobuf] Problem : Serialized in protobuf-net, deserialize in C++ app

2010-03-11 Thread Evan Jones
to figure out where this is happening. My unhelpful guess is that something is not initialized correctly. Good luck, Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email

Re: [protobuf] Variant-length encoding : is it OK to be wasteful?

2010-03-10 Thread Evan Jones
protocol buffer in the same way, but it seems *possible*? Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send

Re: [protobuf] Variant-length encoding : is it OK to be wasteful?

2010-03-10 Thread Evan Jones
Kenton Varda wrote: With Protocol Buffers, we already say that fields can appear in any order, so any security mechanism built around matching the raw bytes of a protobuf message against some blacklist is obviously flawed. Ah, I was not aware of that. Good to know. Evan -- Evan Jones http

Re: [protobuf] Re: Any sample messages/data for optimization out there?

2010-03-09 Thread Evan Jones
of a mergeDelimitedFrom(CodedInputStream) would be welcomed. You can do this yourself. I think the following would work: int length = inputStream.readRawVarint32(); int oldLimit = inputStream.pushLimit(length); builder.mergeFrom(inputStream); inputStream.popLimit(oldLimit); Evan -- Evan Jones http

Re: [protobuf] Any sample messages/data for optimization out there?

2010-03-07 Thread Evan Jones
have, but I fear that's not going to be very representative as they're mostly simple, flat messages where the bulk of the data is inside an opaque bytes field. If it is representative for *your* application that is probably the most important part. Evan -- Evan Jones http://evanjones.ca

Re: [protobuf] Re: Placing byte buffer into string

2010-02-14 Thread Evan Jones
On Feb 14, 2010, at 17:23 , SyRenity wrote: Actually, answering myself it seems to be as simple as: sData.assign(buffer, size); //Where size is the full buffer size Yes, that works. See the following for more help: http://www.sgi.com/tech/stl/basic_string.html Evan -- Evan Jones http

Re: [protobuf] Re: Dynamically determine the type of a message

2010-02-11 Thread Evan Jones
, unless I'm thinking about this in the wrong way? I haven't actually thought about this very hard. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto

Re: [protobuf] Dynamically determine the type of a message

2010-02-10 Thread Evan Jones
want. See the techniques section on Union Types: http://code.google.com/apis/protocolbuffers/docs/techniques.html#union Hope this helps, Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post

Re: [protobuf] How can I use protoBuf with a socket

2010-02-06 Thread Evan Jones
, std::string assumes buf is a NULL terminated C string. You should use the following to create the string with a specific length: string another_copy(buf, msgStr.size()); Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups

Re: [protobuf] protobuf usage for socket communication

2010-02-06 Thread Evan Jones
/techniques.html#union If you have many possible types, you may wish to consider implementing a more complicated protocol where you send a header with the message type string before the message. Hope this helps, Evan -- Evan Jones http://evanjones.ca/ -- You received this message because

Re: [protobuf] Using protobuf in cpp program

2010-02-04 Thread Evan Jones
need to compile without RTTI, simply #define GOOGLE_PROTOBUF_NO_RTTI. // On MSVC, this should be detected automatically. So add -DGOOGLE_PROTOBUF_NO_RTTI and it should work? Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups

Re: [protobuf] Bi-Directional protocol buffers

2010-01-25 Thread Evan Jones
buffer messages in both directions. In fact, if you want to write an RPC implementation, you'll have to do exactly that: send request messages one way, and reply messages the other. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google

Re: [protobuf] Is there a way to pass C string without triggering construction of string object in C++?

2010-01-07 Thread Evan Jones
= protobuffer.name().data() this gets a pointer to the raw C string inside the C++ string object inside the protocol buffer object. No allocations or copies will be performed. Setting the string, one copy is always performed, as Kenton mentioned. Evan -- Evan Jones http://evanjones.ca/ -- You

Re: [protobuf] Re: protobuf rpc over http

2009-12-10 Thread Evan Jones
? Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to proto...@googlegroups.com. To unsubscribe from this group, send email to protobuf+unsubscr...@googlegroups.com. For more

Re: [protobuf] Re: Receiving/Parsing Messages

2009-12-07 Thread Evan Jones
On Dec 7, 2009, at 0:02 , lexer wrote: Adam, Could you please show simple example how to use tag number for dispatch purpose. I believe he was referring to this sort of thing: http://code.google.com/apis/protocolbuffers/docs/techniques.html#union Hope this helps, Evan -- Evan Jones http

Re: [protobuf] Re: Protocol Buffers and asynchronous sockets

2009-11-24 Thread Evan Jones
for each message. The reason to use something more complicated is because lots of applications already have some sort of buffer, and you want to try and avoid extra copies. Evan -- Evan Jones http://evanjones.ca/ -- You received this message because you are subscribed to the Google Groups

[protobuf] Re: Retrieving data with field number and wire type

2009-11-10 Thread Evan Jones
at CodedInputStream C++: http://code.google.com/apis/protocolbuffers/docs/reference/cpp/google.protobuf.io.coded_stream.html Java: http://code.google.com/apis/protocolbuffers/docs/reference/java/index.html?com/google/protobuf/CodedInputStream.html -- Evan Jones http://evanjones.ca

[protobuf] Re: [Debian] Compilation problem

2009-11-10 Thread Evan Jones
/~beechung/ref/gcc-intro.html Evan -- Evan Jones http://evanjones.ca/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe

[protobuf] Re: documentation within proto files

2009-11-02 Thread Evan Jones
volunteering to do the work though. :) Evan -- Evan Jones http://evanjones.ca/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com

[protobuf] Re: Java and generic conversion

2009-10-31 Thread Evan Jones
a mapping from message names to Message prototypes, so you can create a Message of an appropriate type. Hope this helps, Evan -- Evan Jones http://evanjones.ca/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups

[protobuf] Java CodedOutputStream performance: buffer size and OutputStream

2009-10-22 Thread Evan Jones
, and are of course data dependent. More benchmarks should be done before making a performance related code change. Evan -- Evan Jones http://evanjones.ca/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers

[protobuf] Re: Java CodedOutputStream performance: buffer size and OutputStream

2009-10-22 Thread Evan Jones
in that context. Evan -- Evan Jones http://evanjones.ca/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Protocol Buffers group. To post to this group, send email to protobuf@googlegroups.com To unsubscribe from

Re: RpcController.setFailed: Need to call the callback?

2009-10-19 Thread Evan Jones
() or callback.fail() would probably be clearer and less error prone (eg. calling setFailed() and forgetting to call callback.run()). I would definitely accept a breaking API change like this, but I can understand that others who are using the RPC interface may not agree with me. Evan -- Evan

<    1   2