Hi; I'd love to help you on this - I'm the protobuf-net author, and I also know more than a little about redis; it *might* be a little off-piste for this group though. Running some tests locally with 5000 instances, I get times like 1ms, but it might be that I'm misunderstanding your object model. It would be great to perhaps see a realistic payload that I can play with to see what's going on here.
Question: how fast is "fast enough" here, and how big are your payloads? I have some incomplete changes in the pipe which at a stroke double the throughput, but without knowing the answers to these questions, it is hard to know whether that is sufficient Depending on your target scenario, if the problem here turns out to be allocations (of either the SubObj or the array), I have recently been doing a lot of work in "arena allocators" which might be directly relevant - I have a future plan to allow protobuf-net to consume / use the arena allocator I've been working on, which would eliminate both of those completely. Example output from my test (the first serialize/deserialize is always more expensive) - which is around 68k in payload size: Serialize: 47ms Serialize: 1ms Serialize: 1ms Serialize: 1ms Serialize: 1ms Deserialize: 6ms Deserialize: 1ms Deserialize: 1ms Deserialize: 1ms Deserialize: 1ms My test code: https://gist.github.com/mgravell/7afc08f432661f60138f6798efe2b15b If you want to email me directly (especially if your data contains "real" things that you don't want to send to the entire group), please feel free to do so. On Tue, 26 Mar 2019 at 18:32, Shweta Sharma <shwetad...@gmail.com> wrote: > I am prototyping an application to store data in redis using > StackExhange.Redis client. I am using Protobuf for serializing my objects > and storing them as key/value pair in redis. When reading 20K keys using > pipeline (splitting keys in batches of 5000), I observe deserialization > times in 60-300 ms for a batch of say 5000 objects. Serialization code is > using protobuf-net library version. Deserialization code is using > protobuf-net 2.4.0.0. > > Is there anyway I can speed up desrialization time since this will impact > my api performance. > > The object I am trying to desrialize is not too complex. > > [ProtoContract] > public struct MyObject > { > [ProtoMember(1)] > public int Key { get; set; } > [ProtoMember(2)] > public byte Attr1 { get; set; } > [ProtoMember(3)] > public bool IsUsed { get; set; } > [ProtoMember(4)] > public byte Attr2 { get; set; } > [ProtoMember(5)] > public SubObj[] SubObjects { get; set; } > > public static MyObject Null > { > get > { > return default(MyObject); > } > } > > public bool IsNull > { > get { return Attr1 == default(byte) && SubObjects == null; } > } > } > [ProtoContract] > public class SubObj > { > [ProtoMember(1)] > public int StartOffset { get; set; } > [ProtoMember(2)] > public int EndOffset { get; set; } > [ProtoMember(3)] > public byte Attr1 { get; set; } > [ProtoMember(4)] > public byte Attr2 { get; set; } > [ProtoMember(5)] > public bool IsUsed { get; set; } > > public static SubObj Null > { > get > { > return default(SubObj); > } > } > > public bool IsNull > { > get { return Attr1 == default(byte); } > } > } > > Here're some sample timings using StopWatch: > > Time to desrialize 5000 keys from redis:134 > Time to desrialize 5000 keys from redis:147 > Time to desrialize 5000 keys from redis:160 > Time to desrialize 5000 keys from redis:65 > Time to desrialize 5000 keys from redis:67 > Time to desrialize 5000 keys from redis:242 > Time to desrialize 5000 keys from redis:371 > Time to desrialize 5000 keys from redis:190 > > Thanks, > Shweta > > -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to protobuf+unsubscr...@googlegroups.com. > To post to this group, send email to protobuf@googlegroups.com. > Visit this group at https://groups.google.com/group/protobuf. > For more options, visit https://groups.google.com/d/optout. > -- Regards, Marc -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to protobuf+unsubscr...@googlegroups.com. To post to this group, send email to protobuf@googlegroups.com. Visit this group at https://groups.google.com/group/protobuf. For more options, visit https://groups.google.com/d/optout.
