[capnproto] Random access clarified

[Johannes Dröge]

Hi there!

The FAQ states *"*Random access*:* You can read just one field of a message 
without parsing the whole thing". *However, does that also apply to List 
indexing*? I have a flexible-length list of potentially large objects, and 
I need to access the nth list element from disk without having to hold 
other elements in memory.

I started using capnp for internal serialization in a prototype, with a 
more dynamic approach to data types and data structures. For this, I'm 
mostly attracted by the fast implementation and random access option, which 
gives me the possibility to mmap data structures to lazy-load attributes 
from disk. I'm currently using the Python interface but I might switch to 
C++, Rust or go at a later stage.

I will try to profile this with a toy example. Nevertheless, I'd be 
thankful for a theoretical consideration here!

Cheers
Johannes

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/capnproto/48035174-9253-48d3-ad3d-b3fe69d249a3n%40googlegroups.com.

Re: [capnproto] Random access clarified

['Kenton Varda' via Cap'n Proto]

Hi Johannes,

Yes, it applies to list indexing.

-Kenton

On Mon, Sep 4, 2023 at 10:43 AM Johannes Dröge  wrote:

> Hi there!
>
> The FAQ states *"*Random access*:* You can read just one field of a
> message without parsing the whole thing". *However, does that also apply
> to List indexing*? I have a flexible-length list of potentially large
> objects, and I need to access the nth list element from disk without having
> to hold other elements in memory.
>
> I started using capnp for internal serialization in a prototype, with a
> more dynamic approach to data types and data structures. For this, I'm
> mostly attracted by the fast implementation and random access option, which
> gives me the possibility to mmap data structures to lazy-load attributes
> from disk. I'm currently using the Python interface but I might switch to
> C++, Rust or go at a later stage.
>
> I will try to profile this with a toy example. Nevertheless, I'd be
> thankful for a theoretical consideration here!
>
> Cheers
> Johannes
>
> --
> You received this message because you are subscribed to the Google Groups
> "Cap'n Proto" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to capnproto+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/capnproto/48035174-9253-48d3-ad3d-b3fe69d249a3n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/capnproto/CAJouXQnJ%2BvDy64wBw-i9o0HaVapznMz2--CoooX5XQ0nhASEKA%40mail.gmail.com.