[capnproto] Can I do better?

2022-02-25 Thread Jitesh Khandelwal
 

I am trying to implement a middle layer which serialises some data from a 
data structure into a buffer, which is owned by and whose contents are 
written on the wire by an outer layer.

Currently, my code looks as follows

constexpr size_t N = 2048;
char arena_buffer[N]{0};
auto arena = arrayPtr(reinterpret_cast(arena_buffer), N / 
sizeof(word));

Encode (some_data_structure, arena, buffer)
{
auto builder = MallocMessageBuilder(arena);

// Step 1. set fields from some_data_structure using builder into arena


// Step 2. convert to flat array (this allocates another array)
auto message = messageToFlatArray(builder);
auto bytes   = message.asBytes();

// Step 3. copy to destination
memcpy(buffer, bytes.begin(), bytes.size());
}


My question is, Can I do better?

My understanding is that the extra allocation in Step 2, is because of the 
segment framing protocol ?

Can I bypass it somehow, as I am sure that the arena is big enough for my 
message and there'll be only 1 segment ?

And if yes, then may be I can use the buffer as the arena, as in build 
directly into the output buffer?



Thanks,
Jitesh

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/capnproto/a6378e3d-fd2e-4009-8cf4-595011170dc9n%40googlegroups.com.


Re: [capnproto] Can I do better?

2022-02-25 Thread 'Kenton Varda' via Cap'n Proto
Hi Jitesh,

Yes, you can avoid the allocation as follows:

1. Construct a kj::ArrayOutputStream that uses your destination buffer as
the output.
2. Use capnp::writeMessage() to write the message to that stream.

-Kenton

On Fri, Feb 25, 2022 at 12:31 PM Jitesh Khandelwal 
wrote:

> I am trying to implement a middle layer which serialises some data from a
> data structure into a buffer, which is owned by and whose contents are
> written on the wire by an outer layer.
>
> Currently, my code looks as follows
>
> 
> constexpr size_t N = 2048;
> char arena_buffer[N]{0};
> auto arena = arrayPtr(reinterpret_cast(arena_buffer), N /
> sizeof(word));
>
> Encode (some_data_structure, arena, buffer)
> {
> auto builder = MallocMessageBuilder(arena);
> 
> // Step 1. set fields from some_data_structure using builder into arena
> 
>
> // Step 2. convert to flat array (this allocates another array)
> auto message = messageToFlatArray(builder);
> auto bytes   = message.asBytes();
>
> // Step 3. copy to destination
> memcpy(buffer, bytes.begin(), bytes.size());
> }
>
> 
>
> My question is, Can I do better?
>
> My understanding is that the extra allocation in Step 2, is because of the
> segment framing protocol ?
>
> Can I bypass it somehow, as I am sure that the arena is big enough for my
> message and there'll be only 1 segment ?
>
> And if yes, then may be I can use the buffer as the arena, as in build
> directly into the output buffer?
>
>
>
> Thanks,
> Jitesh
>
> --
> You received this message because you are subscribed to the Google Groups
> "Cap'n Proto" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to capnproto+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/capnproto/a6378e3d-fd2e-4009-8cf4-595011170dc9n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/capnproto/CAJouXQkanKYk%3D1dgdB%2BkDFc7xR%3DQ4o5d%3DoW1o4kBk7RaBu5__g%40mail.gmail.com.