Re: [capnproto] Can I do better?

[Jitesh Khandelwal]

Thanks Kenton, this is working well. Now the allocation in 
messageToFlatArray is not happening.

Now, is it possible to build the message in the buffer directly ? 
Currently, I'm building in arena and then calling writeMessage to write 
into the buffer.
--
auto builder = MallocMessageBuilder(*arena*);
...
...
auto message = arrayPtr(reinterpret_cast(*buffer*), N);  

   
auto aos = kj::ArrayOutputStream(message);  

 
capnp::writeMessage(aos, builder);
--
On Saturday, 26 February 2022 at 07:07:44 UTC+5:30 ken...@cloudflare.com 
wrote:

> Hi Jitesh,
>
> Yes, you can avoid the allocation as follows:
>
> 1. Construct a kj::ArrayOutputStream that uses your destination buffer as 
> the output.
> 2. Use capnp::writeMessage() to write the message to that stream.
>
> -Kenton
>
> On Fri, Feb 25, 2022 at 12:31 PM Jitesh Khandelwal  
> wrote:
>
>> I am trying to implement a middle layer which serialises some data from a 
>> data structure into a buffer, which is owned by and whose contents are 
>> written on the wire by an outer layer.
>>
>> Currently, my code looks as follows
>>
>> 
>> constexpr size_t N = 2048;
>> char arena_buffer[N]{0};
>> auto arena = arrayPtr(reinterpret_cast(arena_buffer), N / 
>> sizeof(word));
>>
>> Encode (some_data_structure, arena, buffer)
>> {
>> auto builder = MallocMessageBuilder(arena);
>> 
>> // Step 1. set fields from some_data_structure using builder into 
>> arena
>> 
>>
>> // Step 2. convert to flat array (this allocates another array)
>> auto message = messageToFlatArray(builder);
>> auto bytes   = message.asBytes();
>>
>> // Step 3. copy to destination
>> memcpy(buffer, bytes.begin(), bytes.size());
>> }
>>
>> 
>>
>> My question is, Can I do better?
>>
>> My understanding is that the extra allocation in Step 2, is because of 
>> the segment framing protocol ?
>>
>> Can I bypass it somehow, as I am sure that the arena is big enough for my 
>> message and there'll be only 1 segment ?
>>
>> And if yes, then may be I can use the buffer as the arena, as in build 
>> directly into the output buffer?
>>
>>
>>
>> Thanks,
>> Jitesh
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Cap'n Proto" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to capnproto+...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/capnproto/a6378e3d-fd2e-4009-8cf4-595011170dc9n%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/capnproto/a6378e3d-fd2e-4009-8cf4-595011170dc9n%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/capnproto/b09290d5-e2d3-4711-b309-9bcdaaade8e5n%40googlegroups.com.

[capnproto] Can I do better?

[Jitesh Khandelwal]

 

I am trying to implement a middle layer which serialises some data from a 
data structure into a buffer, which is owned by and whose contents are 
written on the wire by an outer layer.

Currently, my code looks as follows

constexpr size_t N = 2048;
char arena_buffer[N]{0};
auto arena = arrayPtr(reinterpret_cast(arena_buffer), N / 
sizeof(word));

Encode (some_data_structure, arena, buffer)
{
auto builder = MallocMessageBuilder(arena);

// Step 1. set fields from some_data_structure using builder into arena


// Step 2. convert to flat array (this allocates another array)
auto message = messageToFlatArray(builder);
auto bytes   = message.asBytes();

// Step 3. copy to destination
memcpy(buffer, bytes.begin(), bytes.size());
}


My question is, Can I do better?

My understanding is that the extra allocation in Step 2, is because of the 
segment framing protocol ?

Can I bypass it somehow, as I am sure that the arena is big enough for my 
message and there'll be only 1 segment ?

And if yes, then may be I can use the buffer as the arena, as in build 
directly into the output buffer?



Thanks,
Jitesh

-- 
You received this message because you are subscribed to the Google Groups 
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to capnproto+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/capnproto/a6378e3d-fd2e-4009-8cf4-595011170dc9n%40googlegroups.com.