Re: [capnproto] [Feature Request] Putting List() object at last and allow reuse of the MallocMessageBuilder

2023-06-05 Thread 'Kenton Varda' via Cap'n Proto
Cross-segment references are in fact expressed as a pair of a segment number and an offset today. So with sufficiently complex allocation logic, you could indeed allocate the side objects in a separate segment from the list itself, and thus allow the list to be resized continuously. But the C++

Re: [capnproto] [Feature Request] Putting List() object at last and allow reuse of the MallocMessageBuilder

2023-06-05 Thread Jonathan Shapiro
Chiming in here in relative ignorance… Offhand, it seems to me that the advantage to placing strings last is that you can buffer them in another segment making a single pass over the struct array and then appending them without a second touch. It plays fairly nicely with the cache, and writev()

Re: [capnproto] [Feature Request] Putting List() object at last and allow reuse of the MallocMessageBuilder

2023-06-05 Thread 'Kenton Varda' via Cap'n Proto
Hi Gokul, There's currently no way to "inline" one struct into another. The fundamental problem you have here is this: Say you have a list containing pointer values, and the list is actually at the end of the message. So, you can resize it without fragmentation. You add one new element. Then you

Re: [capnproto] [Feature Request] Putting List() object at last and allow reuse of the MallocMessageBuilder

2023-05-31 Thread Gokul Rajiv
Hi Kenton, As a follow up to the case of structs containing pointer/non-primitive types, might there currently exist workarounds to being able to resize lists of these structs (like the one below) at the back of a message: perhaps removing the allocated non-primitive data or embedding them

Re: [capnproto] [Feature Request] Putting List() object at last and allow reuse of the MallocMessageBuilder

2022-12-12 Thread Hui min
Hi Kenton, Got it, it makes sense. So if I remove `labelString` data from the definition, I could use truncate function to operate my detections @4 :List(Detection2d); object. Thanks! On Tuesday, 6 December 2022 at 10:43:17 pm UTC+8 ken...@cloudflare.com wrote: > Hi Hui, > > Again sorry for

[capnproto] [Feature Request] Putting List() object at last and allow reuse of the MallocMessageBuilder

2022-11-05 Thread Hui min
Hi Cap'n Proto Team, Thanks for the amazing tool you have created so far. Understand that with the current design of arena style memory allocation, we cannot simply reuse the message, and re-init List() object to send it again. This will cause the message to grow every time we send (memory