[capnproto] Detect closed connection using StreamFdMessageReader?
Hi, I use capnp::StreamFdMessageReader(fd) to read a message on a file descriptor created using network sockets. And it works fine. However, I'm trying to detect if the client socket has closed. Normally, if a read() call returns 0, this would show the client connection has closed. How would I go about this by using StreamFdMessageReader? -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/c5b44e93-dde3-41ec-bced-7d839fde6185n%40googlegroups.com.
Re: [capnproto] compiling with automake
Hi Kenton, I don't have an issue with setting up and building libraries and linking my own programs to external libs like capnproto using Autotools. My problem is that, I have a test.capnp file and I don't know how to call/run the capnp tool via a Makefile.am to produce the *.capnp.h and *.capnp.c++ files. Or did I miss your point in understanding that this would happen by nesting packages? On Thursday, October 21, 2021 at 1:02:05 PM UTC-5 ken...@cloudflare.com wrote: > It should work as a subpackage. See: > > https://www.gnu.org/software/automake/manual/html_node/Subpackages.html > > -Kenton > > On Thu, Oct 21, 2021 at 12:45 PM SR D wrote: > >> I would like to make use of capnproto in an autotools project and >> wondering if anyone has any references or examples as to how to do basic >> compiling with a Makefile.am. Especially w.r.t. the cmake equivalent of >> capnp_generate_cpp(). >> >> Any help appreciated. >> >> >> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Cap'n Proto" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to capnproto+...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/capnproto/1fc1a675-2ed5-41fc-8dd2-d81e23517cfbn%40googlegroups.com >> >> <https://groups.google.com/d/msgid/capnproto/1fc1a675-2ed5-41fc-8dd2-d81e23517cfbn%40googlegroups.com?utm_medium=email_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/038394b5-d57d-4997-b77b-63f8ef19bcfcn%40googlegroups.com.
[capnproto] compiling with automake
I would like to make use of capnproto in an autotools project and wondering if anyone has any references or examples as to how to do basic compiling with a Makefile.am. Especially w.r.t. the cmake equivalent of capnp_generate_cpp(). Any help appreciated. -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/1fc1a675-2ed5-41fc-8dd2-d81e23517cfbn%40googlegroups.com.
Re: [capnproto] Selecting the correct message type?
Maybe I'll look into carrying the byte buffer around then.
On Monday, July 19, 2021 at 5:16:03 PM UTC-5 SR D wrote:
> Ok, thanks for clearing that up for me.
>
> On Monday, July 19, 2021 at 4:56:51 PM UTC-5 ken...@cloudflare.com wrote:
>
>> Sorry, I'm not sure what you mean. `myns::Person` isn't a real type, it's
>> just an empty class that acts as a namespace containing the `Reader` and
>> `Builder` types. `Reader` is your "handle to the data", and its getters are
>> the only way to read the underlying data (the getters read directly from
>> the raw underlying byte buffer).
>>
>> -Kenton
>>
>> On Mon, Jul 19, 2021 at 3:45 PM SR D wrote:
>>
>>> Ok, so after some thinking, I can put something together to communicate
>>> the type. Let's say I now know the type upon receiving the data, I make
>>> this call ...
>>>
>>> auto m = reader.getRoot();
>>>
>>> Here, m is of type myns::Person::Reader. So although I know the type, I
>>> looked at the person.capnp.h file and found no way to get a handle to the
>>> actual data I care about, the myns::Person object, rather just the getters.
>>>
>>> I understand I can rebuild the object by calling the individual fields
>>> like m.getXXX()'s, but I'm trying to do this in a generic way and not have
>>> to call the individual fields to get a myns::Person and pass the object to
>>> another thread for processing. So, my goal is to not have to call any proto
>>> object methods (getters or setters) and hope to call something from the
>>> Reader to get some sort of reference to the object since I know it's type.
>>>
>>> Is that possible?
>>>
>>>
>>> On Saturday, July 17, 2021 at 12:22:10 PM UTC-5 ken...@cloudflare.com
>>> wrote:
>>>
>>>> There is no way to find out a message's type if you don't know it
>>>> already. So, you'll need to arrange to communicate the type yourself. The
>>>> easiest way is to write an "envelope" message that is a union of all your
>>>> other messages.
>>>>
>>>> struct OuterMessage {
>>>> union {
>>>> foo @0 :FooMessage;
>>>> bar @1 :BarMessage;
>>>> baz @2 :BazMessage;
>>>> }
>>>> }
>>>>
>>>> Then, always send OuterMessage as the top-level message type.
>>>>
>>>> -Kenton
>>>>
>>>> On Sat, Jul 17, 2021 at 11:55 AM SR D
>>>> wrote:
>>>>
>>>>> If the sender is sending any number of different data types that have
>>>>> been serialized across the network, how does one know which type is
>>>>> coming
>>>>> in so that it can be pulled off using FlatArrayMessageReader's::getRoot<>?
>>>>>
>>>>> E.g., say this while loop is constantly reading serialized data from
>>>>> the network and results in a vector of message buffers.
>>>>>
>>>>> for (auto& msg : messages) {
>>>>> auto arr = kj::ArrayPtr(
>>>>> reinterpret_cast(msg),
>>>>> size / sizeof(capnp::word));
>>>>>
>>>>> capnp::FlatArrayMessageReader reader(arr);
>>>>>
>>>>> // how would one know what type to use here?
>>>>> auto msg = reader.getRoot();
>>>>> }
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "Cap'n Proto" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to capnproto+...@googlegroups.com.
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/capnproto/c89926fe-3b18-4e0f-9d30-8f8134b78a0an%40googlegroups.com
>>>>>
>>>>> <https://groups.google.com/d/msgid/capnproto/c89926fe-3b18-4e0f-9d30-8f8134b78a0an%40googlegroups.com?utm_medium=email_source=footer>
>>>>> .
>>>>>
>>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Cap'n Proto" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to capnproto+...@googlegroups.com.
>>>
>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/capnproto/9545bbfc-38a3-44d4-bf5d-2f059e32dfa3n%40googlegroups.com
>>>
>>> <https://groups.google.com/d/msgid/capnproto/9545bbfc-38a3-44d4-bf5d-2f059e32dfa3n%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to capnproto+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/capnproto/a4076811-361c-4f78-895e-cc51255595a0n%40googlegroups.com.
Re: [capnproto] Selecting the correct message type?
Ok, thanks for clearing that up for me.
On Monday, July 19, 2021 at 4:56:51 PM UTC-5 ken...@cloudflare.com wrote:
> Sorry, I'm not sure what you mean. `myns::Person` isn't a real type, it's
> just an empty class that acts as a namespace containing the `Reader` and
> `Builder` types. `Reader` is your "handle to the data", and its getters are
> the only way to read the underlying data (the getters read directly from
> the raw underlying byte buffer).
>
> -Kenton
>
> On Mon, Jul 19, 2021 at 3:45 PM SR D wrote:
>
>> Ok, so after some thinking, I can put something together to communicate
>> the type. Let's say I now know the type upon receiving the data, I make
>> this call ...
>>
>> auto m = reader.getRoot();
>>
>> Here, m is of type myns::Person::Reader. So although I know the type, I
>> looked at the person.capnp.h file and found no way to get a handle to the
>> actual data I care about, the myns::Person object, rather just the getters.
>>
>> I understand I can rebuild the object by calling the individual fields
>> like m.getXXX()'s, but I'm trying to do this in a generic way and not have
>> to call the individual fields to get a myns::Person and pass the object to
>> another thread for processing. So, my goal is to not have to call any proto
>> object methods (getters or setters) and hope to call something from the
>> Reader to get some sort of reference to the object since I know it's type.
>>
>> Is that possible?
>>
>>
>> On Saturday, July 17, 2021 at 12:22:10 PM UTC-5 ken...@cloudflare.com
>> wrote:
>>
>>> There is no way to find out a message's type if you don't know it
>>> already. So, you'll need to arrange to communicate the type yourself. The
>>> easiest way is to write an "envelope" message that is a union of all your
>>> other messages.
>>>
>>> struct OuterMessage {
>>> union {
>>> foo @0 :FooMessage;
>>> bar @1 :BarMessage;
>>> baz @2 :BazMessage;
>>> }
>>> }
>>>
>>> Then, always send OuterMessage as the top-level message type.
>>>
>>> -Kenton
>>>
>>> On Sat, Jul 17, 2021 at 11:55 AM SR D
>>> wrote:
>>>
>>>> If the sender is sending any number of different data types that have
>>>> been serialized across the network, how does one know which type is coming
>>>> in so that it can be pulled off using FlatArrayMessageReader's::getRoot<>?
>>>>
>>>> E.g., say this while loop is constantly reading serialized data from
>>>> the network and results in a vector of message buffers.
>>>>
>>>> for (auto& msg : messages) {
>>>> auto arr = kj::ArrayPtr(
>>>> reinterpret_cast(msg),
>>>> size / sizeof(capnp::word));
>>>>
>>>> capnp::FlatArrayMessageReader reader(arr);
>>>>
>>>> // how would one know what type to use here?
>>>> auto msg = reader.getRoot();
>>>> }
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "Cap'n Proto" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to capnproto+...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/capnproto/c89926fe-3b18-4e0f-9d30-8f8134b78a0an%40googlegroups.com
>>>>
>>>> <https://groups.google.com/d/msgid/capnproto/c89926fe-3b18-4e0f-9d30-8f8134b78a0an%40googlegroups.com?utm_medium=email_source=footer>
>>>> .
>>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "Cap'n Proto" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to capnproto+...@googlegroups.com.
>>
> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/capnproto/9545bbfc-38a3-44d4-bf5d-2f059e32dfa3n%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/capnproto/9545bbfc-38a3-44d4-bf5d-2f059e32dfa3n%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to capnproto+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/capnproto/50987430-84fd-4c12-9c7e-fed0e673bba9n%40googlegroups.com.
Re: [capnproto] Selecting the correct message type?
Ok, so after some thinking, I can put something together to communicate the
type. Let's say I now know the type upon receiving the data, I make this
call ...
auto m = reader.getRoot();
Here, m is of type myns::Person::Reader. So although I know the type, I
looked at the person.capnp.h file and found no way to get a handle to the
actual data I care about, the myns::Person object, rather just the getters.
I understand I can rebuild the object by calling the individual fields like
m.getXXX()'s, but I'm trying to do this in a generic way and not have to
call the individual fields to get a myns::Person and pass the object to
another thread for processing. So, my goal is to not have to call any proto
object methods (getters or setters) and hope to call something from the
Reader to get some sort of reference to the object since I know it's type.
Is that possible?
On Saturday, July 17, 2021 at 12:22:10 PM UTC-5 ken...@cloudflare.com wrote:
> There is no way to find out a message's type if you don't know it already.
> So, you'll need to arrange to communicate the type yourself. The easiest
> way is to write an "envelope" message that is a union of all your other
> messages.
>
> struct OuterMessage {
> union {
> foo @0 :FooMessage;
> bar @1 :BarMessage;
> baz @2 :BazMessage;
> }
> }
>
> Then, always send OuterMessage as the top-level message type.
>
> -Kenton
>
> On Sat, Jul 17, 2021 at 11:55 AM SR D wrote:
>
>> If the sender is sending any number of different data types that have
>> been serialized across the network, how does one know which type is coming
>> in so that it can be pulled off using FlatArrayMessageReader's::getRoot<>?
>>
>> E.g., say this while loop is constantly reading serialized data from the
>> network and results in a vector of message buffers.
>>
>> for (auto& msg : messages) {
>> auto arr = kj::ArrayPtr(
>> reinterpret_cast(msg),
>> size / sizeof(capnp::word));
>>
>> capnp::FlatArrayMessageReader reader(arr);
>>
>> // how would one know what type to use here?
>> auto msg = reader.getRoot();
>> }
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Cap'n Proto" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to capnproto+...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/capnproto/c89926fe-3b18-4e0f-9d30-8f8134b78a0an%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/capnproto/c89926fe-3b18-4e0f-9d30-8f8134b78a0an%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to capnproto+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/capnproto/9545bbfc-38a3-44d4-bf5d-2f059e32dfa3n%40googlegroups.com.
Re: [capnproto] How to align capnproto data to architectures word size?
Thank you.
On Saturday, July 17, 2021 at 10:04:01 AM UTC-5 ken...@cloudflare.com wrote:
> Ideally, you'd update the implementation of `getBuff()` so that it
> allocates an aligned buffer. Note that malloc() always returns aligned
> buffers, so if you can make it so that the pointer returned by getBuff() is
> one that was allocated directly by malloc(), then it should work. Probably,
> though, getBuff() is returning a pointer into some larger buffer that was
> read off the wire, including some framing bytes that are pushing the data
> out of alignment. In this case it may be impractical to get the data to
> come out aligned, so you may have no choice but to make a copy. You can do
> something like:
>
> auto copy = kj::heapArray(size / sizeof(capnp::word));
> memcpy(copy.begin(), buff, size);
>
> The copy will then be aligned.
>
> -Kenton
>
> On Fri, Jul 16, 2021 at 2:39 PM SR D wrote:
>
>> I am trying to reconstruct received data using Capnproto, but am getting
>> the following error.
>>
>> terminate called after throwing an instance of 'kj::ExceptionImpl'
>> what(): capnp/arena.c++:76: failed: expected
>> reinterpret_cast(segment.begin()) % sizeof(void*) == 0 [4 == 0];
>> Detected unaligned data in Cap'n Proto message. Messages must be aligned to
>> the architecture's word size. Yes, even on x86: Unaligned access is
>> undefined behavior under the C/C++ language standard, and compilers can and
>> do assume alignment for the purpose of optimizations. Unaligned access may
>> lead to crashes or subtle corruption. For example, GCC will use SIMD
>> instructions in optimizations, and those instrsuctions require alignment.
>> If you really insist on taking your changes with unaligned data, compile
>> the Cap'n Proto library with -DCAPNP_ALLOW_UNALIGNED to remove this check.
>> stack: 7f4233c40ca5 7f423bbc40cf6 7f42bbccaa27 55f370c02321
>> 55f770cc18e3 55f77ac01331 55f77a0b12da 55f7a0cb127b 55fc70c0d1ed
>> 55fe70c01137 7f42bffa7de3 7f42b5cc7608 7f42b5aa6292
>>
>> It is from this snippet where I am receiving the buffer and trying to
>> rebuild the data structure.
>>
>> const void* buff = getBuff();
>>
>> auto arr = kj::ArrayPtr(
>> reinterpret_cast(buff),
>> size / sizeof(capnp::word));
>>
>> capnp::FlatArrayMessageReader msgReader(arr);
>>
>> auto msg = msgReader.getRoot(); // error occurs at this line
>> auto fieldA = msg.getFieldA();
>> auto fieldB = msg.getFieldB();
>>
>>
>> The buffer was constructed and sent using:
>>
>> capnp::MallocMessageBuilder message;
>> auto stuff = message.initRoot();
>> stuff.setFieldA("A");
>> stuff.setFieldB("B");
>>
>> const auto msgToSend = capnp::messageToFlatArray(message);
>> const auto msgAsChars = msgToSend.asChars();
>>
>> // make const void* and send
>>
>> How do I align the data to the architectures word size as the error says?
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Cap'n Proto" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to capnproto+...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/capnproto/c6755765-f1f3-4de6-af29-dfdb2bb83881n%40googlegroups.com
>>
>> <https://groups.google.com/d/msgid/capnproto/c6755765-f1f3-4de6-af29-dfdb2bb83881n%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to capnproto+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/capnproto/692e403e-0297-460f-b6df-35520a4d939bn%40googlegroups.com.
[capnproto] Selecting the correct message type?
If the sender is sending any number of different data types that have been
serialized across the network, how does one know which type is coming in so
that it can be pulled off using FlatArrayMessageReader's::getRoot<>?
E.g., say this while loop is constantly reading serialized data from the
network and results in a vector of message buffers.
for (auto& msg : messages) {
auto arr = kj::ArrayPtr(
reinterpret_cast(msg),
size / sizeof(capnp::word));
capnp::FlatArrayMessageReader reader(arr);
// how would one know what type to use here?
auto msg = reader.getRoot();
}
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to capnproto+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/capnproto/c89926fe-3b18-4e0f-9d30-8f8134b78a0an%40googlegroups.com.
[capnproto] How to align capnproto data to architectures word size?
I am trying to reconstruct received data using Capnproto, but am getting
the following error.
terminate called after throwing an instance of 'kj::ExceptionImpl'
what(): capnp/arena.c++:76: failed: expected
reinterpret_cast(segment.begin()) % sizeof(void*) == 0 [4 == 0];
Detected unaligned data in Cap'n Proto message. Messages must be aligned to
the architecture's word size. Yes, even on x86: Unaligned access is
undefined behavior under the C/C++ language standard, and compilers can and
do assume alignment for the purpose of optimizations. Unaligned access may
lead to crashes or subtle corruption. For example, GCC will use SIMD
instructions in optimizations, and those instrsuctions require alignment.
If you really insist on taking your changes with unaligned data, compile
the Cap'n Proto library with -DCAPNP_ALLOW_UNALIGNED to remove this check.
stack: 7f4233c40ca5 7f423bbc40cf6 7f42bbccaa27 55f370c02321
55f770cc18e3 55f77ac01331 55f77a0b12da 55f7a0cb127b 55fc70c0d1ed
55fe70c01137 7f42bffa7de3 7f42b5cc7608 7f42b5aa6292
It is from this snippet where I am receiving the buffer and trying to
rebuild the data structure.
const void* buff = getBuff();
auto arr = kj::ArrayPtr(
reinterpret_cast(buff),
size / sizeof(capnp::word));
capnp::FlatArrayMessageReader msgReader(arr);
auto msg = msgReader.getRoot(); // error occurs at this line
auto fieldA = msg.getFieldA();
auto fieldB = msg.getFieldB();
The buffer was constructed and sent using:
capnp::MallocMessageBuilder message;
auto stuff = message.initRoot();
stuff.setFieldA("A");
stuff.setFieldB("B");
const auto msgToSend = capnp::messageToFlatArray(message);
const auto msgAsChars = msgToSend.asChars();
// make const void* and send
How do I align the data to the architectures word size as the error says?
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to capnproto+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/capnproto/c6755765-f1f3-4de6-af29-dfdb2bb83881n%40googlegroups.com.
