To be honest I'm not sure if I've ever used kj::File to load Cap'n-Proto-format data. The file API was sort of built for other things I was doing that happened to be built on KJ.
But if I were using it, I think I'd do: - Reads always using mmap(), not streams. You'll need a reinterpret_cast to get an ArrayPtr<capnp::word>, but no shims needed here. - Writes probably using Directory::appendFile() to open the file for appending. This gives you an OutputStream so you can directly use capnp::writeMessage(), no shims needed. > And newFileAppender() is a *bad* idea since it calls stat() on every single file write to query the current length I don't understand what you mean here. kj::newFileAppender() is explicitly documented as not doing this, with the caveat that it won't work correctly if there are multiple concurrent appenders. On the other hand, if you use Directory::appendFile() to open the file, then it uses the O_APPEND flag which makes it the operating system's job to ensure all bytes are appended -- no need for stat(). -Kenton On Sun, Aug 15, 2021 at 5:58 AM nyanpasu64 <nyanpasu...@gmail.com> wrote: > I've built an (unfinished) application around Cap'n Proto and the kj > filesystem library (which has the nice property of handling valid and > invalid Unicode paths properly on both Windows and Linux, using WTF-8 or > raw bytes), but sadly the filesystem library doesn't have online > documentation, only doc comments which I struggled to understand the "big > picture" from. > > With that in mind, I'm surprised that Cap'n Proto interoperates so poorly > with kj, in that capnp can't write to a kj::File or read from a > kj::ReadableFile without a shim I wrote. And newFileAppender() is a *bad* > idea since it calls stat() on every single file write to query the current > length, instead of storing the current write position in the object. > > Writes: > https://gitlab.com/exotracker/exotracker-cpp/-/blob/5109bd9411c9baaca837ae349358da3f5c3742bc/src/serialize.cpp#L421 > > Reads: > https://gitlab.com/exotracker/exotracker-cpp/-/blob/5109bd9411c9baaca837ae349358da3f5c3742bc/src/serialize.cpp#L1330 > > It's also unfortunate that it's so much work to convert between > kj::ArrayPtr and gsl/std::span, kj::StringPtr (null terminated, not > slicable) and std::string_view (not always null terminated, slicable), > juggling ConstData and Data::Reader and char and unsigned char/uint8_t, > etc. I wish it could be done better, maybe with more conversion operators, > but it's understandable considering kj was designed to avoid the C++ > standard library. > > -- > You received this message because you are subscribed to the Google Groups > "Cap'n Proto" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to capnproto+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/capnproto/97117ffc-3567-40eb-acb6-9007c0a2ad73n%40googlegroups.com > <https://groups.google.com/d/msgid/capnproto/97117ffc-3567-40eb-acb6-9007c0a2ad73n%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Cap'n Proto" group. To unsubscribe from this group and stop receiving emails from it, send an email to capnproto+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/capnproto/CAJouXQm864DYtDBYbzW2pcAerS5T7%3DcxMOewddJU%3DRKduDaaAg%40mail.gmail.com.
