Re: [capnproto] Does Cap'n Proto support Python 64 in windows?

Hi Jeffrey, To be honest I'm not sure if pycapnp works on Windows at all. I don't think anyone has tried that. Cap'n Proto's C++ implementation supports both Win32 and Win64. The pre-built binary available for download on capnproto.org is just the code generator tool -- a tool you'd normally run

Re: [capnproto] Unity support?

Hi Drew, It would certainly be cool if Cap'n Proto were better-supported under Unity! The "Cap'n Proto team", to the extent that there is one, is composed of various people that use Cap'n Proto in our other projects. Generally, each contributor is focused on making improvements to Cap'n Proto for

Re: [capnproto] Transferring large arrays

Hi, Yes, Cap'n Proto should handle this use case much better than Protobufs. By default there is a message size limit of 64MB, but this is for security purposes, not performance, and you can easily configure a much larger limit as needed using `ReaderOptions`. You could write your messages to a s

Re: [capnproto] Random access over network

Hi Björn, The easiest way to make this work would be to have the data set live on a network filesystem (e.g. NFS) or block device (e.g. NBD, iSCSI) which you can mount on your local system and then use mmap(). If mounting a remote filesystem is not an option, it is technically possible to do ever

Re: [capnproto] Reg. Building for Android - Makefile tests issue

Hi Amit, When cross-compiling, you must first install capnp natively on the build system. Then, you can configure the cross build to use the already-installed version rather than the in-tree build using: ./configure --with-external-capnp If you don't want to install `capnp` on the build syst

Re: [capnproto] Relations between num. calls to allocateSegment() and num. segments in getSegmentsForOutput()

Hi Junhyun, I think that, at present, the only thing that would break your assumptions is Orphanage::referenceExternalData(). You could, of course, document that your MessageBuilder does not permit referencing external data. Other than that, I think your assumptions currently hold -- but I don't

Re: [capnproto] How are unions encoded - will they take the size of the largest child datatype?

Hi Thomas, On Wed, Aug 22, 2018 at 5:33 PM, wrote: > I am trying to send a small payload over UDP and wanted to know, as the > documentation is hard to find for unions, how the encoding works for unions. > > >1. Do unions take up a byte-size equivalent to their largest data type >within?

[capnproto] 0.7.0 release candidate

Hi all, Here's a release candidate for version 0.7.0. Please try it out and let me know if you see any problems. http://capnproto.org/capnproto-c++-0.7.0-rc1.tar.gz http://capnproto.org/capnproto-c++-win32-0.7.0-rc1.zip (windows binaries) -Kenton -- You received this message because you are su

Re: [capnproto] First (alpha) release of a Haskell implementation.

Nice! Let me know when you want this added to the "other languages" page. How does zero-copy work out in Haskell? It seems like you can't really do zero-copy writes in a purely-functional way, right? -Kenton On Sun, Aug 26, 2018 at 12:04 PM, Ian Denhardt wrote: > Hey All, > > I've been working

Re: [capnproto] Transferring large arrays

Hi Codie, While Cap'n Proto is compatible with shared memory, it currently does not provide much in the way of helpers for using it. You'll have to do a number of things yourself. For how to set up shared memory, take a look at: http://man7.org/linux/man-pages/man7/shm_overview.7.html You can us

Re: [capnproto] First (alpha) release of a Haskell implementation.

On Mon, Aug 27, 2018 at 9:16 PM, Ian Denhardt wrote: > Quoting Kenton Varda (2018-08-27 20:41:57) > >Nice! Let me know when you want this added to the "other languages" > >page. > > Go right ahead. > Done (with the 0.7 release). > >How does zero-copy work out in Haskell? It seems l

Re: [capnproto] Load balancing and metrics

Hi, While Cap'n Proto is certainly capable of replacing gRPC here, obviously Google has a lot more people working on the gRPC ecosystem, and so more infrastructure has been built out there. With Cap'n Proto you will have to do more things yourself. If your needs fit well into the model supported b

Re: [capnproto] Implementation of Interfaces with Inheritance in C++

Hi Henning, I'm not sure if there's a clean answer here. C++ multiple inheritance is extremely complicated in the details and tends to blow up like this. At the very least, you will need to declare all your inheritance "virtual", like: class A: virtual public AA::Server { ... } class B:

Re: [capnproto] Difference between Cap'n Proto and pthreads

Hi Pappu, Sorry, I don't understand your question. Cap'n Proto is a serialization and RPC framework, used for communicating between services on a network or storing data on disk. pthreads is a standard low-level API to OS threading functionality. These are mostly unrelated concepts, and Cap'n Prot

Re: [capnproto] How to match version of capnproto with that of capnproto-java?

Hi emil, This looks to me like a problem with the Arch packaging specifically. It looks like the capnproto-java package was built against an older version of the capnproto package, but failed to declare the specific library version dependency. The package needs to be rebuilt against the latest ver

Re: [capnproto] Re: Dynamic reflection HowTo: Saving Schema as string and creating Dynamic Reader from string.

Hi Kerstin, Sorry for the slow reply. It looks like your code is calling loader.getAllLoaded() to get the array of schemas and then serializing them all, concatenated, into a single string. So far so good. But on the receiving end, you are reading in this string and parsing only the first schema

Re: [capnproto] Websockets server and TwoPartyVatNetwork starting point?

Hi Asa, I don't think anyone has actually implemented TwoPartyVatNetwork over WebSockets yet -- at least, I haven't heard of anyone doing it. It certainly should be possible, though. First you'll need to decide if you want to reuse WebSocket message framing at all, or just throw away the frame bou

Re: [capnproto] Websockets server and TwoPartyVatNetwork starting point?

Hi Asa, The canonical Cap'n Proto RPC protocol is not HTTP-based -- Cap'n Proto messages are written directly to the socket. So, there's no way to add a /healthz endpoint to the same port as RPC. If you want to expose HTTP, you need to create a kj::HttpServer. Here's some rough code to start with

Re: [capnproto] Websockets server and TwoPartyVatNetwork starting point?

> > On Fri, Sep 21, 2018 at 2:32 PM Asa Hammond wrote: > >> Thanks to both of you... I'll see when I can hack forward more on this >> particular line. Looks very nice. >> >> On Fri, Sep 21, 2018 at 1:27 PM Ian Denhardt wrote: >> >>> Quoting 'Ke

Re: [capnproto] How to make a constant list of enumeration values in capn proto schema file

Hi Forest, You can do: const fooStates :List(foo) = [unset, bar, bars]; The parser knows that you're parsing values of type "foo", so it doesn't need the "foo." context. -Kenton On Tue, Oct 2, 2018 at 4:28 PM wrote: > Hi. > > I am trying to make a list of constant enumeration values in a

Re: [capnproto] Self-contained structures in C++

Hi Thomas, At the moment, no one is actively working on this. It's on my list of features that I really want to build, but I seem to have too many projects and not enough time. :/ -Kenton On Thu, Oct 4, 2018 at 9:09 AM wrote: > Hi everyone, > > Is there any news on this topic? Is Anybody worki

Re: [capnproto] Capn'Proto (C++) RPC return List as return value

This line: auto lst = capnp::List< capnp::Text>::Builder(); is basically constructing `lst` to be a null pointer. So when you try to set a value through it, you get an exception (or a segfault). Try this: auto lst = context.getResults.initAccounts(1); lst.set(0, "reza"); (The 1 is

Re: [capnproto] How far away is level 4?

Hi Ralph, For basically the entire history of Cap'n Proto RPC, I've been claiming that level 3 would be implemented "soon", but so far it hasn't actually happened. Today, I can once again suspect that level 3 will be implemented "soon" for use cases inside Cloudflare (where I currently work)... A

Re: [capnproto] pycapnp RPC call with feedback.

Hi Ryan, Cap'n Proto uses a single-threaded event loop model. In order for it to be able to perform network I/O, it's necessary that the program returns to the event loop periodically. If you need to run a function that continuously executes for a long time, then you will need to arrange for that

Re: [capnproto] capnproto auf redis

Hi Ralph, A capnp::word is just an aligned 8-byte value. ArrayPtr is a pointer to a raw data buffer. The reason we use ArrayPtr instead of ArrayPtr is to show that the buffer is aligned on an 8-byte boundary and its size is a multiple of 8 bytes. You can do `array.asBytes()` to convert any ArrayP

Re: [capnproto] max union value

Hi Ortal, On Wed, Nov 28, 2018 at 10:23 AM ortal levi wrote: > I am using cap'n proto to serialize messages in C. > I haven't used the C implementation. You might have more success e-mailing the authors directly or posting a Github issue on the specific project, as I don't know if they monitor

Re: [capnproto] capnproto auf redis

An ArrayPtr is just a pointer and a size. I don't know what you mean about making it point to Redis, which I thought was a database. You can do writable mmap(), but it's tricky. You need to implement a custom subclass of MessageBuilder. See the comments in capnp/message.h for details. Note, though

Re: [capnproto] put Text in AnyPointer

Hi William, When init()ing a Text field, you need to specify the size of the text: req.getPayload().initAs(123); You can also directly set the field to an existing string: req.getPayload().setAs("foo"); -Kenton On Fri, Dec 7, 2018 at 10:30 AM wrote: > Hello, > > using cap'n'proto c+

Re: [capnproto] Parallelism in capnp

Hi Pappu, "Parallelism" can mean many things and is a huge topic. You'll probably need to ask a more specific question to get the answers you seek. Cap'n Proto serialization is agnostic to concurrency. It's up to you to make sure a thread has exclusive access to a message before trying to modify

Re: [capnproto] send from python but recv in c++

I would think so, yes. I haven't used ZMQ so I can't comment on its python vs. C++ implementations. Presumably they are compatible. Cap'n Proto is definitely compatible across programming languages. So yes, you can use them together. -Kenton On Thu, Dec 20, 2018 at 11:46 AM wrote: > can i use

Re: [capnproto] build / test failure of capnproto 0.7.0 on os x Mojave

Did this happen just once or every time? We run our tests on OSX on CI and I haven't seen this before, so that's odd... -Kenton On Sun, Dec 16, 2018 at 11:55 AM Jonathan Gerber wrote: > > Attempt to follow unix directions on os x. Build succeeded but a test failed. > > PASS: capnp-evolution-test

Re: [capnproto] Two clients in capnp

Hi Omega, Sorry, I don't understand your question. Cap'n Proto RPC uses an event loop concurrency model. "Polling" shouldn't ever be necessary. -Kenton On Sun, Dec 23, 2018 at 8:47 PM Omega Ishendra wrote: > > Hello world, > > I am new to capnp. I want to know whether we can implement two serv

Re: [capnproto] Two clients in capnp

Yes, you can have as many servers as you want. In fact, once connected, Cap'n Proto doesn't know the difference between a client and a server -- either side can send messages to the other equally. Cap'n Proto is designed as a peer-to-peer protocol; any notion of client or server happens at the appl

Re: [capnproto] Two Layer TLS in capnproto

Hi, There's no off-the-shelf solution for this, but you could build something on top of what exists. The most obvious approach would be to create a Cap'n Proto interface which represents a byte stream, like: interface ByteStream { write @0 (chunk :Data); end @1 (); # indicates E

Re: [capnproto] Re: [BEGINNER] RPC Return List Problem C++

Hi Raphael, Lists of structs are a bit awkward. There is no set() method, but there is a setWithCaveats() method, which is mostly the same but can lead to some gotchas. See the comments here: https://github.com/capnproto/capnproto/blob/677a52ab2412f34028e43400a96a468618b94493/c++/src/capnp/list.h

Re: [capnproto] Re: Multiple EventLoops in a thread.

Hi Reza, Normally you should create your event loop in the main() function or somewhere similarly high-up so that everything in the program can share it. Then you don't need to check whether one exists already. If you can't do that, then you will need to develop your own mechanism for detecting i

Re: [capnproto] Inter process communication used in capnp

I think the way I'd do shared memory Cap'n Proto would be to have a large shared memory space mapped upfront, allocate/write each message within the space, and then signal to the other process "you can find a new message located at position X", "I am done with the message located at position Y", et

Re: [capnproto] Announcing: level 1 RPC support in the Haskell implementation

Nice! (Sorry for long delay -- e-mail overload as usual.) Shall I move Haskell into the "Serialization+RPC" category on https://capnproto.org/otherlang.html ? Out of curiosity, is there a WebAssembly story for Haskell? I've been wondering lately if Cap'n Proto makes sense as a way to communicate

Re: [capnproto] Announcing: level 1 RPC support in the Haskell implementation

On Mon, Feb 4, 2019 at 11:14 AM Ian Denhardt wrote: > >Shall I move Haskell into the "Serialization+RPC" category > >on� [1]https://capnproto.org/otherlang.html ? > > Yes, thanks. > Done! -Kenton -- You received this message because you are subscribed to the Google Groups "Cap'n Prot

Re: [capnproto] concurrent request

Yes, Cap'n Proto is fully designed for concurrent requests, using a single-threaded event loop concurrency approach. This is the default behavior in Cap'n Proto -- you don't have to do anything special to enable it. -Kenton On Thu, Feb 7, 2019 at 8:51 AM wrote: > is it possible for cap'n proto

Re: [capnproto] concurrent request

The model is essentially the same model as used in JavaScript. Any time you want to wait for something, you have to register a callback to be executed when the event completes. Only one callback can be executed at a time, but between the time that you register the callback and the time that it is c

Re: [capnproto] serialize to string

When you say "serialize to a string", do you mean text? If so, Cap'n Proto can't provide any help here; Cap'n Proto is a binary serialization framework. If you mean "string" more generally as "a string of arbitrary bytes", then, yes, Cap'n Proto is likely to out-perform Protocol Buffers for this u

Re: [capnproto] How to compile `.capnp` to `.py`

On Tue, Feb 12, 2019 at 9:20 AM wrote: > I have a pythone plugin witch called in c++(by pybind11) when I use > **import exp_capnp** in python plugin the c++ application crashed > (Segmentation fault). > 1. Does anyone have a solution? > I think we'd need to see a minimal, complete, self-contain

Re: [capnproto] Enum type problem

Hi Hong, Cap'n Proto doesn't support defining enums with non-contiguous numeric values. If you must represent your enum this way, then you will need to encode it as an integer in Cap'n Proto -- you won't be able to declare an equivalent enum type. That said, it does seem like a bug in Python's Ca

Re: [capnproto] Re: Compile `.capnp` to `.py`

Hi Fakhri, Is it possible that your pycapnp plugin is compiled against a different version of libcapnp than the one you are using in C++? If so, this could certainly cause conflicts that would lead to crashes. Unfortunately, libcapnp is not ABI-compatible between versions as C++ makes ABI compatib

Re: [capnproto] Can't compile capnpc-maven with updates

Hi Tyler, PATH contains a list of directories which the system searches to find programs. It looks like you added capnpc-java itself (an executable, not a directory) to the PATH. That won't work. You need to add the containing directory to the PATH, like: PATH=$PATH:/Users/tyler//capnproto-ma

Re: [capnproto] Reading data from ::capnp::MallocMessageBuilder into inputStream

Hi, Sorry, I don't understand what question you're asking here. You mention MallocMessageBuilder, which is used for building and outputting messages, but then you mention InputStreamReader. Are you trying to loop back data in-memory? You say you don't want to use writeMessageToFd() because you wan

Re: [capnproto] How is it decided whether a double-far pointer is needed or not? Why is there ever a need?

Hi, A double-far pointer is only needed when there is no space in the target object's segment to add a single-pointer landing pad. In this case, the landing pad itself must be allocated in a different segment and must be a double-far. In the C++ implementation, this comes up when using orphans: i

Re: [capnproto] Re: Capnp for games

Hi, In my blog post, when I said "Authors' preferred use case", I meant the respective authors of the various serialization frameworks -- NOT the author of the blog post (me). That is to say, the author of Flatbuffers had gaming in mind when he designed it. I think Cap'n Proto would be a great fi

Re: [capnproto] Failures in capnproto-c++0.7.0 test suite on make check

Hi Ashwin, It does seem likely that these failures are due to using ZFS, yes. It looks like possible ZFS does not implement the renameat2() flags and so returns EINVAL -- we should probably treat this the same as ENOSYS and fall back to rename(). However, it looks like there are also some cases

Re: [capnproto] Failures in capnproto-c++0.7.0 test suite on make check

On Tue, Mar 12, 2019 at 11:10 AM wrote: > It probably has something to do with space calculation and block sizes -- > which are configuration-dependent on ZFS. That would explain the "DiskFile holes" test failure, but not the others. The holes test is indeed pretty brittle across filesystems, b

Re: [capnproto] Failures in capnproto-c++0.7.0 test suite on make check

Unfortunately I doubt I'll have time to debug this myself, even with a machine to use. :/ I think most of the failures likely could be solved by adding "case EINVAL:" next to "case ENOSYS:" (so that the two errors are handled the same) in these two locations: https://github.com/capnproto/capnprot

Re: [capnproto] Schema design for list of children that (almost always) has a single child

Ian is correct. In this specific case, you don't save any bytes. In general, 25% of the time you'll save no bytes and 75% of the time you'll save 8 bytes, for an average of 6 bytes. In my opinion this usually isn't worth it, since it complicates application code and is likely to lead to bugs. -Ke

Re: [capnproto] How to store messages for later use without leaking memory?

Hi Colin, The reason your first two classes use more memory is because MallocMessageBuilder pre-allocates space, and your messages are probably much smaller than what it pre-allocates. You can pass values to MallocMessageBuilder's constructor to tell it to allocate less space. That said, you shou

Re: [capnproto] How can a server return an interface and still own it?

Hi Joe, Take a look at the doc comments for thisCap(): inline Capability::Client thisCap(); // Get a capability pointing to this object, much like the `this` keyword. // // The effect of this method is undefined if: // - No capability client has been created pointing to this object. (Th

Re: [capnproto] How can a server return an interface and still own it?

Hi Joe, Hmm, I'm not sure. It's helpful to read the comments that you removed, which can be found here: https://github.com/capnproto/capnproto/blob/d68cffe544e8479f3cb01a88d8a547eff8782d71/c++/src/kj/async-win32.c++#L57-L91 IoPromiseAdapter is waiting for a specific event to complete. The comple

Re: [capnproto] How can a server return an interface and still own it?

Hi Joe, Well, this is a use-after-free problem. We could in theory add code to detect various cases of use-after-free, if we see places where it happens commonly, but there's no good way for us to detect all cases. I recommend instead running your code under valgrind, which is extremely good at ca

Re: [capnproto] What should an implementation do with a "reserved" pointer type?

I'd treat it like any other invalid pointer. Realistically I don't think we can come up with a way to treat these that would allow graceful compatibility if we were to actually introduce a new pointer type in the future. -Kenton On Mon, Apr 22, 2019 at 10:07 AM Ian Denhardt wrote: > The encodin

Re: [capnproto] How to store a message and then return it later

Hi Joe, Check out this utility class OwnCapnp from the Sandstorm codebase: https://github.com/sandstorm-io/sandstorm/blob/4d86a8144cdb43120ea12845738d0fe4a6ffcda1/src/sandstorm/util.h#L495-L525 I really need to move this utility class into the Cap'n Proto library at some point... -Kenton On Fr

Re: [capnproto] Modeling a widget tree?

Hi Alex, (You didn't say which language you're using so I'll assume C++, but roughly the same should apply in other languages.) You don't need AnyPointer here. Cap'n Proto supports upcasting and downcasting capabilities, as any inheritance heirarchy should. E.g. a TextNode::Client can implicitly

Re: [capnproto] How can I detect when a client disconnects?

Hi Joe, I recommend using the TwoPartyServer class, rather than EzRpcServer. The "EZ" classes are really only meant for the simplest of use cases; if you find you need to do something they don't support, then it's time to move on to the lower-level APIs. -Kenton On Sun, May 12, 2019 at 11:52 AM

Re: [capnproto] Safety of retroactive unionization

Hi Yaron, Ian already answered the question, but I thought I'd add: For protocols that are published publicly and used by arbitrary parties that you don't control, retroactive unionization may indeed be too unsafe to really use. Many protocols, though, are used privately between components of a

Re: [capnproto] Safety of retroactive unionization

Hi Yaron, There actually is a compatibility validator library in C++. If you load the old and new schemas into the same SchemaLoader object, it will throw an exception if they aren't compatible, according to the documented compatibility rules. However, I don't think this is as useful as people im

Re: [capnproto] Safety of retroactive unionization

On Sun, May 19, 2019 at 1:38 PM Yaron Minsky wrote: > Yeah, I've heard that mentioned before. I'm curious what notion of > compatibility it checks. Forwards, backwards, or both? e.g., does it > flag the lack of forwards compatibility for retroactive unionization? > SchemaLoader only tries to d

Re: [capnproto] Multithreading RPC servers

Hi Eamonn, Definitely not a dumb question! This kind of thing is the topic of a great deal of argument and differing opinions from smart people. :) I don't know much about how the Go implementation works -- I imagine due to the nature of Go, it's more likely to utilize multiple threads. As Ian po

Re: [capnproto] Modeling a widget tree?

+David Renshaw who wrote the Rust implementation, maybe he can answer? On Sun, May 26, 2019 at 8:58 AM wrote: > So I had a chance to set down with this again, and I can not for the life > of me figure out how to get the typeId of a capability using the rust > generator. Since I'm new to the lan

Re: [capnproto] Cap'n Proto for Elm

On Wed, May 29, 2019 at 8:38 PM Ian Denhardt wrote: > The biggest one is nested namespaces, per discussion. If it's any consolation, even though C++ has nested namespaces, the code doesn't end up any less verbose. When you're using the declarations from a capnp file in C++ code, you either need

Re: [capnproto] Cap'n Proto for Elm

On Wed, May 29, 2019 at 11:27 PM Ian Denhardt wrote: > One bit of food for thought: you can't exactly mmap() in elm, That's not as true that you might think. I'm not familiar with Elm specifically, but as I understand it, it transpiles to JavaScript for execution in the browser. JavaScript ha

Re: [capnproto] Cap'n Proto for Elm

On Thu, May 30, 2019 at 10:50 AM prasanth somasundar wrote: > Pointer field defaults: Field defaults in general are not features I feel > super great about. Not that I’ve thought about this in horribly great > depth, but they seem to be very problematic if they are ever updated – your > binaries

Re: [capnproto] Cap'nproto C language implementation status

Hi Tamir, Sorry for the slow response, your e-mail for some reason landed in my spam folder. The links you provided seem to be older forks of c-capnproto, found here: https://github.com/opensourcerouting/c-capnproto However, that version has also been idle for 2 years. There's no reason to

Re: [capnproto] Cap'n Proto for Elm

On Fri, May 31, 2019 at 11:10 AM Prasanth Somasundar wrote: > >Actual line of code from Cloudflare Workers: > >case PipelineDef::Stage::Worker::Global::Value::JSON: > >So again, I'm not sure this is a problem specific to certain > languages. > >:) > > Oof, point taken. Slightly of

Re: [capnproto] Cap'n Proto for Elm

On Thu, May 30, 2019 at 3:40 PM Ian Denhardt wrote: > Oof, point taken. Slightly off topic for this list, but how would you > feel about accepting (wire compatible) patches to the sandstorm schema > to flatten the namespace? Here's another doozy (from the output of the > go code generator): > >

Re: [capnproto] Cap'n Proto for Elm

On Fri, May 31, 2019 at 2:54 PM David Renshaw wrote: > Note that Prasanth's example aliases a module that's at an intermediate > point in the nested hierarchy. I doubt that Haskell would let you do > something like: > > type Global = Pipeline'Stage'Worker'Global; > ... > case Global'Value'J

Re: [capnproto] Reading/modifying/writing config structs

Hi Ádám, You can initialize a Builder from a Reader, like: fooBulider.setBar(someBarReader); Or for the top-level MessageReader/MessageBuilder: messageBuilder.setRoot(messageReader.getRoot()); This does require a copy, but for your use case, that copy is probably not a big deal. Config

Re: [capnproto] Reading/modifying/writing config structs

I'd personally go with the union. The only drawback of unions is if for some reason you can't list all possibilities in a single file (e.g. because you want to allow extension by third parties, or something). -Kenton On Sun, Jun 2, 2019 at 11:09 PM Ádám Balázs wrote: > Hi Kenton, > > Thank you

Re: [capnproto] Cap'n Proto for Elm

On Thu, Jun 6, 2019 at 7:41 PM Prasanth Somasundar wrote: > Hey guys, > I've updated the doc > > with a new write API suggestion. The gist is that writes never happen to > the byte array. Instead, w

Re: [capnproto] Schema Checking

Hi gaussgun, The default-construction idea won't work: All default-constructed Cap'n Proto structs are effectively identical; encoding an unmodified struct and decoding it as a different type will in fact get you the default content for that other type. This is because structs are simply zero-init

Re: [capnproto] C++ forward declaration

Hi Adam, Is cmake running the capnp tool every time, or only when the capnp files change? I would suggest configuring it to only re-run the code generation when the .capnp files have changed. I don't personally know cmake but it surely supports this. Unfortunately you are correct that there's no

Re: [capnproto] Schema Checking

Yes, you could indeed use type IDs in that way. Also worth noting that if someone accidentally reuses an ID and tried to link both schemas into the same program, they'll get a linker error. -Kenton On Mon, Jun 10, 2019, 7:10 AM wrote: > I was looking deeper at the ids, and noticed something I'

Re: [capnproto] Cap'n Proto for .NET Core

Wow, this is huge! I dug in a bit and see you even implemented disembargos. Nice. Are there any docs on how to use it? Looks like the readme is pretty empty right now. Did you have to do anything unusual to map Cap'n Proto to C#? Any war stories? -Kenton On Thu, Jun 13, 2019 at 1:19 AM wrote:

Re: [capnproto] capt'n proto schema for webassembly

Hi mjbecze, This is interesting! Is this related to: https://github.com/WebAssembly/design/issues/1274 ? On Sat, Jun 15, 2019 at 9:08 AM wrote: > Hello, > I'm currently investigating whether capt'n proto schema would be > appropriate for describing Webassebly interfaces. I'm attempting to > t

Re: [capnproto] capt'n proto schema for webassembly

; schema/idl file parses into? With such an AST translated to JSON, I would >> also like to explore changing our marshaling system so that it only allows >> through calls described in the IDL and made available to the comm system at >> runtime via a derived JSON representation suitable for

Re: [capnproto] Cap'n Proto for .NET Core

Hi Christian, I'm re-adding the list to this mail since I think people will find the discussion interesting. (Sorry for my slow reply, I'm technically on parental leave...) Yes, I imagine async/await makes Cap'n Proto RPC much, much nicer. Can't wait to have it in C++. Using domain objects rath

Re: [capnproto] How can I detect when a client disconnects?

Hi Joe, RpcSystem has an alternative constructor which lets you provide a factory function to generate a new bootstrap capability (i.e. the main interface) for each connection: https://github.com/capnproto/capnproto/blob/f94b1a6fe8f60b95dd32b810193b42ad4023bbcf/c++/src/capnp/rpc.h#L87 Maybe that

Re: [capnproto] Cap'n Proto for .NET Core

Hi Christian, Is it OK for me to link your github repo from capnproto.org/otherlang.html now? Or would you prefer to hold off? Please let me know! -Kenton On Tue, Jun 18, 2019 at 6:58 AM Kenton Varda wrote: > Hi Christian, > > I'm re-adding the list to this mail since I think people will find

Re: [capnproto] Reflection-driven schema-free RPC and ser/deser

Hi Christopher, Cap'n Proto defines a compiled format for schemas -- itself expressed as Cap'n Proto -- in schema.capnp: https://github.com/capnproto/capnproto/blob/master/c++/src/capnp/schema.capnp This is actually the input format for code generators, so it definitely contains all information

Re: [capnproto] A critique of the schema language.

Nice write-up, and good feedback. I'd push back a little bit on the idea of "design for the call site", i.e. that ergonomics of application code take precedence over that of the schema language. My main concern with that notion is that schemas represent an interface, and interfaces need to be read

Re: [capnproto] A critique of the schema language.

On Thu, Jun 27, 2019 at 7:55 PM Ian Denhardt wrote: > Quoting Kenton Varda (2019-06-26 07:13:03) > > >One thing that is part of that is making sure related declarations can > >be grouped together, so that they can easily be read and digested > >together, without scrolling back and for

Re: [capnproto] Exceptions on shutdown

Hi Joe, That stack trace isn't very helpful because: 1. The exception description is missing. 2. The source file names are missing (despite line numbers being present). 3. It's a synchronous trace from the point where the exception was thrown, so it mostly just shows the event loop machinery rath

Re: [capnproto] Parser caching: transformers would need to accept const lvalue references.

Hi Kuba, As a quick fix, you could always pass a pointer instead of a reference. Or you could wrap the reference in a struct. I agree that lvalue references "should" work, but it might be tricky to design the parser code such that it doesn't accidentally decide to use a reference where inappropri

Re: [capnproto] Re: Kademlia DHT with capnp

Hi Harlan, Sorry, but I don't understand your original question. I don't see how obtaining the client's address is related to three-party handoff. I'm not sure I understand exactly what the issue is here; maybe you could provide more details? gRPC doesn't have any concept of three-party handoff -

Re: [capnproto] Re: Kademlia DHT with capnp

Hi Harlan, In the C++ library, you can get the client's IP address if you decompose a few classes. First, make sure you're using rpc-twoparty.h directly and not ez-rpc.h. On the server side, you'll want to copy the class capnp::TwoPartyServer and modify it a bit. When the code receives a connectio

Re: [capnproto] Re: Kademlia DHT with capnp

Hi Harlan, You could write a custom VatNetwork implementation that uses UDP, but it'd be a big project. We don't have anything off-the-shelf for that today. -Kenton On Wed, Jul 10, 2019 at 2:34 PM Harlan Connor wrote: > Brilliant! I shall have a go at that tomorrow. > > Would it be possible fo

Re: [capnproto] c-capnproto and fieldgetset output for union fields

Hi Mike, FWIW, I'm not sure if the authors of c-capnproto pay attention to this mailing list. You might have to contact them more directly. (Sorry, I've never used it myself.) -Kenton On Tue, Jul 16, 2019 at 9:11 PM wrote: > Hi, > > I'm finding my way around the c-capnproto implementation (fro

Re: [capnproto] Reading a multi-segment message with zero-copy

Ian is almost right. It's: 1. read() first 8 bytes, which contains the number of segments and size of the first segment. 2. (Only if more than 1 segment) read() the rest of the segment table. 3. read() the entire message content (all segments) into one big array. So in the case of a single-segmen

Re: [capnproto] Reading a multi-segment message with zero-copy

Hi Sune, On Sat, Jul 20, 2019 at 12:02 PM Sune Sash wrote: > > Thanks to both of you. > > I see that the writeMessage() in serialize.h creates a segment table and > copies over the individual segments. So I presume, in order to send a > multi-segment without incurring a copy, the application > w

Re: [capnproto] Reading a multi-segment message with zero-copy

Hi Sune, I guess SysV / POSIX message queues do not have any gather-write interface (like writev()), therefore it is probably impossible to send a multi-segment message over a message queue without performing an upfront copy to concatenate the segments. I suppose you could send each segment as a s

Re: [capnproto] Unknown field retention

Hi Dmitry, Unknown fields are automatically copied when you copy a struct -- that is, when you set a struct field to a Reader for the same type, like: someBuilder.setMyStructField(someReader.getMyStructField()); If you want to inspect or manipulate the unknown fields, you need to use the "an

Re: [capnproto] Coming from protobufs - should I still use parallel arrays?

Hi Vitali, Good news: The two schemas you wrote have identical wire size. I'd recommend going with the cleaner struct list rather than parallel lists. Details: - Struct lists are "flattened", meaning the structs are stored consecutively with nothing in between. - Fields of a struct are not "tagge

Re: [capnproto] Coming from protobufs - should I still use parallel arrays?

Hi Vitali, In this case, `input` and `output` will each be a pointer pointing to a Point2 located elsewhere. So the `Aggregate` is 16 bytes (two pointers) while each `Point2` is 8 bytes, for a total of 32 bytes. Whereas if you wrote something like: struct Aggregate { input :group { x @0 :Int32;

<    1   2   3   4   5   6   >