I think this is mostly a good summary. I'd maybe call out/emphasize that
I think the *most* fundamental difference at the serialization level (as
opposed to RPC) is that Cap'n Proto is designed for efficient in-memory
access, not requiring an up-front parsing step. Ironically, this is most
For distinguishing the type, I would recommend wrapping your messages in
a common union type, e.g:
struct Message {
union {
login @0 :Login;
commandGetData @1 :CommandGetData;
# etc.
}
}
The rpc protocol itself does something similar on the wire; see
rpc.capnp.
For determining
I guess my question is, what even needs defining? It's tempting to try
to use the independent streams for flow control, but:
- By now capnp-c++ already has some built-in support for this that will
need to stay there (and probably be extended eventually) for other
transports.
- The go
I think the typescript implementation was recently updated to use
javascript's bigints for UInt64, rather than defining a special datatype
for them.
Quoting ruhdv (2022-01-31 08:30:06)
>Hi
>
>I created a struct with a UInt64 property ...
>
>It like this in .capnp.d.ts file :
>
>
Quoting Kenton Varda (2021-11-23 19:20:50)
>On Tue, Nov 23, 2021 at 5:01 PM Ian Denhardt <[1]i...@zenhack.net>
>wrote:
>
> Ok, I think I get it, let me know if I have this right:
> [...]
>
>Right.
Ok, great. Thanks for your patience.
> C
? Assuming of course there
is something like the flow limit provided by the vat on the other side
of the connection.
-Ian
Quoting Kenton Varda (2021-11-23 17:32:44)
>On Tue, Nov 23, 2021 at 3:59 PM Ian Denhardt <[1]i...@zenhack.net>
>wrote:
>
> What are apps *su
(Adding Louis to cc per his request)
Quoting Kenton Varda (2021-11-23 14:50:20)
>On Tue, Nov 23, 2021 at 12:41 PM Ian Denhardt <[1]i...@zenhack.net>
>wrote:
>
> Wouldn't releasing it on return allow the caller to cause runaway
> memory
> usa
Quoting Kenton Varda (2021-11-23 13:02:32)
>Hmm, I think the intention was that the flow limit should be released
>on Return, independent of Finish. But I can totally believe I
>implemented it wrong. Could we just change it to be based on Return?
>FWIW by default there is no flow
Hey all,
A few days ago one of my co-maintainers (Louis) alerted me to a deadlock
in the Go implementation. We've pinned down the cause, and while trying
to figure out how to fix it, I looked into how the C++ implementation
handles backpressure.
>From what I can tell, the only way backpressure
Quoting Daurnimator (2021-11-14 19:00:47)
> Some misc notes:
>
> - I recall issues with using zig structs, as there is no way to e.g.
> little-endian integer type
> - Zig doesn't have anonymous fields
> - Things would be nicer with https://github.com/ziglang/zig/issues/6478
Yeah, I would
I've vaguely wanted this in the past, glad someone is interested and
motivated enough to get their hands dirty :)
Assuming you're interested in doing this only for C++, you'd want to make
the changes in the capnpc-c++ code generator plugin. Maybe Kenton can
provide more specific guidance than
Quoting Julián Díaz (2021-03-29 15:58:17)
> I'll probably bring that back because the gulpfile never worked quite
> as well as the Makefile did.
+1.
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop
Quoting Ryan Patterson (2021-03-25 22:04:01)
>This might not be what you're looking for, but I took the
>fasterthanlime fork of capnp-ts and upgraded all of the dependencies,
>and in my test application I was able to use a go capnp server with a
>Typescript capnp client over web
Quoting pepij...@gmail.com (2021-03-25 12:38:43)
> Similarly I saw a few mentions of HTTP and websocket stuff, but is
> there actually a way to use Capnproto over a websocket? I need none of
> that right now, but it's interesting to know what the options are.
Not out of the box in the C++
Quoting Matt Stern (2021-02-16 14:21:43)
> Is this safe to do? If it's relevant, team A produces with C++ and
> team B consumes with Java.
No; the representations of Void and InternalData are different (I assume
the latter is a struct), so you'll end up with two types that are
wire-incompatible.
Quoting Vaci (2020-11-26 11:54:48)
>Hi,
>
>I now have a reasonably complete implementation of Capnproto RPC in
>pure Java.
Nice!
>
>The major features still missing are:
>
> * file descriptor passing
Fwiw, you're in good company here -- I think the C++ implementation is
the
The ID doesn't affect the encoding itself, so the basic things will
still work.
You can avoid changing the id by specifying it explicitly, e.g.
struct Bar 0xfeefefffefeefefe {
val @0 :UInt32;
}
You can discover the current id by running:
capnp compile -ocapnp myschema.capnp
Which will
Hey all,
I was looking at schema.capnp, and noticed that the enum variant of the
Type struct has a brand field. This doesn't make sense to me: unless I'm
missing something, enums can't contain pointers, so it doesn't make
sense to have type parameters on an enum. What is this field for?
-Ian
--
Quoting Smurf En Drek (2020-10-16 20:46:37)
> To avoid this, I assume the methods should be made aware of them being
> pipelined so they can pass a set of locks on some keys around between
> method calls so that everything stays consistent, even with pipelined
> calls. Are there Cap'nProto
after next.
>As it happens I am available at 13:00 PDT (15:00 CDT) Friday if you
>still want to chat then.
>-Kenton
>On Fri, Sep 25, 2020 at 10:59 AM Ian Denhardt <[3]i...@zenhack.net>
>wrote:
>
> Works for me.
> Quoting lthibault (2020-09-2
tember 25, 2020 at 10:52:25 AM UTC-4 rli...@gmail.com
>wrote:
>
> I'm available next Friday (Oct 2) from 1p PT onward.
>
>On Friday, September 25, 2020 at 7:36:07 AM UTC-7 lthibault wrote:
>
> Sounds good to me � looking forward to it.
> - Louis
&
I just opened an issue:
https://github.com/capnproto/go-capnproto2/issues/160
I believe Kenton is in Texas these days. For my part I could manage most
afternoons/evenings (eastern time) though Monday & Thursday afternoons
are bad. But if Ross and/or Kenton are interested in joining us I
Quoting lthibault (2020-09-23 14:41:13)
> How do you suggest we proceed? I'm thinking it might be useful to get
> synchronized over a Zoom call since (a) I could benefit from a guided
> tour of v3 and (b) my initial high-level questions about the Level 3
> protocol remain open. Would be cool if
If you're keen on working on this I can probably be roped in; I've used
the go implementation a fair bit and contributed some patches, and it
would be neat to have a level 3 implementation out there.
Maybe you're aware of this, but you probably want to start from the v3
branch of the Go
Quoting Ryan Patterson (2020-09-16 00:44:05)
> That said, I'm not convinced about your store use case. It doesn't
> seem like something that needs to be integral to the capnp protocol,
> since there's already a shared schema between the peers.
Having slept on it I think you're right; the data
I was actually thinking about reflection like this this morning, though
my use case is a bit different: I'm building a data store on top of
capnproto, and would like to be able to introspect the stored objects
without a-priori knowing their schema. This probably means needing to
store the schema,
The closest analogue to Rust's enums is a union. You could define
`Result` like:
struct Result(T, E) {
union {
ok @0 :T;
err @1 :E;
}
}
Note however that rust-style error checking has problems in an rpc
context, in particular using this as a return value will defeat
ist(foo(b)) that is the problem capnp compile command
>generates an error saying this is impossible
>
>On Mon, May 4, 2020 at 7:12 pm, Ian Denhardt wrote:
>
>"Pointers" aren't an explicit thing in the schema language. Data, Text,
>structs, lists and int
Quoting 'Kenton Varda' via Cap'n Proto (2020-04-30 18:51:40)
>Yeah, I think if we really really wanted to do this, the approach I'd
>take is to have the capnp code generator output full specializations
>for each permutation of primitive type arguments. The really hard part
>is
Yes, this is correct.
Quoting Vitali Lovich (2020-04-16 19:45:05)
>If I change UInt64 to Int64 provided that I know I never sent a value
>that's > 2^63 - 1 ? Even if I did am I correct that it's the equivalent
>behaviour of treating it equivalent to a C++ cast?
>
>--
>You
Correct, setting a field with a primitive type will not allocate
additional space.
-Ian
Quoting vlov...@gmail.com (2020-04-16 16:53:46)
>I don't see this explicitly called out
>in� [1]https://capnproto.org/cxx.html� but I just wanted to
>double-check that something like this:
>
Did this ever go anywhere? I'm working on some elm <-> haskell
interaction for a sandstorm app and this would be nice to have.
Quoting prasanth somasundar (2019-05-29 05:00:46)
>Hey Everyone,
>
>I'm thinking about building out a Cap'n Proto implementation in Elm for
>the fun of it.
Yes, the capnproto spec officially forbids sharing inside a message
Different implementations may have varying levels of tolerance for
this, but it's probably best not to bend the spec here.
Have a look at schema.capnp, in particular the way the `Node` type is
managed by id, for some inspiration.
I'm generally supportive of this, but also worth considering if the
change doesn't land: as an alternative to the current unsafe
bytes_to_words, you could provide a version that returns a Result, which
is Err unless the argument is not 8-byte aligned or the cpu architecture
is known to be able to
Zig is neat! Yes, you'll want to write a schema compiler plugin, and
will also need to write some code to handle non-schema-specific details
of the format; I would start by reading the encoding docs and build some
support for reading "untyped" structs/lists/etc.
You'll want to decide what
If it's a pointer type (structs, lists, interfaces), you can change the
field's type to AnyPointer without affecting layout, which will let you
delete the unused type definition.
Other than that there's not really support for this; you'll see
obsoleteFoo and deprecatedFoo in other codebases and
I'll weigh in on a few of these, though others will have to cover the
C++ specific details; I've mostly used the Go and Haskell
implementations.
Quoting 'Newbugreport' via Cap'n Proto (2019-10-22 06:01:02)
>2. The documentation describes capnp as a capability based system. This
>is a new
Quoting Kenton Varda (2019-09-20 15:01:44)
> As to the original question, again, I think what Kuba is trying to do
> ought to work.
I think I've come around to agreeing here.
> And as to what's allowed at runtime, it seems reasonable that we could
> support covariance on struct-typed setters.
ualize overarching design goals that might have driven this.
>Now I do appreciate that implementation realities often curtail fully
>developed designs, so I'm not trying to imply that the present way
>Capnproto works is somehow inherently "bad" - it is what it
Quoting 'Kenton Varda' via Cap'n Proto (2019-09-16 16:14:59)
>Anyway, I guess given that there's no such thing as a constant
>capability currently, we don't need to worry about that? And covariance
>is correct for all other types? So we could support it?
It's sound for constants, but
+1.
As a data point: in the Haskell runtime library's RPC support, I ended
up manually defining newtype wrappers around Ids for Import/Export and
Question/Answer -- this is kinda sad since the schema actually already
define aliases, but the schema compiler doesn't convey this in any way
to the
Quoting ezy...@mit.edu (2019-09-09 23:15:45)
> So Cap'n Proto is first and foremost a serialization library, and only
> incidentally a data structure description language (at least, until
> https://groups.google.com/forum/#!searchin/capnproto/refcounted%7Csort:
>
Haven't looked at the code for the C++ implementation, but based on my
knowledge of the wire format[1] I would assume:
1. read() 4 bytes to get the number of segments
2. read() the list of segment sizes
3. readv() to read in all the segments
[1]:
With a hypothetical level-3 capnproto implementation, I don't think you
actually need this feature; you could do something like this:
interface Node {
ping @0 ();
store @1 (key :Data, val :Data);
findNode @2 (id :NodeId) -> (results :List(Result));
findValue @3
Quoting Kenton Varda (2019-06-26 07:13:03)
>One thing that is part of that is making sure related declarations can
>be grouped together, so that they can easily be read and digested
>together, without scrolling back and forth. I'd say this is the main
>motivation for nested
Hey all,
A few weeks ago in the thread about the Elm implementation, I mentioned
that I had a longer critique of the schema language that I'd been
meaning to write up. I finally got around to it; the blog post is here:
Quoting Kenton Varda (2019-06-07 09:45:37)
>On Thu, Jun 6, 2019 at 7:41 PM Prasanth Somasundar
><[1]mezu...@live.com> wrote:
>
>Hey guys,
>I've updated [2]the doc with a new write API suggestion. The gist is
>that writes never happen to the byte array. Instead, we build a
Quoting Prasanth Somasundar (2019-06-01 23:12:20)
> What about just dumping everything into the same module, but without
> the prefixed namespace for every value/type/type constructor. The
> reason that we're prefixing everything with underscores is to avoid
> name collisions, but we can simply
Quoting Prasanth Somasundar (2019-06-01 04:21:18)
>Except for module names. Like I said, I'll try to fix this as best I
>can and see if it makes sense to pursue. My current plan is to
>topologically sort the graph, break any cycles with extra generics, and
>then hide the generics
Quoting 'Kenton Varda' via Cap'n Proto (2019-05-30 14:22:59)
>Actual line of code from Cloudflare Workers:
>case PipelineDef::Stage::Worker::Global::Value::JSON:
>So again, I'm not sure this is a problem specific to certain languages.
>:)
Oof, point taken. Slightly off topic for
Quoting prasanth somasundar (2019-05-30 13:50:26)
> I'm hoping that I can use `elm-bytes` in a better way than being
> forced to decode it into a full Elm data type - though as I say this,
> it seems that I'd need some buy-in that I can't be guaranteed. I may
> end up with that solution in the
Quoting prasanth somasundar (2019-05-29 22:08:27)
> Thanks for the feedback. This is really helpful.
You're welcome.
> * Not supporting every feature/Optimize for well written schema
> Very interesting point. I'm not generally good at making those sorts of
> concessions, but I probably should
Quoting David Renshaw (2019-05-29 21:33:03)
>This has piqued my interest. Which parts of the schema language don't
>map well to Haskell/Elm?
The biggest one is nested namespaces, per discussion. Neither language
has intra-module namespaces, so you either end up doing a bunch of
complex
Oops, forgot to cc the list.
Quoting Ian Denhardt (2019-05-29 16:13:32)
> Neat. Some feedback:
>
> * Re: why not the Haskell implementation, it's definitely stable enough
> and complete enough to write a compiler plugin. At the serialization
> layer there are only a couple
Quoting 'Kenton Varda' via Cap'n Proto (2019-05-24 19:39:38)
>I don't know much about how the Go implementation works -- I imagine
>due to the nature of Go, it's more likely to utilize multiple threads.
Indeed, with the Go implementation you should generally get as much CPU
parallelism
Quoting eamonn.nugent via Cap'n Proto (2019-05-23 20:50:04)
>My question is: is there space in the capnproto protocol to have
>multithreaded receivers, or have the scheduling use multiple threads
>for its event queue?
By receivers do you mean a single object?
If so, the default is
Quoting Yaron Minsky (2019-05-14 16:06:29)
> If I write something in the new spec that uses the age branch of the
> union, the old struct can try to read it, and get very confused.
> In particular, if someone tries to read the email for a struct that
> actually populates age, they'll end up
The encoding spec mentions in the description of capability pointers
that values where B != 0 are reserved for future use, but doesn't
describe what an implementation not recognizing such a pointer should
do. What is the correct behavior here?
-Ian
--
You received this message because you are
Quoting 'Kenton Varda' via Cap'n Proto (2019-02-11 13:51:41)
>However, if all you need to serialize is a single large
>array of floats, then you could also just reinterpret your array as
>bytes and read/write it directly. This is essentially what Cap'n Proto
>will do under tho
Quoting Kenton Varda (2019-02-04 12:21:07)
>Shall I move Haskell into the "Serialization+RPC" category
>on� [1]https://capnproto.org/otherlang.html ?
Yes, thanks.
> Out of curiosity, is there a WebAssembly story for Haskell?
There are folks working on it, but not currently. There's a
Hey All,
This morning I tagged version 0.4 of the Haskell implementation, which
includes a Level 1 RPC implementation! It's a bit rough, there are some
key gaps in the API, but it works. Links:
* https://hackage.haskell.org/package/capnp
* https://github.com/zenhack/haskell-capnp
Cheers,
-Ian
The C++ RPC implementation is not thread safe, if that's what you're
asking. The only stable RPC implementation that is is the Go one. (The
Haskell implementation I'm working on is as well, but it's not quite
ready yet).
Quoting Pappu (2018-12-07 16:38:07)
>Hi,
>I am new to capnp. Is
Quoting r.a.m...@gmail.com (2018-11-04 12:58:16)
> interface AccountManager {
> echo @0 (title: Text) -> (result: Text);
> list @1 ()-> (accounts : List(Text));
> # get @1 (title: Text) -> (account: Account);
> }
This looks fine.
>also I use following code for my server
Quoting Kenton Varda (2018-08-28 21:34:13)
> has a split, where you can either parse the whole thing up front
> and forget about it, or write very imperative-looking code with
> explicit error checks everywhere.
>
>Interesting. I would think that for validation errors, at
Quoting Kenton Varda (2018-08-27 20:41:57)
>Nice! Let me know when you want this added to the "other languages"
>page.
Go right ahead.
>How does zero-copy work out in Haskell? It seems like you can't really
>do zero-copy writes in a purely-functional way, right?
The low-level
Hey All,
I've been working on a Haskell implementation of Cap'N Proto, and the
other day finally tagged a first release and published the package:
https://hackage.haskell.org/package/capnp
The source repo is here:
https://github.com/zenhack/haskell-capnp
It's alpha quality, but
Quoting n...@8thwall.com (2018-02-21 13:39:06)
>Per capnp "evolving your schema" we tried changing the name of a struct
>but keeping its id the same e.g. changing this:
>struct MyMessageList {
> mList @0: List(MyMessage);
>}
>struct MyMessage {
> field0 @0: Int32;
>
This is also somewhat relevant to something I've been playing around
with. Nothing particularly novel, just a set of interfaces for exporting
unixy filsystems. It would be nice if I could get a union back (with
variants for directory vs. regular file) when doing something like open.
But, this
� � # and stream.done() has completed successfully.
>-Kenton
>
> On Tue, May 23, 2017 at 12:56 PM, Ian Denhardt <[1]i...@zenhack.net>
>wrote:
>
> I'm talking about reporting an error to the receiver of the data,
> not
> the caller of write().� E.
Kenton Varda (2017-05-23 11:42:44)
>Hi Ian,
>I'm not sure I understand. write() can throw an exception. Does that
>not solve the problem?
>-Kenton
>On Mon, May 22, 2017 at 5:01 PM, Ian Denhardt <[1]i...@zenhack.net>
>wrote:
>
> Are th
Are there established best practices for handling errors that occur with
async/push style interfaces, such as sandstorm's Util.ByteStream[1]?
That interface doesn't seem to supply a way to report e.g. an IO error
that occurs while streaming, and since the call that obtained the
ByteStream has
Quoting Kenton Varda (2017-04-09 18:35:48)
>1) libcapnp and libkj together add up to some 730k of code (text
>segment) these days. Unless emscripten builds are significantly
>smaller, that's probably too big.
Hard to know without trying it, but it may well be the case that wasm
Quoting Wink Saville (2017-04-07 23:31:43)
> Of course its very early days for wasm, but I'm following it and just
> beginning to play around with it.
Worth pointing out, there's also Emscripten, which has been around a
while. So that could be used while wasm support filters into things.
Has
Quoting Wink Saville (2017-04-07 23:57:48)
>The last official release looks to be v0.5.3 from 2015, I see quite a
>few commits to the repository since then, why no newer release?
IIRC, there have been problems getting master to build with MSVC, so a
new release has been blocked on that.
Quoting Kenton Varda (2017-03-11 23:25:28)
>I'm not totally convinced that this is a solvable problem (nor that it
>is in any way unique to Cap'n Proto).
The trickiest problem that's occured to me is that capnproto's
expressiveness gives you enough rope to hang yourself in ways that
Ideally the boilerplate could be cut down by having a standard type
like:
struct Option(T) {
union {
Some @0 :T;
None @1 :Void;
}
}
And then just do:
struct Foo {
age @0 :Option(Int32);
}
But the wire format doesn't really do what
Quoting Kenton Varda (2016-08-12 18:11:14)
>Cool, keep us updated!
>-Kenton
Will do!
--
You received this message because you are subscribed to the Google Groups
"Cap'n Proto" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
77 matches
Mail list logo
