Re: [Lightning-dev] Lightning RPC
On Mon, Jan 24, 2022 at 01:54:49PM +1030, Rusty Russell wrote: Christian Decker writes: William Casarin writes: I think the end goal of an RPC bolt [blip] would be super powerful, so that lnsocket could talk to any lightning node, but that could be further down the line. Choosing the right data format seemed like an important step in that direction. Would love to hear your thoughts on this! I agree. Exchanging the transport layer underneath grpc doesn't change semantics, but does unlock a number of potential use-cases. I think either the JSON-RPC or grpc can serve as a basis for a common RPC definition that can have any number of bindings, since we generate conversion code to/from JSON-RPC and grpc we can transparently map them back and forth. Yeah, I don't think we'll end up with a control standard. But I've been pleasantly surprised before: certainly a common subset would be nice! I ended up just using json+commando for my prototype[1]. I'm not going to overengineer anything yet. If there's a way write plugins for the other implementations I could start hacking away at a common control subset, since I do eventually want an iOS app that controls all node implementations. I will try to get something working across multiple implementations before writing up a spec. Cheers, Will [1] http://git.jb55.com/lnsocket/file/rpc.c.html ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Lightning RPC
Christian Decker writes: >> I noticed the commando c-lightning plugin just uses the JSON-RPC >> payload, but perhaps something more compact and rpc-friendly like grpc >> would be better... which is why this cln-grpc PR peaked my curiosity. > > Yep, JSON-RPC is rather bad with binary data, and doesn't have any > concept of streaming. I personally like grpc because it ticks a lot of > boxes: secure transport over TLS, mutual authentication via mTLS, > possibility to add metadata to calls (technically prohibited by the > JSON-RPC spec) which can help us use macaroons/runes in future, > streaming support and compact binary format. The rest is true, but spec doesn't disallow extra fields that I can find? > Having an IDL to describe the interface is also rather nice, even though > for cln-grpc we actually generate that from the JSON-RPC schemas, so > it's a bit less expressive than .proto files. > >> I think the end goal of an RPC bolt would be super powerful, so that >> lnsocket could talk to any lightning node, but that could be further >> down the line. Choosing the right data format seemed like an important >> step in that direction. Would love to hear your thoughts on this! > > I agree. Exchanging the transport layer underneath grpc doesn't change > semantics, but does unlock a number of potential use-cases. I think > either the JSON-RPC or grpc can serve as a basis for a common RPC > definition that can have any number of bindings, since we generate > conversion code to/from JSON-RPC and grpc we can transparently map them > back and forth. Yeah, I don't think we'll end up with a control standard. But I've been pleasantly surprised before: certainly a common subset would be nice! Cheers! Rusty. ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
Re: [Lightning-dev] Lightning RPC
Hi Will, > I noticed you are doing RPC stuff... I'm looking to do RPC over > lightning itself. I started a C library called lnsocket[1], scrounged > from clightning parts, so that I can send messages from iOS to control > my lightning node. Sounds interesting, and similar to commando's goals. Rusty also has a summer of bitcoin project attempting to expose a websocket directly to browsers in order to provide another way to communicate with your node, and of course there's commando. > I've got to the point with lnsocket where I can send TLVs to my node, > and now I'm starting to think about what format the RPC commands should > be. > > I noticed the commando c-lightning plugin just uses the JSON-RPC > payload, but perhaps something more compact and rpc-friendly like grpc > would be better... which is why this cln-grpc PR peaked my curiosity. Yep, JSON-RPC is rather bad with binary data, and doesn't have any concept of streaming. I personally like grpc because it ticks a lot of boxes: secure transport over TLS, mutual authentication via mTLS, possibility to add metadata to calls (technically prohibited by the JSON-RPC spec) which can help us use macaroons/runes in future, streaming support and compact binary format. Having an IDL to describe the interface is also rather nice, even though for cln-grpc we actually generate that from the JSON-RPC schemas, so it's a bit less expressive than .proto files. > I think the end goal of an RPC bolt would be super powerful, so that > lnsocket could talk to any lightning node, but that could be further > down the line. Choosing the right data format seemed like an important > step in that direction. Would love to hear your thoughts on this! I agree. Exchanging the transport layer underneath grpc doesn't change semantics, but does unlock a number of potential use-cases. I think either the JSON-RPC or grpc can serve as a basis for a common RPC definition that can have any number of bindings, since we generate conversion code to/from JSON-RPC and grpc we can transparently map them back and forth. > I've cc'd clightning/lightning-dev as well to see if anyone else is > working on this or thinking about this stuff right now. Definitely open to suggestions, comments and criticism: the cln-grpc [1] crate is rather new, and will see a number of rebases and fixups, but should be reviewable as is. The cln-plugin [2] crate is a bit less well-fleshed-out, but has the core functionality needed for cln-grpc-plugin which was the goal of this first exploration. The cln-rpc [4] crate is also missing many RPC commands, but that's just grunt work that I plan to tackle separately :-) Cheers, Christian [1] https://github.com/ElementsProject/lightning/pull/5011 [2] https://github.com/ElementsProject/lightning/pull/5012 [3] https://github.com/ElementsProject/lightning/pull/5013 [4] https://github.com/ElementsProject/lightning/pull/5010 ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
[Lightning-dev] Lightning RPC
Hey Christian, I noticed you are doing RPC stuff... I'm looking to do RPC over lightning itself. I started a C library called lnsocket[1], scrounged from clightning parts, so that I can send messages from iOS to control my lightning node. I've got to the point with lnsocket where I can send TLVs to my node, and now I'm starting to think about what format the RPC commands should be. I noticed the commando c-lightning plugin just uses the JSON-RPC payload, but perhaps something more compact and rpc-friendly like grpc would be better... which is why this cln-grpc PR peaked my curiosity. I think the end goal of an RPC bolt would be super powerful, so that lnsocket could talk to any lightning node, but that could be further down the line. Choosing the right data format seemed like an important step in that direction. Would love to hear your thoughts on this! I've cc'd clightning/lightning-dev as well to see if anyone else is working on this or thinking about this stuff right now. Cheers, Will [1] http://git.jb55.com/lnsocket On Tue, Jan 18, 2022 at 06:01:46AM -0800, Christian Decker wrote: This is the final PR in the cln-* series. It uses all the primitives we built in the previous 3 PRs and uses them to expose the JSON-RPC over grpc, with mTLS authentication builtin. You can view, comment on, or merge this pull request online at: https://github.com/ElementsProject/lightning/pull/5013 -- Commit Summary -- * cln-grpc-plugin: Add scaffolding for the cln-grpc-plugin * make: Add a hook for us to depend on generated files for tests * make: Generate grpc bindings if we want to test with rust enabled ___ Lightning-dev mailing list Lightning-dev@lists.linuxfoundation.org https://lists.linuxfoundation.org/mailman/listinfo/lightning-dev
