Hi, GObject Plasma bindings mentioned by Philipp is the official C bindings for Plasma (Plasma GLib):
https://github.com/apache/arrow/tree/master/c_glib/plasma-glib Ruby bindings use it. So we'll maintain and improve it. There were examples to generate Go bindings for Arrow from Arrow GLib automatically: https://github.com/apache/arrow/tree/apache-arrow-0.9.0/c_glib/example/go I removed it because we started native Go bindings. The same mechanism can be used for Plasma but I don't think we should use it. We'll be able to implement Go bindings with cgo and Plasma GLib. I think that Go bindings for Plasma shouldn't export GLib related API when we use Plasma GLib. I can help when we use Plasma GLib. Thanks, -- kou In <CAOzUVQVzu=P7E7+J43A9REK_=iL7Mk_dDrJMvK_rwFAZ6Ex=3...@mail.gmail.com> "Re: plan for Go implementation of Plasma" on Wed, 19 Dec 2018 17:23:58 -0500, Dustin Long <dust...@qri.io> wrote: > Neat! Thank you for the suggestions, I'll take a look into these other > approaches. Sticking with cgo does sound promising; I had dismissed it due > to needing to maintain a C interface, but if there's already some bindings > that might become official, that negates that issue. > > On Wed, Dec 19, 2018 at 3:26 PM Philipp Moritz <pcmor...@gmail.com> wrote: > >> Hey Dustin, >> >> Thanks for getting in touch! Here are two additional ways to do it: >> >> 5. Native go client library: If Go has support to ship file descriptors >> over unix domain sockets (which I think it has, see >> >> https://github.com/opencontainers/runc/blob/master/libcontainer/utils/cmsg.go >> ) >> and interact with memory mapped files it might also be possible to make a >> version of >> https://github.com/apache/arrow/blob/master/cpp/src/plasma/client.cc that >> is native go. The advantage is that it wouldn't need any additional >> compilation steps on the go side, the disadvantage is that it would need to >> be updated if the plasma client internals change (like they did recently >> with the removal of the release buffer). >> >> 6. GObject wrapper: Possibly one could use the GObject plasma bindings >> that kou and his team are managing to build a wrapper (not sure how >> feasible that is or if there is a mature GObject go implementation). >> >> I would encourage you to start by writing write down the ideal Go API for >> the client and then see how it can be implemented after that (to make sure >> the API, which is the most important piece, is not influenced by the >> implementation choice). >> >> Then, going the cgo route seems the most promising for me since that's I >> think the route that most go code interfaces with native libraries. There >> are some C bindings that have been written: >> https://github.com/plures/pxnd/tree/master/libplasma. If they are useful >> to >> you, we can make a plan to integrate them into the repo. >> >> Best, >> Philipp. >> >> >> >> On Wed, Dec 19, 2018 at 12:04 PM Dustin Long <dust...@qri.io> wrote: >> >> > Hi all! >> > >> > I am a developer on qri <https://qri.io/>, a data-science tool built on >> > IPFS written in go. We're interested in integrating Arrow and especially >> > Plasma, in order to be able to share datasets with other apps like >> Jupyter >> > Notebook. Having this functionality is going to be key for how we plan to >> > integrate with existing frameworks. >> > >> > I've been investigating possible approaches for how to use Plasma in our >> > codecase. I realize that Plasma is still a work in progress, and doesn't >> > have stable API yet, but we're also a ways off from being ready to fully >> > integrate it on our side. Just figured it would be good to start this >> > conversation early in order to plan ahead for how development should >> > proceed. >> > >> > So, the prototypes I've been hacking on have revealed a few choices of >> how >> > to make our golang codebase call Plasma's C++, and I wanted to see what >> the >> > Plasma devs think about these approaches or if they have any preference >> for >> > how the go bindings should be behave. >> > >> > Here are the options in order of what seems to be least to most usable: >> > >> > 1. cgo >> > Use go's builtin cgo facility to call the Plasma C++ implementation. >> cgo >> > is relatively easy to use, however it only can call C functions. So this >> > would require writing and maintaining a pure C language wrapper around >> the >> > C++ functionality we want to expose. A lot would be lost in translation >> and >> > the resulting go code would look nothing like the original library. >> > >> > 2. dlopen >> > Install Plasma as a library on the user's system, then load the library >> > at run-time, looking up function calls and data structures as needed. >> > Removes the need for a static dependency, but still requires a lot of >> shim >> > code to be written to load the shared library calls. C++'s name mangling >> > gets in the way a lot. >> > >> > 3. Swig >> > Wrap a swig interface file that exposes whatever functionality we want >> to >> > golang. The standard go tool has builtin swig support, which knows when >> to >> > invoke the swig generator, in order to create go bindings that resemble >> the >> > C++ original. The build process is relatively uninterrupted. >> > >> > I noticed there doesn't seem to be any swig in use currently in the arrow >> > codebase, which made me think there might have been a reason that it has >> > been avoided for other languages. I'm interested to hear any thoughts, or >> > see if there are other suggestions on how to proceed? >> > >> > Regards, >> > Dustin >> > >>
