Hey Dustin, Thanks for getting in touch! Here are two additional ways to do it:
5. Native go client library: If Go has support to ship file descriptors over unix domain sockets (which I think it has, see https://github.com/opencontainers/runc/blob/master/libcontainer/utils/cmsg.go) and interact with memory mapped files it might also be possible to make a version of https://github.com/apache/arrow/blob/master/cpp/src/plasma/client.cc that is native go. The advantage is that it wouldn't need any additional compilation steps on the go side, the disadvantage is that it would need to be updated if the plasma client internals change (like they did recently with the removal of the release buffer). 6. GObject wrapper: Possibly one could use the GObject plasma bindings that kou and his team are managing to build a wrapper (not sure how feasible that is or if there is a mature GObject go implementation). I would encourage you to start by writing write down the ideal Go API for the client and then see how it can be implemented after that (to make sure the API, which is the most important piece, is not influenced by the implementation choice). Then, going the cgo route seems the most promising for me since that's I think the route that most go code interfaces with native libraries. There are some C bindings that have been written: https://github.com/plures/pxnd/tree/master/libplasma. If they are useful to you, we can make a plan to integrate them into the repo. Best, Philipp. On Wed, Dec 19, 2018 at 12:04 PM Dustin Long <dust...@qri.io> wrote: > Hi all! > > I am a developer on qri <https://qri.io/>, a data-science tool built on > IPFS written in go. We're interested in integrating Arrow and especially > Plasma, in order to be able to share datasets with other apps like Jupyter > Notebook. Having this functionality is going to be key for how we plan to > integrate with existing frameworks. > > I've been investigating possible approaches for how to use Plasma in our > codecase. I realize that Plasma is still a work in progress, and doesn't > have stable API yet, but we're also a ways off from being ready to fully > integrate it on our side. Just figured it would be good to start this > conversation early in order to plan ahead for how development should > proceed. > > So, the prototypes I've been hacking on have revealed a few choices of how > to make our golang codebase call Plasma's C++, and I wanted to see what the > Plasma devs think about these approaches or if they have any preference for > how the go bindings should be behave. > > Here are the options in order of what seems to be least to most usable: > > 1. cgo > Use go's builtin cgo facility to call the Plasma C++ implementation. cgo > is relatively easy to use, however it only can call C functions. So this > would require writing and maintaining a pure C language wrapper around the > C++ functionality we want to expose. A lot would be lost in translation and > the resulting go code would look nothing like the original library. > > 2. dlopen > Install Plasma as a library on the user's system, then load the library > at run-time, looking up function calls and data structures as needed. > Removes the need for a static dependency, but still requires a lot of shim > code to be written to load the shared library calls. C++'s name mangling > gets in the way a lot. > > 3. Swig > Wrap a swig interface file that exposes whatever functionality we want to > golang. The standard go tool has builtin swig support, which knows when to > invoke the swig generator, in order to create go bindings that resemble the > C++ original. The build process is relatively uninterrupted. > > I noticed there doesn't seem to be any swig in use currently in the arrow > codebase, which made me think there might have been a reason that it has > been avoided for other languages. I'm interested to hear any thoughts, or > see if there are other suggestions on how to proceed? > > Regards, > Dustin >
