I don't think the C# bindings would use the Glib-based libraries on Windows if it requires installing MSYS2 or Cygwin on the end-user's Windows machine. So don't go through the work building the Glib-based libraries with MSVC on account of the C# library.
-----Original Message----- From: Sutou Kouhei <k...@clear-code.com> Sent: Monday, September 9, 2019 4:43 PM To: dev@arrow.apache.org Subject: Re: Plasma scenarios Hi, > In theory you could use the GLib-based library with MSVC, the main > requirement is gobject-introspection > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith > ub.com%2FGNOME%2Fgobject-introspection%2Fblob%2Fmaster%2FMSVC.README.r > st&data=02%7C01%7CEric.Erhardt%40microsoft.com%7Cca22053d07d84cc6d > 98a08d7356ec83b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637036622 > 234878257&sdata=2V8%2Fdf1jBeXmgZChjnTJU37ZqOQTf0GrLKw5d%2B%2FsFaY% > 3D&reserved=0 Generally, we can use the GLib-based library without GObject Introspection if we write bindings by hand. (We can generate bindings automatically with GObject Introspection.) But we need to some tasks to build the GLib-based library with MSVC. I'll work on it in a few months. Thanks, -- kou In <CAJPUwMC--Qj4fdEm5_4HCNH+0NBdAh0=9HVMTVX-At6rokN2=a...@mail.gmail.com> "Re: Plasma scenarios" on Mon, 9 Sep 2019 12:00:00 -0500, Wes McKinney <wesmck...@gmail.com> wrote: > hi Eric, > > On Fri, Sep 6, 2019 at 5:09 PM Eric Erhardt > <eric.erha...@microsoft.com.invalid> wrote: >> >> I was looking for the high level scenarios for the Plasma In-Memory Object >> Store. A colleague of mine suggested we could use it to pass data between a >> C# process and a Python process. >> >> I've read the intro blog [0] on Plasma, which describes using the same data >> set from multiple processes - which sounds like the same scenario as above. >> >> I am trying to prioritize creating C# bindings for the Plasma client. So I'd >> like to know all the scenarios that would could be enabled with Plasma. >> >> For example: >> - could using Plasma speed up Pandas UDFs in PySpark? Because the data >> wouldn't have to go across the socket between Java and Python, but instead >> would be memory-mapped. We have similar functionality in .NET for Apache >> Spark. > > Memory still would need to be copied into the memory-mappable file, so > it's unclear whether this would be faster than passing the data > through a socket as it's being done now. > >> - Is Plasma being used by Nvidia RAPIDS? > > AFAIK it is not. It doesn't seem out of the question, though, given > that we have some level of CUDA support in Plasma now. > >> >> I know Plasma today is not supported on Windows, but I think support could >> be added since Windows supports memory mapped files (through a different API >> than mmap) and it now supports Unix Domain Sockets [1]. >> >> Also - side question about the c_glib bindings. I assume those will only >> ever work on Windows with something like Cygwin or MSYS2, right? Would >> people be opposed to adding pure "C" exports to the plasma library so the C# >> bindings could use it? (similar to the JNI support today). >> > > In theory you could use the GLib-based library with MSVC, the main > requirement is gobject-introspection > > https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgith > ub.com%2FGNOME%2Fgobject-introspection%2Fblob%2Fmaster%2FMSVC.README.r > st&data=02%7C01%7CEric.Erhardt%40microsoft.com%7Cca22053d07d84cc6d > 98a08d7356ec83b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637036622 > 234883247&sdata=8o2XPCj7xIkUgQSNMwJYMdqHVG2BNlNOqvE0P00TvEE%3D& > ;reserved=0 > > Note that GLib itself is LGPL-licensed -- since it is an optional > component in Apache Arrow, it is OK for optional components to have an > LGPL dependency (though ASF projects aren't allowed to have > mandatory/hard dependencies on LGPL). So if you do go that route just > beware the possible issues you might have down the road. > > I have no objection to adding a "plasma/plasma-c.h" with C exports. > >> Eric >> >> [0] >> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fray >> -project.github.io%2F2017%2F08%2F08%2Fplasma-in-memory-object-store.h >> tml&data=02%7C01%7CEric.Erhardt%40microsoft.com%7Cca22053d07d84cc >> 6d98a08d7356ec83b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637036 >> 622234883247&sdata=XcXFtxsbgjXntJzX3foLTJQfgdQabEHQkneQeRQDWU0%3D >> &reserved=0 [1] >> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdev >> blogs.microsoft.com%2Fcommandline%2Faf_unix-comes-to-windows%2F&d >> ata=02%7C01%7CEric.Erhardt%40microsoft.com%7Cca22053d07d84cc6d98a08d7 >> 356ec83b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637036622234883 >> 247&sdata=%2BN3hCDkppSQAHM2AMgk6SBunF70mjgXwD%2Boesz41aq0%3D& >> reserved=0
