Linux Dynamic Loading of shared libraries
Martin Nowak's Gihub druntime Page has
module main;
import core.runtime, core.thread;
void main() {
auto lib = Runtime.loadLibrary("./liba.so");
auto thr = new Thread({
auto lib = Runtime.loadLibrary("./liba.so");
Runtime.unloadLibrary(lib);
});
thr.start();
thr.join();
Runtime.unloadLibrary(lib);
}
module liba;
import std.stdio;
shared static this() { writeln("shared static this()"); }
shared static ~this() { writeln("shared static ~this()"); }
static this() { writeln("static this()"); }
static ~this() { writeln("static ~this()"); }
Now suppose that my D shared library contains a class, rather
that just module ctors/dtors, how do I go about creating an
instance of that class and using its methods?
Steve
Re: Linux Dynamic Loading of shared libraries
Now suppose that my D shared library contains a class, rather that just module ctors/dtors, how do I go about creating an instance of that class and using its methods? Steve For this, you create an "Interface" that matches to the method declaration of your class. But notice that instead of defining methods, you will define attributes those types' match to that class's methods. I did this before and it works. At least with Posix "dlsym" function's help.
Re: Linux Dynamic Loading of shared libraries
On Sunday, 9 March 2014 at 12:07:22 UTC, Steve Teale wrote:
Martin Nowak's Gihub druntime Page has
module main;
import core.runtime, core.thread;
void main() {
auto lib = Runtime.loadLibrary("./liba.so");
auto thr = new Thread({
auto lib = Runtime.loadLibrary("./liba.so");
Runtime.unloadLibrary(lib);
});
thr.start();
thr.join();
Runtime.unloadLibrary(lib);
}
module liba;
import std.stdio;
shared static this() { writeln("shared static this()"); }
shared static ~this() { writeln("shared static ~this()"); }
static this() { writeln("static this()"); }
static ~this() { writeln("static ~this()"); }
Now suppose that my D shared library contains a class, rather
that just module ctors/dtors, how do I go about creating an
instance of that class and using its methods?
Steve
If you know the fully qualified name of the class and an interface
auto newClassIF =
cast(interfacename)object.factory("libmodule.libclass");
Re: Linux Dynamic Loading of shared libraries
On Sunday, 9 March 2014 at 14:09:28 UTC, Tolga Cakiroglu wrote:
For this, you create an "Interface" that matches to the method
declaration of your class. But notice that instead of defining
methods, you will define attributes those types' match to that
class's methods. I did this before and it works. At least with
Posix "dlsym" function's help.
OK, so then what goes wrong here:
module exta;
class ExtA
{
int n;
this(int _n) { n = _n; }
int foo() { return ++n; }
}
ExtA getInstance(int n)
{
return new ExtA(n);
}
compiled with:
dmd exta.d -c -fPIC -shared
dmd exta.o -shared -defaultlib=libphobos2.so -L-rpath=.
module main;
import core.runtime;
import std.stdio;
extern(C) void* dlsym(void*, const char*);
extern(C) void dlclose(void*);
interface ExtA
{
int foo();
}
void main() {
void* lib = Runtime.loadLibrary("exta.so");
if (lib is null)
{
writeln("library not loaded");
return;
}
writeln("loaded");
void* vp = dlsym(lib,
"_D4exta11getInstanceFiZC4exta4ExtA\0".ptr);
if (vp is null)
{
writeln("symbol not found");
return;
}
writeln("got symbol");
ExtA function(int) f = cast(ExtA function(int)) vp;
ExtA x = f(42);
if (x is null)
{
writeln("no class object");
return;
}
int n = x.foo();
writefln("n = %d", n);
Runtime.unloadLibrary(lib);
}
compiled with:
dmd -c main.d
dmd main.o -L-ldl -defaultlib=libphobos2.so -L-rpath=.
output:
loaded
got symbol
n = 9
Segmentation fault (core dumped)
The answer should be 43. The segfault happens on the
Runtime.unloadLibrary(lib); call.
Any ideas?
Steve
Re: Linux Dynamic Loading of shared libraries
On Monday, 10 March 2014 at 06:38:35 UTC, Steve Teale wrote:
On Sunday, 9 March 2014 at 14:09:28 UTC, Tolga Cakiroglu wrote:
For this, you create an "Interface" that matches to the method
declaration of your class. But notice that instead of defining
methods, you will define attributes those types' match to that
class's methods. I did this before and it works. At least with
Posix "dlsym" function's help.
OK, so then what goes wrong here:
module exta;
class ExtA
{
int n;
this(int _n) { n = _n; }
int foo() { return ++n; }
}
ExtA getInstance(int n)
{
return new ExtA(n);
}
compiled with:
dmd exta.d -c -fPIC -shared
dmd exta.o -shared -defaultlib=libphobos2.so -L-rpath=.
module main;
import core.runtime;
import std.stdio;
extern(C) void* dlsym(void*, const char*);
extern(C) void dlclose(void*);
interface ExtA
{
int foo();
}
void main() {
void* lib = Runtime.loadLibrary("exta.so");
if (lib is null)
{
writeln("library not loaded");
return;
}
writeln("loaded");
void* vp = dlsym(lib,
"_D4exta11getInstanceFiZC4exta4ExtA\0".ptr);
if (vp is null)
{
writeln("symbol not found");
return;
}
writeln("got symbol");
ExtA function(int) f = cast(ExtA function(int)) vp;
ExtA x = f(42);
if (x is null)
{
writeln("no class object");
return;
}
int n = x.foo();
writefln("n = %d", n);
Runtime.unloadLibrary(lib);
}
compiled with:
dmd -c main.d
dmd main.o -L-ldl -defaultlib=libphobos2.so -L-rpath=.
output:
loaded
got symbol
n = 9
Segmentation fault (core dumped)
The answer should be 43. The segfault happens on the
Runtime.unloadLibrary(lib); call.
Any ideas?
Steve
confusion between main.Exta and exta.ExtA
following worked for me
module exta;
import main;
//===
class ExtA : ExtA_IF
{
int n;
this(int _n) { n = _n; }
int foo() { return ++n; }
}
ExtA_IF getInstance(int n)
{
return new ExtA(n);
}
//=
//==
module main;
import core.runtime;
import std.stdio;
extern(C) void* dlsym(void*, const char*);
extern(C) void dlclose(void*);
interface ExtA_IF
{
int foo();
}
void main() {
void* lib = Runtime.loadLibrary("exta.so");
if (lib is null)
{
writeln("library not loaded");
return;
}
writeln("loaded");
//use extern (C) to avoid mangling
void* vp = dlsym(lib,
"_D4exta11getInstanceFiZC4main7ExtA_IF\0".ptr);
if (vp is null)
{
writeln("symbol not found");
return;
}
writeln("got symbol");
ExtA_IF function(int) f = cast(ExtA_IF function(int)) vp;
ExtA_IF x = f(42);
if (x is null)
{
writeln("no class object");
return;
}
int n = x.foo();
writefln("n = %d", n);
// or free or destroy or whatever it's supposed to be to
//to avoid the seg fault (bug?)
delete x;
Runtime.unloadLibrary(lib);
}
Re: Linux Dynamic Loading of shared libraries
On Sunday, 9 March 2014 at 12:07:22 UTC, Steve Teale wrote:
Now suppose that my D shared library contains a class, rather
that just module ctors/dtors, how do I go about creating an
instance of that class and using its methods?
After wandering down several dead-end paths, and help from other
contributors, I have finally come up with something that looks
like the basis of a plugin pattern for Linux DMD using shared
objects (.so files). This is somewhat long for a forum post. You
can download this readme and the associated files from
britseyeview.com/plugin101.tar.bz2
To get started, you need a base class that provides declarations
for all functions that the plugin will be allowed to use
externally. Why base class, and not interface? Well I guess
because interfaces don't provide any information about data. If
you create a shared library based on an interface, then all the
shared object methods that reference data in the class that
implements the interface fail miserably. I'm sure someone will
explain why - probably some obvious thing I have overlooked.
OK, so my base class is:
module plugin;
class Plugin
{
int n;
this(int _n) { n = _n; }
int foo() { return int.min; }
void bar() {}
}
The class that implements this base in the shared library is:
module exta;
import plugin;
import std.stdio;
import std.math;
class ExtA: Plugin
{
double d;
this(int n) { super(n); d = PI; }
override int foo() { return ++n; }
override void bar() { writefln("Done my thing (%f)", d); }
}
Plugin getInstance(int n)
{
return new ExtA(n);
}
shared static this() {
writeln("exta.so shared static this");
}
shared static ~this() {
writeln("exta.so shared static ~this");
}
The module ctor/dtor are included because that has become
conventional in discussions about dynamic loading. Otherwise, the
so has the class implementation - ExtA, and a shared method to
create an instance of same. It includes references to methods in
Phobos.
The test program is as follows:
module main;
import core.runtime;
import std.stdio;
import plugin;
extern(C) void* dlsym(void*, const char*);
alias Plugin function(int) pfi;
Plugin getPlugin(string name)
{
void* lib = Runtime.loadLibrary(name~".so");
if (lib is null)
{
writeln("failed to load plugin shared object");
return null;
}
void* vp = dlsym(lib,
"_D4exta11getInstanceFiZC6plugin6Plugin\0".ptr);
if (vp is null)
{
writeln("plugin creator function not found");
return null;
}
pfi f = cast(pfi) vp;
Plugin x = f(42);
if (x is null)
{
writeln("creation of plugin failed");
return null;
}
return x;
}
void main()
{
Plugin x = getPlugin("exta");
int n = x.foo();
writefln("n = %d", n);
x.bar();
}
The long symbol name used in the dlsym() call is of course from
the .map file generated when the .so file is created
These can be built using the following primitive makefile, whose
main purpose is to spell out the required compiler flags:
main :
dmd -c plugin.d
dmd -c -shared -fPIC exta.d
dmd exta.o -shared -defaultlib=libphobos2.so -map
dmd -c main.d
dmd main.o plugin.o -L-ldl -defaultlib=libphobos2.so -L-rpath=.
This assumes that the plugins will be in the same directory as
the executable (rpath=.).
Note that there is no call to Runtime.unloadLibrary(). The
assumption her is that once the plugin has been loaded it will be
there for the duration of the program. If you want to unload it
you'll probably have to make sure the plugin object is purged
from memory first, and I have not discovered how to do that yet
;=(
Steve
Re: Linux Dynamic Loading of shared libraries
Bless you, mate. You described precisely what I was looking for. Nonetheless, I am still looking for a load and unload command to select and reject plugins in runtime.
Re: Linux Dynamic Loading of shared libraries
On Monday, 10 March 2014 at 11:59:20 UTC, Steve Teale wrote: Note that there is no call to Runtime.unloadLibrary(). The assumption her is that once the plugin has been loaded it will be there for the duration of the program. If you want to unload it you'll probably have to make sure the plugin object is purged from memory first, and I have not discovered how to do that yet ;=( A long time ago, Andrei suggested creating a GC interface for mapped memory. I think something similar might be appropriate here. Perhaps the location of instantiated classes could be determined by their vtbl pointer? Then the entire vtbl range could be treated as a dynamically allocated struct of sorts, and its dtor would queue up a job to unmap the library after collection is complete (ie. not immediately, since the order of destruction during a collection is undefined). So... (just thinking out loud) when a library is loaded, you basically perform an in-place construction of this LoadedLibrary struct on top of the vtbl range. You'd need an interface similar to the "GC tracks memory mapped files" idea to tell the GC to track references to this range of memory that lives outside its own pool set, and then some kind of post-collection job queue that's externally appendable so the LoadedLibrary struct could add an unload call when it's collected. Heck, we could really handle all dtors this way, so normal dtors would be inserted at the front of the list and special cases like this would be inserted at the back. It doesn't sound tremendously difficult, though we'd need the memory-mapped file support API first. Is there something I'm missing that makes this unfeasible?
Re: Linux Dynamic Loading of shared libraries
Can't retrieve the archive from that URL.
britseyeview.com/plugin101.tar.bz2
Interested, so can you please fix?
On Monday, 10 March 2014 at 11:59:20 UTC, Steve Teale wrote:
On Sunday, 9 March 2014 at 12:07:22 UTC, Steve Teale wrote:
Now suppose that my D shared library contains a class, rather
that just module ctors/dtors, how do I go about creating an
instance of that class and using its methods?
After wandering down several dead-end paths, and help from
other contributors, I have finally come up with something that
looks like the basis of a plugin pattern for Linux DMD using
shared objects (.so files). This is somewhat long for a forum
post. You can download this readme and the associated files
from britseyeview.com/plugin101.tar.bz2
To get started, you need a base class that provides
declarations for all functions that the plugin will be allowed
to use externally. Why base class, and not interface? Well I
guess because interfaces don't provide any information about
data. If you create a shared library based on an interface,
then all the shared object methods that reference data in the
class that implements the interface fail miserably. I'm sure
someone will explain why - probably some obvious thing I have
overlooked.
OK, so my base class is:
module plugin;
class Plugin
{
int n;
this(int _n) { n = _n; }
int foo() { return int.min; }
void bar() {}
}
The class that implements this base in the shared library is:
module exta;
import plugin;
import std.stdio;
import std.math;
class ExtA: Plugin
{
double d;
this(int n) { super(n); d = PI; }
override int foo() { return ++n; }
override void bar() { writefln("Done my thing (%f)", d); }
}
Plugin getInstance(int n)
{
return new ExtA(n);
}
shared static this() {
writeln("exta.so shared static this");
}
shared static ~this() {
writeln("exta.so shared static ~this");
}
The module ctor/dtor are included because that has become
conventional in discussions about dynamic loading. Otherwise,
the so has the class implementation - ExtA, and a shared method
to create an instance of same. It includes references to
methods in Phobos.
The test program is as follows:
module main;
import core.runtime;
import std.stdio;
import plugin;
extern(C) void* dlsym(void*, const char*);
alias Plugin function(int) pfi;
Plugin getPlugin(string name)
{
void* lib = Runtime.loadLibrary(name~".so");
if (lib is null)
{
writeln("failed to load plugin shared object");
return null;
}
void* vp = dlsym(lib,
"_D4exta11getInstanceFiZC6plugin6Plugin\0".ptr);
if (vp is null)
{
writeln("plugin creator function not found");
return null;
}
pfi f = cast(pfi) vp;
Plugin x = f(42);
if (x is null)
{
writeln("creation of plugin failed");
return null;
}
return x;
}
void main()
{
Plugin x = getPlugin("exta");
int n = x.foo();
writefln("n = %d", n);
x.bar();
}
The long symbol name used in the dlsym() call is of course from
the .map file generated when the .so file is created
These can be built using the following primitive makefile,
whose main purpose is to spell out the required compiler flags:
main :
dmd -c plugin.d
dmd -c -shared -fPIC exta.d
dmd exta.o -shared -defaultlib=libphobos2.so -map
dmd -c main.d
dmd main.o plugin.o -L-ldl -defaultlib=libphobos2.so -L-rpath=.
This assumes that the plugins will be in the same directory as
the executable (rpath=.).
Note that there is no call to Runtime.unloadLibrary(). The
assumption her is that once the plugin has been loaded it will
be there for the duration of the program. If you want to unload
it you'll probably have to make sure the plugin object is
purged from memory first, and I have not discovered how to do
that yet ;=(
Steve
