Re: Is it possible to dynamically load a @safe function from a shared library ?
On 3/13/20 5:06 PM, H. S. Teoh wrote:
On Fri, Mar 13, 2020 at 04:31:16PM -0400, Steven Schveighoffer via
Digitalmars-d-learn wrote:
[...]
I would expect that something could be written to turn a signature
string into a mangling and also provide the correct type upon return.
Something like:
auto f = getFunction!(@safe void function(int))("package.module.symbol");
and have it properly mangle the expected function name and pull it
from the dynamic library.
[...]
This would still have to be @trusted, of course, since there's no
telling what's actually inside the object file.
Of course it has to be trusted. The underlying function uses void *.
But not because you can't trust the mangling.
But this sort of facility totally should be in Phobos, or at least in
some dub package somewhere. It will make working with dynamically
loaded libraries in D so much more convenient.
I'm not really motivated to make such a package, as I don't do any
software with runtime loading of libraries. But I bet it would be either
accepted into Phobos and/or wouldn't be that hard to add for a library.
-Steve
Re: Is it possible to dynamically load a @safe function from a shared library ?
On 3/14/20 6:06 AM, wjoe wrote:
On Friday, 13 March 2020 at 20:31:16 UTC, Steven Schveighoffer wrote:
On 3/13/20 4:22 PM, wjoe wrote:
I would expect that something could be written to turn a signature
string into a mangling and also provide the correct type upon return.
Something like:
auto f = getFunction!(@safe void function(int))("package.module.symbol");
and have it properly mangle the expected function name and pull it
from the dynamic library.
Thanks for your reply.
core.demangle: mangle; comes to mind. And in fact, because the exports
weren't extern(C), that's how I imported the symbols; prior to reading
H. S. Teoh's reply. That's when I realized that I've got a problem.
Even though I know that the function being exported are all compiled
@safe, that doesn't mean I can count on the fact.
Since a plugin is a separate file it can be swapped out with a version
that exports all the same (forged) symbols names but with a @system
implementation.
Forging these symbol names is as easy as something like mangle!(int
function(int))("a.b")); (=_D1a1bPFiZi) and copy/pasting that into
pragma(mangle, "_D1a1bPFiZi") and voila, the loader can nothing but
trust that this is true.
Forging is maybe too hard a word but my vocabulary lacks a better one.
My original point is that forging is possible without using a dynamic
library. Anyone who has pragma(mangle)'d the symbol can "forge" anything
they want, it's not that hard.
Even without using pragma(mangle, it's possible):
mod1.d:
__gshared someGlobal;
extern(C) void realImpl() // no mangling
{
someGlobal = 5; // yay side effects!
*(cast(ubyte*)0xdeadbeef) = 5; // yay no safety!
throw new Exception("yay, I can throw!");
}
mod2.d:
extern(C) void realImpl() @safe pure nothrow;
void foo() @safe pure nothrow // mangled properly
{
realImpl();
}
So why worry about the dynamic library case? I would say if you obey the
mangling rules, the compiler should trust the mangling.
-Steve
Re: Is it possible to dynamically load a @safe function from a shared library ?
On Friday, 13 March 2020 at 20:31:16 UTC, Steven Schveighoffer
wrote:
On 3/13/20 4:22 PM, wjoe wrote:
I would expect that something could be written to turn a
signature string into a mangling and also provide the correct
type upon return. Something like:
auto f = getFunction!(@safe void
function(int))("package.module.symbol");
and have it properly mangle the expected function name and pull
it from the dynamic library.
-Steve
Thanks for your reply.
core.demangle: mangle; comes to mind. And in fact, because the
exports weren't extern(C), that's how I imported the symbols;
prior to reading H. S. Teoh's reply. That's when I realized that
I've got a problem.
Even though I know that the function being exported are all
compiled @safe, that doesn't mean I can count on the fact.
Since a plugin is a separate file it can be swapped out with a
version that exports all the same (forged) symbols names but with
a @system implementation.
Forging these symbol names is as easy as something like
mangle!(int function(int))("a.b")); (=_D1a1bPFiZi) and
copy/pasting that into pragma(mangle, "_D1a1bPFiZi") and voila,
the loader can nothing but trust that this is true.
Forging is maybe too hard a word but my vocabulary lacks a better
one.
Re: Is it possible to dynamically load a @safe function from a shared library ?
On Fri, Mar 13, 2020 at 04:31:16PM -0400, Steven Schveighoffer via
Digitalmars-d-learn wrote:
[...]
> I would expect that something could be written to turn a signature
> string into a mangling and also provide the correct type upon return.
> Something like:
>
> auto f = getFunction!(@safe void function(int))("package.module.symbol");
>
> and have it properly mangle the expected function name and pull it
> from the dynamic library.
[...]
This would still have to be @trusted, of course, since there's no
telling what's actually inside the object file.
But this sort of facility totally should be in Phobos, or at least in
some dub package somewhere. It will make working with dynamically
loaded libraries in D so much more convenient.
T
--
If you think you are too small to make a difference, try sleeping in a closed
room with a mosquito. -- Jan van Steenbergen
Re: Is it possible to dynamically load a @safe function from a shared library ?
On Fri, Mar 13, 2020 at 08:22:53PM +, wjoe via Digitalmars-d-learn wrote: [...] > So from what I understand, because, at least on Posix, since there's > only a symbol name there's nothing I can do in my loader to verify > that a function is or does what it claim to be/do. [...] As far as I know, it's the same thing on Windows PE format. Basically, once your code is compiled down to the object file level, there really isn't anything at the level of higher programming language concepts anymore, like type signatures and stuff. As far as object files are concerned, it's just a bunch of opaque binary data with string labels tacked on them, plus some extra information like relocation data and other such OS-level concepts. What's in that opaque binary data isn't really the purview of the object file format; things like types and function signatures are an interpretation laid upon the data by higher-level application code. The OS doesn't know what it is, and doesn't care (nor should it, that's not its job). So given some arbitrary object file, there's really no real guarantee as to what the contents are inside. Anybody can craft an object file that exports symbol names that look like the symbols generated by some higher-level programming language, but the actual binary data the names point to may do something completely different. In fact, this is exactly why pragma(mangle) is so useful: Adam Ruppe's jni.d, for example, makes extensive use of this in order to make the D code inside the object file appear like Java JNI symbols to the JVM. D's C++ interop is also based on the same concept: export symbols that look like C++ mangled symbols, but inside is actually D code, not C++ code. Far from being "disappointing", I think this stuff is very powerful, and lots of fun if you do it right. But it does come with the caveat that you're essentially meddling around under the hood, so it's your responsibility not to do something that will cause the engine to blow up. Hence the @trusted tag. T -- Never step over a puddle, always step around it. Chances are that whatever made it is still dripping.
Re: Is it possible to dynamically load a @safe function from a shared library ?
On 3/13/20 4:22 PM, wjoe wrote:
I wasn't aware that pragma(mangle, ..) can practically name any function
anything. So from what I understand, because, at least on Posix, since
there's only a symbol name there's nothing I can do in my loader to
verify that a function is or does what it claim to be/do.
But that's the same as the linker anyway. It's possible to stub the
function as @safe, but implement it as @system and just pragma(mangle)
the thing. Therefore the compiler is no safer.
I would expect that something could be written to turn a signature
string into a mangling and also provide the correct type upon return.
Something like:
auto f = getFunction!(@safe void function(int))("package.module.symbol");
and have it properly mangle the expected function name and pull it from
the dynamic library.
-Steve
Re: Is it possible to dynamically load a @safe function from a shared library ?
On Friday, 13 March 2020 at 18:30:51 UTC, H. S. Teoh wrote: On Fri, Mar 13, 2020 at 06:11:01PM +, wjoe via Digitalmars-d-learn wrote: On Friday, 13 March 2020 at 17:05:32 UTC, Mike Parker wrote: > On Friday, 13 March 2020 at 16:11:53 UTC, wjoe wrote: > > On Friday, 13 March 2020 at 16:04:06 UTC, Mike Parker > > wrote: [...] > > > bindSymbol(cast(void**), "VersionOfAPI"); [...] This also means that LoadPlugin() can't be @safe - or at least the call to bindSymbol. [...] Of course it cannot be @safe, because it depends on whether the symbol defined in the library you loaded is actually @safe. You cannot know that for sure (for example, maybe it exports a symbol that happens to coincide with the mangling of a @safe function, but isn't in fact @safe). Similarly, at least on Posix, shared libraries only export symbol names, the actual type is not part of the shared library API other than what is encoded in the mangled symbol. So you don't know for sure that you're actually casting to the correct type, for example; if you make a mistake, you might get UB and memory corruption. So essentially, you're trusting that the symbol you just looked up is actually pointing to what you think it's pointing to. Therefore, it makes sense that such calls have to be @trusted. T I wasn't aware that pragma(mangle, ..) can practically name any function anything. So from what I understand, because, at least on Posix, since there's only a symbol name there's nothing I can do in my loader to verify that a function is or does what it claim to be/do. This is kind of disappointing but well worth the lessons learned. Thanks for your reply.
Re: Is it possible to dynamically load a @safe function from a shared library ?
On Fri, Mar 13, 2020 at 06:11:01PM +, wjoe via Digitalmars-d-learn wrote: > On Friday, 13 March 2020 at 17:05:32 UTC, Mike Parker wrote: > > On Friday, 13 March 2020 at 16:11:53 UTC, wjoe wrote: > > > On Friday, 13 March 2020 at 16:04:06 UTC, Mike Parker wrote: [...] > > > > bindSymbol(cast(void**), "VersionOfAPI"); [...] > This also means that LoadPlugin() can't be @safe - or at least the > call to bindSymbol. [...] Of course it cannot be @safe, because it depends on whether the symbol defined in the library you loaded is actually @safe. You cannot know that for sure (for example, maybe it exports a symbol that happens to coincide with the mangling of a @safe function, but isn't in fact @safe). Similarly, at least on Posix, shared libraries only export symbol names, the actual type is not part of the shared library API other than what is encoded in the mangled symbol. So you don't know for sure that you're actually casting to the correct type, for example; if you make a mistake, you might get UB and memory corruption. So essentially, you're trusting that the symbol you just looked up is actually pointing to what you think it's pointing to. Therefore, it makes sense that such calls have to be @trusted. T -- If it breaks, you get to keep both pieces. -- Software disclaimer notice
Re: Is it possible to dynamically load a @safe function from a shared library ?
On Friday, 13 March 2020 at 17:05:32 UTC, Mike Parker wrote: On Friday, 13 March 2020 at 16:11:53 UTC, wjoe wrote: On Friday, 13 March 2020 at 16:04:06 UTC, Mike Parker wrote: On Friday, 13 March 2020 at 15:16:06 UTC, wjoe wrote: bindSymbol(, "VersionOfAPI"); } Is it possible to convince the compiler to look the other way while binding @safe functions from the plugin ? It probably has nothing to do with @safe, but is because of the void**. bindSymbol(cast(void**), "VersionOfAPI"); Than works, thanks :) But isn't apiVersion a function pointer ? Yes, but when you take the address of *any* kind of pointer, you can't assign it to void** without a cast. Didn't know that. I'm curious as to why this is the case. This also means that LoadPlugin() can't be @safe - or at least the call to bindSymbol. Thanks for the reply.
Re: Is it possible to dynamically load a @safe function from a shared library ?
On Friday, 13 March 2020 at 16:11:53 UTC, wjoe wrote: On Friday, 13 March 2020 at 16:04:06 UTC, Mike Parker wrote: On Friday, 13 March 2020 at 15:16:06 UTC, wjoe wrote: bindSymbol(, "VersionOfAPI"); } Is it possible to convince the compiler to look the other way while binding @safe functions from the plugin ? It probably has nothing to do with @safe, but is because of the void**. bindSymbol(cast(void**), "VersionOfAPI"); Than works, thanks :) But isn't apiVersion a function pointer ? Yes, but when you take the address of *any* kind of pointer, you can't assign it to void** without a cast.
Re: Is it possible to dynamically load a @safe function from a shared library ?
On Friday, 13 March 2020 at 16:04:06 UTC, Mike Parker wrote: On Friday, 13 March 2020 at 15:16:06 UTC, wjoe wrote: bindSymbol(, "VersionOfAPI"); } Is it possible to convince the compiler to look the other way while binding @safe functions from the plugin ? It probably has nothing to do with @safe, but is because of the void**. bindSymbol(cast(void**), "VersionOfAPI"); Than works, thanks :) But isn't apiVersion a function pointer ?
Re: Is it possible to dynamically load a @safe function from a shared library ?
On Friday, 13 March 2020 at 15:16:06 UTC, wjoe wrote: bindSymbol(, "VersionOfAPI"); } Is it possible to convince the compiler to look the other way while binding @safe functions from the plugin ? It probably has nothing to do with @safe, but is because of the void**. bindSymbol(cast(void**), "VersionOfAPI");
Is it possible to dynamically load a @safe function from a shared library ?
I've got a plug-in which is a shared library. Like this
module plugin;
@safe int VersionOfAPI()
{
return 1;
}
this is builds to plugin.so
in main.d I'm loading the plugin and bind the those functions
like so:
module app;
@safe:
alias apiverfn = int function();
apiverfn apiVersion;
void main()
{
LoadPlugin("plugin.so");
}
void LoadPlugin(string path)
{
void* plugin = loadLibrary(path);
@trusted void* bindSymbol(void** pfn, const(char)* symbolName)
{
*pfn = dlsym(plugin, symbolName);
}
bindSymbol(, "VersionOfAPI");
}
The compiler now refuses to call bindSymbol:
Error: function bindSymbol(void**, const(char)*) is not
callable using argument types (int function() @safe*, string)
cannot pass of type int function() @safe* to
parameter void** pfn
It makes sense that the compiler refuses to assign a @system
function* to a @safe one but dlsym is a @system function which
returns a @system fn* and I know that the function is @safe.
Is it possible to convince the compiler to look the other way
while binding @safe functions from the plugin ?
