Re: Merge pull request #593 from dawgfoto/dynamicLoading
Most awesome
Merge pull request #593 from dawgfoto/dynamicLoading
We just merged the low-level implementation for full shared library support. There should be an article or some tutorial but I simply lack the time for that. So if you want to experiment with it you should read this and have a look at the test cases (https://github.com/D-Programming-Language/druntime/tree/master/test/shared). I'd be thankful for every bug report, please add the keyword dll (http://d.puremagic.com/issues/buglist.cgi?keywords=dllresolution=---). Just a repost of the github comment (https://github.com/D-Programming-Language/druntime/pull/593#issuecomment-23913789). I updated and finished the low-level implementation. This now supports almost all shared library scenarios merely automatically. It is also tested pretty thoroughly. Here are the main gotchas for experimenting. There is an extra layer of thread local reference accounting over the global one in the runtime linker which has some implications on how to use shared libraries. Loading libraries should only be done through Runtime.loadLibrary/Runtime.unloadLibrary (or rt_loadLibrary/rt_unloadLibrary for C), so that the thread local accounting doesn't miss anything. The recommended pattern to load a D library from a pure C program is to dlopen phobos/druntime first and to use rt_loadLibrary/rt_unloadLibrary hereafter. Loading a library only initializes it and it's linked dependencies for the current thread. Threads where a library is not initialize will not see the library's modules, nor the EH tables and the GC will miss to scan the library's TLS memory. A spawned thread (core.thread.Thread) will initialize the same libraries as it's parent thread. There is an implicit dlopen call for every dynamically loaded library so to prevent the parent thread from unloading. When a thread finishes it will drop/unload all remaining references to loaded libraries. The high-level implementation of the shared libraries support will provide a method to iterate over all loaded libraries, thus it will be possible to explicitly unload unused inherited libraries. Finding the dependencies of a library is somewhat expensive so it is only performed for D libraries and non-recursively. Because all dependencies are loaded before the library itself we do know the recursive dependencies. In case of loading a C library that depends on D libraries or for D-C-D dependency chains we will miss the recursive dependency. This might possibly lead to uninitialized threads when loading such a library in more than one thread. I will try to fix any arising bugs ASAP but I don't have much time left currently. Please tag bugzilla entries with the dll keyword. Work on the high-level interface will start around the end of next week.
Re: Merge pull request #593 from dawgfoto/dynamicLoading
Am 07.09.2013 19:09, schrieb Martin Nowak: We just merged the low-level implementation for full shared library support. There should be an article or some tutorial but I simply lack the time for that. So if you want to experiment with it you should read this and have a look at the test cases (https://github.com/D-Programming-Language/druntime/tree/master/test/shared). I'd be thankful for every bug report, please add the keyword dll (http://d.puremagic.com/issues/buglist.cgi?keywords=dllresolution=---). Just a repost of the github comment (https://github.com/D-Programming-Language/druntime/pull/593#issuecomment-23913789). I updated and finished the low-level implementation. This now supports almost all shared library scenarios merely automatically. It is also tested pretty thoroughly. Here are the main gotchas for experimenting. There is an extra layer of thread local reference accounting over the global one in the runtime linker which has some implications on how to use shared libraries. Loading libraries should only be done through Runtime.loadLibrary/Runtime.unloadLibrary (or rt_loadLibrary/rt_unloadLibrary for C), so that the thread local accounting doesn't miss anything. The recommended pattern to load a D library from a pure C program is to dlopen phobos/druntime first and to use rt_loadLibrary/rt_unloadLibrary hereafter. Loading a library only initializes it and it's linked dependencies for the current thread. Threads where a library is not initialize will not see the library's modules, nor the EH tables and the GC will miss to scan the library's TLS memory. A spawned thread (core.thread.Thread) will initialize the same libraries as it's parent thread. There is an implicit dlopen call for every dynamically loaded library so to prevent the parent thread from unloading. When a thread finishes it will drop/unload all remaining references to loaded libraries. The high-level implementation of the shared libraries support will provide a method to iterate over all loaded libraries, thus it will be possible to explicitly unload unused inherited libraries. Finding the dependencies of a library is somewhat expensive so it is only performed for D libraries and non-recursively. Because all dependencies are loaded before the library itself we do know the recursive dependencies. In case of loading a C library that depends on D libraries or for D-C-D dependency chains we will miss the recursive dependency. This might possibly lead to uninitialized threads when loading such a library in more than one thread. I will try to fix any arising bugs ASAP but I don't have much time left currently. Please tag bugzilla entries with the dll keyword. Work on the high-level interface will start around the end of next week. Which plattforms does this work on?
Re: Merge pull request #593 from dawgfoto/dynamicLoading
On 09/07/2013 07:15 PM, Benjamin Thaut wrote: Which plattforms does this work on? Linux only for now
Re: Merge pull request #593 from dawgfoto/dynamicLoading
On Saturday, 7 September 2013 at 17:09:17 UTC, Martin Nowak wrote: We just merged the low-level implementation for full shared library support. There should be an article or some tutorial but I simply lack the time for that. So if you want to experiment with it you should read this and have a look at the test cases (https://github.com/D-Programming-Language/druntime/tree/master/test/shared). I'd be thankful for every bug report, please add the keyword dll (http://d.puremagic.com/issues/buglist.cgi?keywords=dllresolution=---). Just a repost of the github comment (https://github.com/D-Programming-Language/druntime/pull/593#issuecomment-23913789). I updated and finished the low-level implementation. This now supports almost all shared library scenarios merely automatically. It is also tested pretty thoroughly. Here are the main gotchas for experimenting. There is an extra layer of thread local reference accounting over the global one in the runtime linker which has some implications on how to use shared libraries. Loading libraries should only be done through Runtime.loadLibrary/Runtime.unloadLibrary (or rt_loadLibrary/rt_unloadLibrary for C), so that the thread local accounting doesn't miss anything. The recommended pattern to load a D library from a pure C program is to dlopen phobos/druntime first and to use rt_loadLibrary/rt_unloadLibrary hereafter. Loading a library only initializes it and it's linked dependencies for the current thread. Threads where a library is not initialize will not see the library's modules, nor the EH tables and the GC will miss to scan the library's TLS memory. A spawned thread (core.thread.Thread) will initialize the same libraries as it's parent thread. There is an implicit dlopen call for every dynamically loaded library so to prevent the parent thread from unloading. When a thread finishes it will drop/unload all remaining references to loaded libraries. The high-level implementation of the shared libraries support will provide a method to iterate over all loaded libraries, thus it will be possible to explicitly unload unused inherited libraries. Finding the dependencies of a library is somewhat expensive so it is only performed for D libraries and non-recursively. Because all dependencies are loaded before the library itself we do know the recursive dependencies. In case of loading a C library that depends on D libraries or for D-C-D dependency chains we will miss the recursive dependency. This might possibly lead to uninitialized threads when loading such a library in more than one thread. I will try to fix any arising bugs ASAP but I don't have much time left currently. Please tag bugzilla entries with the dll keyword. Work on the high-level interface will start around the end of next week. Good job, Congrats!!
