Re: Finding class template instantiations via runtime reflection (for openmethods)
On 9/21/17 4:32 PM, Jean-Louis Leroy wrote: It did not take long! Someone tried to create templatized open methods and it didn't work right of the box. I expected that, but in fact there may be a bit of hope. You cannot have virtual function templates in C++ or in D because the layout of the vtables have to be known at compile time - but openmethods creates its method tables at runtime so maybe it can be made to work. I stumbled upon a problem very quickly: it seems that classes that come from class template instantiations are not registered in ModuleInfo - see below, import std.stdio; class Foo {} class Bar(T) : Foo { int i; } alias BarInt = Bar!int; const barInt = new BarInt; void main() { foreach (mod; ModuleInfo) { foreach (c; mod.localClasses) { writeln(c); } } } output: modtemp.Foo core.exception.RangeError core.exception.AssertError etc... Neither 'Bar!int' nor 'BarInt' appear in 'localClasses'. Ideas? It used to be that classes were always in ModuleInfo (and that ModuleInfo was always generated). That's not so much the case any more, as there has been a push to reduce the runtime footprint and complexity (especially toward -betterC). This makes traditional runtime reflection more difficult and sporadic. However, I would expect that if ModuleInfo is generated for a file, and some classes are added, then ALL classes instantiated in the module should be added. I would say it's a bug. -Steve
Re: Finding class template instantiations via runtime reflection (for openmethods)
On 09/22/2017 05:16 AM, Steven Schveighoffer wrote: > On 9/21/17 4:32 PM, Jean-Louis Leroy wrote: >> it seems that classes that >> come from class template instantiations are not registered in >> ModuleInfo > I would say it's a bug. > > -Steve I think so too. I wanted to hear it from Steve first. :) Ali
Re: Finding class template instantiations via runtime reflection (for openmethods)
On Thursday, 21 September 2017 at 20:32:38 UTC, Jean-Louis Leroy wrote: Neither 'Bar!int' nor 'BarInt' appear in 'localClasses'. Ideas? The information you can retrieve through localClasses is limited. AFAIK it only retrieves plain non-template classes declared directly in the module. Last I checked, non-template classes added via mixin were skipped as well. I'm not sure how much work can be expected in this area. Things related to runtime reflection have slowly been getting removed from druntime and dmd. I don't think there's much left of it. Your best bet is to scrape the info you need on your own using D's traits: https://dlang.org/phobos/std_traits.html https://dlang.org/spec/traits.html I would recommend trying to work with std.traits first (as opposed to __traits). If you need to scan entire modules at a time, then start with something like this: foreach(m; __traits(allMembers, fully.qualified.modulename)) { // __traits(getMember, ... , ...) }
Re: Finding class template instantiations via runtime reflection (for openmethods)
On Thursday, 21 September 2017 at 20:32:38 UTC, Jean-Louis Leroy wrote: Neither 'Bar!int' nor 'BarInt' appear in 'localClasses'. Ideas? The information you can retrieve through localClasses is limited. AFAIK it only retrieves plain non-template classes declared directly in the module. Last I checked, non-template classes added via mixin were skipped as well. I'm not sure how much work can be expected in this area. Things related to runtime reflection have slowly been getting removed from druntime and dmd. I don't think there's much left of it. Your best bet is to scrape the info you need on your own using D's traits: https://dlang.org/phobos/std_traits.html https://dlang.org/spec/traits.html I would recommend trying to work with std.traits first (as opposed to __traits). If you need to scan entire modules at a time, then start with something like this: foreach(m; __traits(allMembers, fully.qualified.modulename)) { // __traits(getMember, ... , ...) }
Re: Finding class template instantiations via runtime reflection (for openmethods)
On Thursday, 21 September 2017 at 20:32:38 UTC, Jean-Louis Leroy wrote: Neither 'Bar!int' nor 'BarInt' appear in 'localClasses'. Ideas? The information you can retrieve through localClasses is limited. AFAIK it only retrieves plain non-template classes declared directly in the module. Last I checked, non-template classes added via mixin were skipped as well. I'm not sure how much work can be expected in this area. Things related to runtime reflection have slowly been getting removed from druntime and dmd. I don't think there's much left of it. Your best bet is to scrape the info you need on your own using D's traits: https://dlang.org/phobos/std_traits.html https://dlang.org/spec/traits.html I would recommend trying to work with std.traits first (as opposed to __traits). If you need to scan entire modules at a time, then start with something like this: foreach(m; __traits(allMembers, fully.qualified.modulename)) { // __traits(getMember, ... , ...) }
Re: Finding class template instantiations via runtime reflection (for openmethods)
On Friday, 22 September 2017 at 12:16:52 UTC, Steven Schveighoffer wrote: On 9/21/17 4:32 PM, Jean-Louis Leroy wrote: It did not take long! Someone tried to create templatized open methods and it didn't work right of the box. I expected that, but in fact there may be a bit of hope. You cannot have virtual function templates in C++ or in D because the layout of the vtables have to be known at compile time - but openmethods creates its method tables at runtime so maybe it can be made to work. I stumbled upon a problem very quickly: it seems that classes that come from class template instantiations are not registered in ModuleInfo - see below, import std.stdio; class Foo {} class Bar(T) : Foo { int i; } alias BarInt = Bar!int; const barInt = new BarInt; void main() { foreach (mod; ModuleInfo) { foreach (c; mod.localClasses) { writeln(c); } } } output: modtemp.Foo core.exception.RangeError core.exception.AssertError etc... Neither 'Bar!int' nor 'BarInt' appear in 'localClasses'. Ideas? It used to be that classes were always in ModuleInfo (and that ModuleInfo was always generated). That's not so much the case any more, as there has been a push to reduce the runtime footprint and complexity (especially toward -betterC). This makes traditional runtime reflection more difficult and sporadic. However, I would expect that if ModuleInfo is generated for a file, and some classes are added, then ALL classes instantiated in the module should be added. I would say it's a bug. -Steve aliases are not symbol so it's expected that 'BarInt' doesn't appear in the items. Would "Bar!int" be usable in Object.factory ?
Re: Finding class template instantiations via runtime reflection (for openmethods)
On 9/24/17 12:30 AM, user1234 wrote: On Friday, 22 September 2017 at 12:16:52 UTC, Steven Schveighoffer wrote: It used to be that classes were always in ModuleInfo (and that ModuleInfo was always generated). That's not so much the case any more, as there has been a push to reduce the runtime footprint and complexity (especially toward -betterC). This makes traditional runtime reflection more difficult and sporadic. However, I would expect that if ModuleInfo is generated for a file, and some classes are added, then ALL classes instantiated in the module should be added. I would say it's a bug. aliases are not symbol so it's expected that 'BarInt' doesn't appear in the items. Correct, even with a fixed compiler, BarInt would not appear. Would "Bar!int" be usable in Object.factory ? I'm not sure what the name of the class might be, as this capability depends completely on the compiler. As it stands, no entry for the class type is made in the ModuleInfo, so there's no way to know what the "right name" is -- someone needs to add the capability, and that may be when the decision is made. Note to readers, Object.factory uses exactly the mechanism Jean-Louis has used, so if his code can't find it, neither can Object.factory. -Steve
Re: Finding class template instantiations via runtime reflection (for openmethods)
On Sunday, 24 September 2017 at 04:30:26 UTC, user1234 wrote: aliases are not symbol so it's expected that 'BarInt' doesn't appear in the items. I didn't really expect that either.
Re: Finding class template instantiations via runtime reflection (for openmethods)
Thank you all for your input. I will file a bug report.
Re: Finding class template instantiations via runtime reflection (for openmethods)
On Monday, 25 September 2017 at 13:45:05 UTC, Jean-Louis Leroy wrote: Thank you all for your input. I will file a bug report. Just reopen the old one: https://issues.dlang.org/show_bug.cgi?id=2484 -- Biotronic
Re: Finding class template instantiations via runtime reflection (for openmethods)
On Monday, 25 September 2017 at 13:57:55 UTC, Biotronic wrote: On Monday, 25 September 2017 at 13:45:05 UTC, Jean-Louis Leroy wrote: Thank you all for your input. I will file a bug report. Just reopen the old one: https://issues.dlang.org/show_bug.cgi?id=2484 -- Biotronic Done, thanks.
Re: Finding class template instantiations via runtime reflection (for openmethods)
On 2017-09-21 22:32, Jean-Louis Leroy wrote: It did not take long! Someone tried to create templatized open methods and it didn't work right of the box. I expected that, but in fact there may be a bit of hope. You cannot have virtual function templates in C++ or in D because the layout of the vtables have to be known at compile time - but openmethods creates its method tables at runtime so maybe it can be made to work. I stumbled upon a problem very quickly: it seems that classes that come from class template instantiations are not registered in ModuleInfo - see below, Ideas? Not sure exactly what you need get access to. But you could try to inspect the symbol table at runtime, if what you need is available there. Of course, that's a bit of a PITA, but you would not be dependent on any bugs getting fixed or new features/improvements implemented. -- /Jacob Carlborg
Re: Finding class template instantiations via runtime reflection (for openmethods)
On Monday, 25 September 2017 at 14:17:13 UTC, Jacob Carlborg wrote: Of course, that's a bit of a PITA, but you would not be dependent on any bugs getting fixed or new features/improvements implemented. How do you do that?
Re: Finding class template instantiations via runtime reflection (for openmethods)
On Monday, 25 September 2017 at 14:06:55 UTC, Jean-Louis Leroy wrote: Done, thanks. You might add a link to this thread.
Re: Finding class template instantiations via runtime reflection (for openmethods)
On Thursday, 21 September 2017 at 20:32:38 UTC, Jean-Louis Leroy wrote: It did not take long! Someone tried to create templatized open methods and it didn't work right of the box. I expected that, but in fact there may be a bit of hope. You cannot have virtual function templates in C++ or in D because the layout of the vtables have to be known at compile time - but openmethods creates its method tables at runtime so maybe it can be made to work. I stumbled upon a problem very quickly: it seems that classes that come from class template instantiations are not registered in ModuleInfo - see below, [...] Neither 'Bar!int' nor 'BarInt' appear in 'localClasses'. Ideas? Yeah. You can setup a custom registry that maps names to their TypeInfo_Class. I do something similar in iz (though the real point in iz is a GC-free factory but the principle of the registry would be the the same for you) Example: --- /+ dub.sdl: name "dub_script" dependency "iz" version="0.6.0" +/ module dub_script; import iz.memory, std.stdio; class Foo(T){this(){writeln("fooDeInt");}} TypeInfo_Class[string] registry; // you need that... static this() { registerFactoryClass!(Foo!int)(registry); // ...and that, maybe with another name } void main() { auto fooDeInt = iz.memory.factory(registry, "Foo!int"); destruct(cast(Object) fooDeInt); } ---
Re: Finding class template instantiations via runtime reflection (for openmethods)
On 2017-09-25 16:24, Jean-Louis Leroy wrote: On Monday, 25 September 2017 at 14:17:13 UTC, Jacob Carlborg wrote: Of course, that's a bit of a PITA, but you would not be dependent on any bugs getting fixed or new features/improvements implemented. How do you do that? On Posix you use "dlopen", passing in null as the filename to get a handle to the current executable. Then you iterate the sections until you find the symbol table. For macOS there's this reference for the Mach-O binary format [1]. For Mach-O files there's this helper function as well [3]. [1] https://web.archive.org/web/20090901205800/http://developer.apple.com/mac/library/documentation/DeveloperTools/Conceptual/MachORuntime/Reference/reference.html [2] getsectbynamefromheader_64 -- /Jacob Carlborg
Re: Finding class template instantiations via runtime reflection (for openmethods)
On Tuesday, 26 September 2017 at 15:59:01 UTC, Jacob Carlborg wrote: On 2017-09-25 16:24, Jean-Louis Leroy wrote: On Monday, 25 September 2017 at 14:17:13 UTC, Jacob Carlborg wrote: Of course, that's a bit of a PITA, but you would not be dependent on any bugs getting fixed or new features/improvements implemented. How do you do that? On Posix you use "dlopen", passing in null as the filename to get a handle to the current executable. Then you iterate the sections until you find the symbol table. For macOS there's this reference for the Mach-O binary format [1]. For Mach-O files there's this helper function as well [3]. [1] https://web.archive.org/web/20090901205800/http://developer.apple.com/mac/library/documentation/DeveloperTools/Conceptual/MachORuntime/Reference/reference.html [2] getsectbynamefromheader_64 Ah, I suspected that. I don't want to go down that path, what of Windows? But thanks anyway...
Re: Finding class template instantiations via runtime reflection (for openmethods)
On Tuesday, 26 September 2017 at 04:55:46 UTC, b4s1L3 wrote: On Thursday, 21 September 2017 at 20:32:38 UTC, Jean-Louis Leroy wrote: [...] Yeah. You can setup a custom registry that maps names to their TypeInfo_Class. I do something similar in iz (though the real point in iz is a GC-free factory but the principle of the registry would be the the same for you) Example: --- /+ dub.sdl: name "dub_script" dependency "iz" version="0.6.0" +/ module dub_script; import iz.memory, std.stdio; class Foo(T){this(){writeln("fooDeInt");}} TypeInfo_Class[string] registry; // you need that... static this() { registerFactoryClass!(Foo!int)(registry); // ...and that, maybe with another name } void main() { auto fooDeInt = iz.memory.factory(registry, "Foo!int"); destruct(cast(Object) fooDeInt); } --- Yeah that's what I implemented yesterday (not pushed yet). I have a prototype of a solution for using open methods and templates together - although the syntax is not light: it requires guidance from the user (a la C++ explicit template instantiation).
Re: Finding class template instantiations via runtime reflection (for openmethods)
On 2017-09-26 18:08, Jean-Louis Leroy wrote: Ah, I suspected that. I don't want to go down that path, what of Windows? But thanks anyway... It's possible to do the same on Windows (using LoadLibrary instead of dlopen). You would need to have separate code for each binary format. On Windows there are two, OMF used by DigitalMars and COFF used by Microsoft. DMD can output both. Linux and FreeBSD use ELF. -- /Jacob Carlborg