Re: Runtime introspection, or how to get class members at runtime Fin D
On Friday, 8 June 2018 at 08:21:39 UTC, evilrat wrote: On Friday, 8 June 2018 at 08:06:27 UTC, Arafel wrote: On Thursday, 7 June 2018 at 13:07:21 UTC, evilrat wrote: I don't think so. It clearly states that children must mixin too, which can mean it just grabs symbols in scope only, and base class has no way of knowing about its subclasses. It also has "agressive mode" that will make metadata for all public symbols(?) it can walk, this may or may not be helpful depending on your requirements. Yes, that's what I understood from looking at it, but perhaps I was just missing something. I wonder though how the "agressive mode" would work with separate compilation / dlopen'ed libraries. Perhaps I should give it a try and see what happens. Besides there is no way(not that I am aware of) to make self registering stuff happen, you still need to call it somewhere. The most transparent option is probably just doing a mixin in each module that performs registration of all module symbols in module ctor. The point is that there is absolute requirement to make explicit call for that, be it a module ctor mixin, class mixin or even user provided registration both at compile time or run time. But since it is MIT licensed you can probably use the code as the starting point and adjust to your own needs. BTW plug-ins is something that is right now possible on Linux(not sure about support on other *NIX systems), but in a very primitive form on Windows. This is related to DLL support issues(such as type information not being passed across process/DLL boundaries), these issues also may include runtime issues as well such as inability to delegate the GC, which will mean there will be 2(or more) concurrent running GC's. But again I am not aware of the current situation. Well, I'm already tightly coupled to linux, so this is not a big concern for me :-) I'll keep trying, as I said, my intention was to let plugin writers do it as easily as possible, but well, adding some kind of "register" function might be necessary in the end... A. Yep. Like I said probably the easiest to use way is to place single call in each module. And there probably no other solution, because modules creates sort of isolated graph via imports. And I am not aware of any way to get list of modules passed in with compiler invocation to perform some sort of centralized one-liner registration. But anyway look at this, might give some tips on how it can be done mixin https://github.com/Circular-Studios/Dash/blob/b7d589ad4ca8993445c136b6a4ae170932bb7962/source/dash/components/component.d#L208 (note that it uses static this() - module constructor. I think this behavior was changed around 2015-2016 and now it will cause cyclic dependency errors when modules with ctors import each other) usage https://github.com/Circular-Studios/Dash/blob/b7d589ad4ca8993445c136b6a4ae170932bb7962/source/dash/components/lights.d#L12 Thanks very much for these links! I'm currently also trying to get a crack at runtime introspection for enabling richer serialization capabilities. It is nice to have compile time code generation, but it really sucks when dealing with object hierarchies and API interfaces. I'm doing kind of the same thing as witchcraft with explicit mixins (putting a "mixin reflect" into every stuff I want to reflect on.) But I'd like to have selective reflection/introspection, with a C#-esque flavor of having a "centralised repository" of reflected stuff. Also I need to inject static this(). A serious drawback. On that note, you can pass: --DRT-oncycle=ignore to your compiled app to instruct the runtime to ignore cycle warnings. linux ex.: "./app --DRT-oncycle=ignore" It is ugly as hell to disable this check, but I would accept it gladly if this would be the only impediment of getting runtime reflection. Sadly it is not, and I don't want to ramble right now :)
Re: Runtime introspection, or how to get class members at runtime Fin D
On Friday, 8 June 2018 at 08:06:27 UTC, Arafel wrote: On Thursday, 7 June 2018 at 13:07:21 UTC, evilrat wrote: I don't think so. It clearly states that children must mixin too, which can mean it just grabs symbols in scope only, and base class has no way of knowing about its subclasses. It also has "agressive mode" that will make metadata for all public symbols(?) it can walk, this may or may not be helpful depending on your requirements. Yes, that's what I understood from looking at it, but perhaps I was just missing something. I wonder though how the "agressive mode" would work with separate compilation / dlopen'ed libraries. Perhaps I should give it a try and see what happens. Besides there is no way(not that I am aware of) to make self registering stuff happen, you still need to call it somewhere. The most transparent option is probably just doing a mixin in each module that performs registration of all module symbols in module ctor. The point is that there is absolute requirement to make explicit call for that, be it a module ctor mixin, class mixin or even user provided registration both at compile time or run time. But since it is MIT licensed you can probably use the code as the starting point and adjust to your own needs. BTW plug-ins is something that is right now possible on Linux(not sure about support on other *NIX systems), but in a very primitive form on Windows. This is related to DLL support issues(such as type information not being passed across process/DLL boundaries), these issues also may include runtime issues as well such as inability to delegate the GC, which will mean there will be 2(or more) concurrent running GC's. But again I am not aware of the current situation. Well, I'm already tightly coupled to linux, so this is not a big concern for me :-) I'll keep trying, as I said, my intention was to let plugin writers do it as easily as possible, but well, adding some kind of "register" function might be necessary in the end... A. Yep. Like I said probably the easiest to use way is to place single call in each module. And there probably no other solution, because modules creates sort of isolated graph via imports. And I am not aware of any way to get list of modules passed in with compiler invocation to perform some sort of centralized one-liner registration. But anyway look at this, might give some tips on how it can be done mixin https://github.com/Circular-Studios/Dash/blob/b7d589ad4ca8993445c136b6a4ae170932bb7962/source/dash/components/component.d#L208 (note that it uses static this() - module constructor. I think this behavior was changed around 2015-2016 and now it will cause cyclic dependency errors when modules with ctors import each other) usage https://github.com/Circular-Studios/Dash/blob/b7d589ad4ca8993445c136b6a4ae170932bb7962/source/dash/components/lights.d#L12
Re: Runtime introspection, or how to get class members at runtime Fin D
On Thursday, 7 June 2018 at 13:07:21 UTC, evilrat wrote: I don't think so. It clearly states that children must mixin too, which can mean it just grabs symbols in scope only, and base class has no way of knowing about its subclasses. It also has "agressive mode" that will make metadata for all public symbols(?) it can walk, this may or may not be helpful depending on your requirements. Yes, that's what I understood from looking at it, but perhaps I was just missing something. I wonder though how the "agressive mode" would work with separate compilation / dlopen'ed libraries. Perhaps I should give it a try and see what happens. Besides there is no way(not that I am aware of) to make self registering stuff happen, you still need to call it somewhere. The most transparent option is probably just doing a mixin in each module that performs registration of all module symbols in module ctor. The point is that there is absolute requirement to make explicit call for that, be it a module ctor mixin, class mixin or even user provided registration both at compile time or run time. But since it is MIT licensed you can probably use the code as the starting point and adjust to your own needs. BTW plug-ins is something that is right now possible on Linux(not sure about support on other *NIX systems), but in a very primitive form on Windows. This is related to DLL support issues(such as type information not being passed across process/DLL boundaries), these issues also may include runtime issues as well such as inability to delegate the GC, which will mean there will be 2(or more) concurrent running GC's. But again I am not aware of the current situation. Well, I'm already tightly coupled to linux, so this is not a big concern for me :-) I'll keep trying, as I said, my intention was to let plugin writers do it as easily as possible, but well, adding some kind of "register" function might be necessary in the end... A.
Re: Runtime introspection, or how to get class members at runtime Fin D
On Thursday, 7 June 2018 at 12:32:26 UTC, Arafel wrote: Thanks for all the answers! Is it possible to register, say, a base class, and have all the subclasses then registered automatically? My idea would be to make it as transparent as possible for the plugin implementation, and also not having to depend on it. A. I don't think so. It clearly states that children must mixin too, which can mean it just grabs symbols in scope only, and base class has no way of knowing about its subclasses. It also has "agressive mode" that will make metadata for all public symbols(?) it can walk, this may or may not be helpful depending on your requirements. Besides there is no way(not that I am aware of) to make self registering stuff happen, you still need to call it somewhere. The most transparent option is probably just doing a mixin in each module that performs registration of all module symbols in module ctor. The point is that there is absolute requirement to make explicit call for that, be it a module ctor mixin, class mixin or even user provided registration both at compile time or run time. But since it is MIT licensed you can probably use the code as the starting point and adjust to your own needs. BTW plug-ins is something that is right now possible on Linux(not sure about support on other *NIX systems), but in a very primitive form on Windows. This is related to DLL support issues(such as type information not being passed across process/DLL boundaries), these issues also may include runtime issues as well such as inability to delegate the GC, which will mean there will be 2(or more) concurrent running GC's. But again I am not aware of the current situation.
Re: Runtime introspection, or how to get class members at runtime Fin D
Thanks for all the answers! Is it possible to register, say, a base class, and have all the subclasses then registered automatically? My idea would be to make it as transparent as possible for the plugin implementation, and also not having to depend on it. A. There is a library that creates reflection metadata for you. [1] It seems a bit outdated and has some not-that-obvious compilation errors(for example getting ctor and callling it with runtime known type, or some other non template stuff), but other than that seems to be working (note that I didn't thorougly tested it, but its unittests succeeds on DMD 2.080 for both Windows x86 mscoff & x64 ) [1] https://code.dlang.org/packages/witchcraft
Re: Runtime introspection, or how to get class members at runtime Fin D
On Wednesday, 6 June 2018 at 13:28:02 UTC, Arafel wrote: I know it might not be the most idiomatic D, but as somebody with mostly a Java background (with some C and just a bit of C++) it seems something really straightforward to me: myObject.getClass().getFields() [2]. Also, I know I could add some UDA or even crawl the modules and have this information generated automatically at compilation time and added to the type itself in a member, and I might even end up doing it, but honestly, I think it's something that the language should provide in a kind of easy / accessible way. Powerful as compile-time introspection is, I think runtime shouldn't be forgotten either :-) Thanks, A. [1]: https://dlang.org/library/object/type_info__class.html [2]: https://docs.oracle.com/javase/8/docs/api/java/lang/Class.html#getFields-- There is a library that creates reflection metadata for you. [1] It seems a bit outdated and has some not-that-obvious compilation errors(for example getting ctor and callling it with runtime known type, or some other non template stuff), but other than that seems to be working (note that I didn't thorougly tested it, but its unittests succeeds on DMD 2.080 for both Windows x86 mscoff & x64 ) [1] https://code.dlang.org/packages/witchcraft
Re: Runtime introspection, or how to get class members at runtime Fin D
On 2018-06-06 15:28, Arafel wrote: Hi, What is the state of runtime introspection in D, specifically for classes? Is there any way to get *at runtime* the (public or otherwise accessible) members of a class? I have had a look as TypeInfo_Class [1], but apparently I can only get a list of types and offsets... which would be almost good enough, if not because the names of the members are missing, or at least I haven't been able to find them. In this case, what I'm trying to do is to serialize / dump / print the contents of an object (class instance) without knowing its actual runtime type. Before somebody suggests compile time introspection, the "main" code where this routine lives only provides a base class, and it's up to dlopen'ed plugins to provide the actual implementation... so I'm sorry but no compile-time solution can possibly work. Also, having each derivative class provide their own dumping information is not practical, I'd rather have it automated. I know it might not be the most idiomatic D, but as somebody with mostly a Java background (with some C and just a bit of C++) it seems something really straightforward to me: myObject.getClass().getFields() [2]. Also, I know I could add some UDA or even crawl the modules and have this information generated automatically at compilation time and added to the type itself in a member, and I might even end up doing it, but honestly, I think it's something that the language should provide in a kind of easy / accessible way. Powerful as compile-time introspection is, I think runtime shouldn't be forgotten either :-) The simplest, in my opinion would be to for the subclasses to register themselves with the serializer. This is how Orange works to allow serializing through base class references [1]. The use compile time introspection on the subclass and serialize that as usual. [1] https://github.com/jacob-carlborg/orange/blob/master/tests/BaseClass.d#L73 -- /Jacob Carlborg
Re: Runtime introspection, or how to get class members at runtime Fin D
On 6/6/18 9:28 AM, Arafel wrote: Hi, What is the state of runtime introspection in D, specifically for classes? Is there any way to get *at runtime* the (public or otherwise accessible) members of a class? There is very little runtime reflection capability. The position has always been that you can *build* runtime reflection given compile-time reflection. object has rtInfo, as Adam says, which is built to be able to extend TypeInfo. But at the moment, it's not used for anything, and likely will stay that way for a while. Note, you CAN build runtime reflection using mixins, but other than rtInfo, there's no way to instrument modules you don't control. Even that's a high bar, since you would then have to have a modified druntime. -Steve
Re: Runtime introspection, or how to get class members at runtime Fin D
On 06/06/2018 03:52 PM, Adam D. Ruppe wrote:
It is possible to add it to the runtime library right now (there's a
thing called rtInfo in there made to hold it, it is just null right
now), just people fight over even a few bytes of added metadata. So if
it is added, it would surely be some opt-in thing that will require your
thing be recompiled anyway.
If I wanted to add it myself, would I need to create a personalised D
compiler and/or D Runtime? That would be probably way too much for me :)
Also, it would have to be distributed and used to create the plugins...
If you can recompile the library, you can add a parallel extended info
thing (MyReflectionInfo[TypeInfo] array perhaps, populated by a static
this() ctor created via compile time reflection) that gives what you need.
Yeah, I had some ideas in this regard, still I would like it to be
transparent from the plugin creator point of view, but I don't know if
it would be possible to contain everything in the base class... so far I
had though about a base class like this:
```
import std.traits;
import std.meta;
TypeInfo[string][TypeInfo_Class] RTInfo;
class Base {
this(this C)() {
if (typeid(C) in RTInfo)
return;
RTInfo[typeid(C)] = (TypeInfo[string]).init;
static foreach_reverse(Type; AliasSeq!(C, BaseClassesTuple!C)) {
static foreach(string field; FieldNameTuple!Type) {
RTInfo[typeid(Type)][field] =
typeid(typeof(__traits(getMember, Type, field)));
}
}
}
}
```
But I think children classes can bypass this constructor, so I guess
it's not so easy, will have to keep trying :-)
A templated static this would be cool, though:
class Base {
static this(this C)() {
// ...
}
}
Apparently it's not possible :-(
Re: Runtime introspection, or how to get class members at runtime Fin D
On Wednesday, 6 June 2018 at 13:44:39 UTC, Arafel wrote: Why not (genuine question)? There's even myObject.classinfo, and I can only assume that there's some reason why it's there... It holds just barely enough info for dynamic casting, GC, and other language implementation stuff. (and then Object.factory for some weird reason, which actually causes bloat for so little benefit) Well, thanks for the quick and succinct answer... I guess the question now would be how realistic it would be to propose such an addition to the language... Has it already been discussed? It is possible to add it to the runtime library right now (there's a thing called rtInfo in there made to hold it, it is just null right now), just people fight over even a few bytes of added metadata. So if it is added, it would surely be some opt-in thing that will require your thing be recompiled anyway. If you can recompile the library, you can add a parallel extended info thing (MyReflectionInfo[TypeInfo] array perhaps, populated by a static this() ctor created via compile time reflection) that gives what you need. But if you can't recompile the library, the field names are simply not there
Re: Runtime introspection, or how to get class members at runtime Fin D
On 07/06/2018 1:44 AM, Arafel wrote: On 06/06/2018 03:30 PM, rikki cattermole wrote: You don't want TypeInfo. Why not (genuine question)? There's even myObject.classinfo, and I can only assume that there's some reason why it's there... In this case, what I'm trying to do is to serialize / dump / print the contents of an object (class instance) without knowing its actual runtime type. Before somebody suggests compile time introspection, the "main" code where this routine lives only provides a base class, and it's up to dlopen'ed plugins to provide the actual implementation... so I'm sorry but no compile-time solution can possibly work. Also, having each derivative class provide their own dumping information is not practical, I'd rather have it automated. I know it might not be the most idiomatic D, but as somebody with mostly a Java background (with some C and just a bit of C++) it seems something really straightforward to me: myObject.getClass().getFields() [2]. Doesn't exist. Well, thanks for the quick and succinct answer... I guess the question now would be how realistic it would be to propose such an addition to the language... Has it already been discussed? (I tried searching the forum, but didn't find anything relevant) I know it's got a runtime penalty, but realistically speaking, spending some bytes for the field names in the TypeInfo of a class shouldn't be that much of a problem? It is not an easy task building a reflection API from scratch. I'm one of the many that have tried. There is also push back from those who consider it "bloat" and don't need it. You can't just extend TypeInfo, it was never designed for it.
Re: Runtime introspection, or how to get class members at runtime Fin D
On 06/06/2018 03:30 PM, rikki cattermole wrote: You don't want TypeInfo. Why not (genuine question)? There's even myObject.classinfo, and I can only assume that there's some reason why it's there... In this case, what I'm trying to do is to serialize / dump / print the contents of an object (class instance) without knowing its actual runtime type. Before somebody suggests compile time introspection, the "main" code where this routine lives only provides a base class, and it's up to dlopen'ed plugins to provide the actual implementation... so I'm sorry but no compile-time solution can possibly work. Also, having each derivative class provide their own dumping information is not practical, I'd rather have it automated. I know it might not be the most idiomatic D, but as somebody with mostly a Java background (with some C and just a bit of C++) it seems something really straightforward to me: myObject.getClass().getFields() [2]. Doesn't exist. Well, thanks for the quick and succinct answer... I guess the question now would be how realistic it would be to propose such an addition to the language... Has it already been discussed? (I tried searching the forum, but didn't find anything relevant) I know it's got a runtime penalty, but realistically speaking, spending some bytes for the field names in the TypeInfo of a class shouldn't be that much of a problem?
Re: Runtime introspection, or how to get class members at runtime Fin D
On 07/06/2018 1:28 AM, Arafel wrote: Hi, What is the state of runtime introspection in D, specifically for classes? Is there any way to get *at runtime* the (public or otherwise accessible) members of a class? No. I have had a look as TypeInfo_Class [1], but apparently I can only get a list of types and offsets... which would be almost good enough, if not because the names of the members are missing, or at least I haven't been able to find them. You don't want TypeInfo. In this case, what I'm trying to do is to serialize / dump / print the contents of an object (class instance) without knowing its actual runtime type. Before somebody suggests compile time introspection, the "main" code where this routine lives only provides a base class, and it's up to dlopen'ed plugins to provide the actual implementation... so I'm sorry but no compile-time solution can possibly work. Also, having each derivative class provide their own dumping information is not practical, I'd rather have it automated. I know it might not be the most idiomatic D, but as somebody with mostly a Java background (with some C and just a bit of C++) it seems something really straightforward to me: myObject.getClass().getFields() [2]. Doesn't exist.
