Re: Generically call a function on Variant's payload?
On 08/21/2018 03:27 PM, Steven Schveighoffer wrote: If you examine how the code for variant works, it's quite clever. It establishes a "handler" that is generated with full compile-time type info available when the value is *assigned*, but then cast to a function taking a void pointer and an operation. This way, the user of the variant doesn't have to know what is inside, just the handler does. The code that sets the payload is the one that establishes how to use it. The same type of pattern could be used to, for instance, provide all the functions that a range uses. But it's not something that can be tacked on to Variant. You'd have to write your own type, because you'd have to handle the different functions that you know are common between the types in question (e.g. the operations supported on a variant are here: https://github.com/dlang/phobos/blob/master/std/variant.d#L163) Oooh, that's really cool! I might try my hand at something like that. I bet it actually could be generalized to a certain extent. Variant would just have to be templated on the expected interface. Then it could compile-time loop over the list of functions expected, and introspect each of them enough to build the handler and the forwarders. Doesn't exactly sound fun to implement, but I'd bet it could be done.
Re: Generically call a function on Variant's payload?
On 8/21/18 3:11 PM, Nick Sabalausky (Abscissa) wrote: On 08/21/2018 03:03 AM, Nicholas Wilson wrote: On Monday, 20 August 2018 at 00:27:04 UTC, Nick Sabalausky (Abscissa) wrote: Suppose I've wrapped a Variant in a struct/class which ensures the Variant *only* ever contains types which satisfy a particular constraint (for example: isInputRange, or hasLength, etc...). Is there a way to call a function (ex: popFront) on the Variant, *without* knowing ahead of time all the possible static types it might might contain? I assume you mean at compile time AoT of development of your library? Yes, don't use Variant; use https://github.com/s-ludwig/taggedalgebraic TaggedAlgebraic requires a finite list of every type it can contain. That's the deal-breaker for what I had in mind. I need something less like Algebraic and more like Variant, which *doesn't* require a finite, canonical list of every type it can contain. The more I think about it though, the more I'm not so sure that the extensibility is important enough to warrant all of this, or...more importantly...that all of this is even necessary anyway for users to accomplish the use-cases I had in mind. I think I may go with an entirely different approach. If you examine how the code for variant works, it's quite clever. It establishes a "handler" that is generated with full compile-time type info available when the value is *assigned*, but then cast to a function taking a void pointer and an operation. This way, the user of the variant doesn't have to know what is inside, just the handler does. The code that sets the payload is the one that establishes how to use it. The same type of pattern could be used to, for instance, provide all the functions that a range uses. But it's not something that can be tacked on to Variant. You'd have to write your own type, because you'd have to handle the different functions that you know are common between the types in question (e.g. the operations supported on a variant are here: https://github.com/dlang/phobos/blob/master/std/variant.d#L163) -Steve
Re: Generically call a function on Variant's payload?
On 08/21/2018 03:03 AM, Nicholas Wilson wrote: On Monday, 20 August 2018 at 00:27:04 UTC, Nick Sabalausky (Abscissa) wrote: Suppose I've wrapped a Variant in a struct/class which ensures the Variant *only* ever contains types which satisfy a particular constraint (for example: isInputRange, or hasLength, etc...). Is there a way to call a function (ex: popFront) on the Variant, *without* knowing ahead of time all the possible static types it might might contain? I assume you mean at compile time AoT of development of your library? Yes, don't use Variant; use https://github.com/s-ludwig/taggedalgebraic TaggedAlgebraic requires a finite list of every type it can contain. That's the deal-breaker for what I had in mind. I need something less like Algebraic and more like Variant, which *doesn't* require a finite, canonical list of every type it can contain. The more I think about it though, the more I'm not so sure that the extensibility is important enough to warrant all of this, or...more importantly...that all of this is even necessary anyway for users to accomplish the use-cases I had in mind. I think I may go with an entirely different approach.
Re: Generically call a function on Variant's payload?
On Monday, 20 August 2018 at 00:27:04 UTC, Nick Sabalausky (Abscissa) wrote: Suppose I've wrapped a Variant in a struct/class which ensures the Variant *only* ever contains types which satisfy a particular constraint (for example: isInputRange, or hasLength, etc...). Is there a way to call a function (ex: popFront) on the Variant, *without* knowing ahead of time all the possible static types it might might contain? I assume you mean at compile time AoT of development of your library? Yes, don't use Variant; use https://github.com/s-ludwig/taggedalgebraic
Re: Generically call a function on Variant's payload?
On 08/20/2018 10:57 PM, Jonathan M Davis wrote: Runtime reflection is theoretically possible in D, but it requires generating the appropriate stuff for every type involved so that the runtime stuff has something to work with. Java built all of that into the language and has the JVM to boot, which fundamentally changes some of what can be done. With D, we're basically in exactly the same boat as C or C++ except that we have better compile-time type introspection. In principle, a runtime reflection facility could be built using that, but even if it were part of Phobos, it would still have to be opt-in. So, I don't know how useful such a solution would ever be outside of very specific use cases. Regardless, given that D's object files, linking, etc. are using the C tools, it was never going to be the case that Java-style runtime reflection would be built in to D like it is with Java. Yea. Not to disagree at all with those reasons, but that is unfortunate. I've long been interested in how various features of D (not exclusively D, though) combine in ways that, in effect, obsolete much of traditional OOP inheritence-based polymorphism - offering the same abilities of OOP but without many of the notable downsides. Java-style runtime reflection would take us that much further in this regard. (Plus, it would make D that much more of a universal-toolbox of a language.) Oh well, dreams vs reality ;)
Re: Generically call a function on Variant's payload?
On Monday, August 20, 2018 8:29:30 PM MDT Nick Sabalausky (Abscissa) via Digitalmars-d-learn wrote: > On 08/20/2018 04:34 PM, Jonathan M Davis wrote: > > foreach(T; TypesThatVariantHolds) > > Yea, that's what I would've just done, but I wanted to support > user-created types not already known to my library. > > > You > > can't just call functions on completely unknown types, because the > > compiler wouldn't know what code to generate or what to link against. > > Right. D's templates have really spoiled me. Because of them, I've grown > accustomed to invoking functions on unknown types ;) > > But you're right, that's not going to work for runtime dispatch without > enumerating each possible type. Even if I try (like I was thinking) to > provide a templated function to convert any of the concrete types to an > internal-only supertype, I'd still need something to runtime dispatch to > the right template instance. Darn. > > A lot of times, I really wish I could introspect across *all* linked > modules, not just specific ones. Ex: "Hey compiler, give me a > compile-time list of ALL types in this program with UDA @xyz". Then > things like this could be built out of that. No need to worry about > unknown types/symbols because they would all be known and a finite list > could always be constructed. > > Although, what would REALLY be nice is if, for every type, the compiler > built a list of every member and how to access it (which...really it has > to do anyway), but then uses that info to build a master runtime > dispatch system. Types such as Variant already have runtime knowledge of > what type is currently being held, so that info could be passed to the > compiler-generated "master runtime dispatch" system along with what > member to invoke/access. > > Come to think of it...aren't I just describing standard run-of-the-mill > runtime reflection? I mean, hasn't Java already been able to do that for > ages? But then, in Java everything is already part of the class > hierarchy anyway so maybe that's why it's possible there? Runtime reflection is theoretically possible in D, but it requires generating the appropriate stuff for every type involved so that the runtime stuff has something to work with. Java built all of that into the language and has the JVM to boot, which fundamentally changes some of what can be done. With D, we're basically in exactly the same boat as C or C++ except that we have better compile-time type introspection. In principle, a runtime reflection facility could be built using that, but even if it were part of Phobos, it would still have to be opt-in. So, I don't know how useful such a solution would ever be outside of very specific use cases. Regardless, given that D's object files, linking, etc. are using the C tools, it was never going to be the case that Java-style runtime reflection would be built in to D like it is with Java. - Jonathan M Davis
Re: Generically call a function on Variant's payload?
On 08/20/2018 04:34 PM, Jonathan M Davis wrote: foreach(T; TypesThatVariantHolds) Yea, that's what I would've just done, but I wanted to support user-created types not already known to my library. You can't just call functions on completely unknown types, because the compiler wouldn't know what code to generate or what to link against. Right. D's templates have really spoiled me. Because of them, I've grown accustomed to invoking functions on unknown types ;) But you're right, that's not going to work for runtime dispatch without enumerating each possible type. Even if I try (like I was thinking) to provide a templated function to convert any of the concrete types to an internal-only supertype, I'd still need something to runtime dispatch to the right template instance. Darn. A lot of times, I really wish I could introspect across *all* linked modules, not just specific ones. Ex: "Hey compiler, give me a compile-time list of ALL types in this program with UDA @xyz". Then things like this could be built out of that. No need to worry about unknown types/symbols because they would all be known and a finite list could always be constructed. Although, what would REALLY be nice is if, for every type, the compiler built a list of every member and how to access it (which...really it has to do anyway), but then uses that info to build a master runtime dispatch system. Types such as Variant already have runtime knowledge of what type is currently being held, so that info could be passed to the compiler-generated "master runtime dispatch" system along with what member to invoke/access. Come to think of it...aren't I just describing standard run-of-the-mill runtime reflection? I mean, hasn't Java already been able to do that for ages? But then, in Java everything is already part of the class hierarchy anyway so maybe that's why it's possible there?
Re: Generically call a function on Variant's payload?
On Monday, August 20, 2018 1:38:13 PM MDT Nick Sabalausky (Abscissa) via
Digitalmars-d-learn wrote:
> There are a bunch of discriminated union types available for D, but the
> only one I'm aware of that *doesn't* require a finite-sized list of
> types known ahead-of-time is Phobos's Variant. The only problem is
> calling a function on a type not already known ahead-of-time. Maybe
> that's unsolvable? If so, then I'll fallback to the std.digest approach.
> But I'd prefer to avoid that, if possible.
It can be done if you know the list of types ahead of time, but you
basically have to check which type it is at runtime and then pick which code
to run based on that type, which means that you're essentially duplicating a
portion of the code for every type - though depending, it would be possible
to do something like templatize the code so that you don't have to
explicitly dulpicate it. But in the end, aside from using classes, if you're
using a variant type, I think that you're stuck doing something like
// exact API made up here rather than bothering to take the type to look up
// Variant's API.
void popFront()
{
foreach(T; TypesThatVariantHolds)
{
if(_variant.contains!T)
{
auto v = _variant.get!T;
v.popFront();
return;
}
}
}
On the other hand, if you don't know the exact list of types ahead of time
or require that the code calling this code tell you which type to use, then
you're pretty much stuck using something like classes or delegates. You
can't just call functions on completely unknown types, because the compiler
wouldn't know what code to generate or what to link against.
- Jonathan M Davis
Re: Generically call a function on Variant's payload?
On 08/19/2018 11:31 PM, Paul Backus wrote:
You are basically reinventing OOP here.
Yes, I am. Deliberately, in fact. Class inheritance is my backup plan,
though.
My use-case is actually very, very similar to std.digest:
There are various benefits to using a template-and-constraint based
interface. But one notable downside is the inability to handle the
situation where the actual type needed is only known at runtime. In
std.digest, this dilemma is dealt with by duplicating the entire
interface in BOTH template/constraint AND class-inheritance varieties.
Additionally, an important thing to note in the case of std.digest is
that `isDigest!T` returns FALSE for types using the OOP version of the
interface. I think there is (arguably) a certain merit in that:
The OOP interface obviously uses classes, whereas (in order to obtain
the full potential benefits of a template-and-constraint approach) the
template interface uses (and even requires) structs. Because of this,
instances should be constructed differently ("T instance;" vs "T
instance = new T();"). Thus, any generic code that wants to handle ANY
digest type must be very careful to mind this distinction whenever
creating new instances. Presumably, this may be why std.digest chose to
NOT let OO digests satisfy the template-oriented isDigest.
And then std.digest also needs WrapperDigest, needed to convert a
template-style digest to OO-style.
So...while std.digest succeeds at offering the best-of-both-worlds
between template and OO approaches, it also creates a whole new mess via
completely duplicated interfaces that aren't 100% compatible.
I want to see if I can do better.
So back to the root of the problem:
We have a bunch of types, including user-defined types we don't have
advance knowledge of. And we need a type which can hold any of them at
runtime.
AFAIK, there are two classic ways to do that: One is OOP, the other is
an unbounded discriminated union (a variant). Unlike class-based
inheritance, a variant CAN be implemented as a struct. So, at least in
theory, a variant-based type could potentially be made which implements
the *true* template-based interface, eliminating the need for duplicate
APIs and the mess that entails.
There are a bunch of discriminated union types available for D, but the
only one I'm aware of that *doesn't* require a finite-sized list of
types known ahead-of-time is Phobos's Variant. The only problem is
calling a function on a type not already known ahead-of-time. Maybe
that's unsolvable? If so, then I'll fallback to the std.digest approach.
But I'd prefer to avoid that, if possible.
Re: Generically call a function on Variant's payload?
On Sunday, August 19, 2018 9:08:39 PM MDT Nick Sabalausky (Abscissa) via Digitalmars-d-learn wrote: > On 08/19/2018 10:23 PM, Jonathan M Davis wrote: > > On Sunday, August 19, 2018 6:33:06 PM MDT Nick Sabalausky (Abscissa) via > > > > Digitalmars-d-learn wrote: > >> Maybe something involving using Variant.coerce to convert the payload > >> to > >> a single common type? Not sure how I would do that though. > > > > You could always create a wrapper type. Whatever code you're dealing > > with is going to need to statically know what type it's using even > > that's a base type in a class hierarchy. So, you need to present a type > > which actually has the appropriate API even if internally, it can > > dynamically handle several types which have that API, and it's not > > known ahead of time which type that is. > > I guess the parts I'm unclear on are: > > 1. What mechanism does Variant.coerce!SomeType use to attempt conversion > to SomeType? > > 2. How (if possible) can I provide a way to convert something to type > SomeType which Variant.coerce!SomeType will then recognize and use? Glancing at coerce's implementation, it just uses std.conv.to, but it's fairly restricted on which types it will convert to. The target type has to be numeric, bool, convertible to Object, or an array of characters. It looks like it will convert the object to string to do the conversion, so it's doing slightly more than just using std.conv.to and restricting the conversions, but really, it's basically just calling std.conv.to in a restricted manner. So, looking at coerce, I don't really understand why it exists. It just seems like it would make more sense to tell folks to get the object out with get and then convert it themselves using std.conv.to or whatever makes the most sense for their use case. Presenting a function that uses std.conv.to but restricts which conversions work really doesn't make sense to me. Either way, if you're doing something like using a Variant to hold multiple range types, I very much doubt that coerce is going to do you much good. - Jonathan M Davis
Re: Generically call a function on Variant's payload?
On Monday, 20 August 2018 at 00:27:04 UTC, Nick Sabalausky (Abscissa) wrote: Suppose I've wrapped a Variant in a struct/class which ensures the Variant *only* ever contains types which satisfy a particular constraint (for example: isInputRange, or hasLength, etc...). Is there a way to call a function (ex: popFront) on the Variant, *without* knowing ahead of time all the possible static types it might might contain? You are basically reinventing OOP here. Instead of Variants, use objects that implement an interface (e.g., `std.range.interfaces.InputRange`). Then you can call methods that belong to that interface and rely on virtual method dispatch to choose the correct implementation at runtime.
Re: Generically call a function on Variant's payload?
On 08/19/2018 10:23 PM, Jonathan M Davis wrote: On Sunday, August 19, 2018 6:33:06 PM MDT Nick Sabalausky (Abscissa) via Digitalmars-d-learn wrote: Maybe something involving using Variant.coerce to convert the payload to a single common type? Not sure how I would do that though. You could always create a wrapper type. Whatever code you're dealing with is going to need to statically know what type it's using even that's a base type in a class hierarchy. So, you need to present a type which actually has the appropriate API even if internally, it can dynamically handle several types which have that API, and it's not known ahead of time which type that is. I guess the parts I'm unclear on are: 1. What mechanism does Variant.coerce!SomeType use to attempt conversion to SomeType? 2. How (if possible) can I provide a way to convert something to type SomeType which Variant.coerce!SomeType will then recognize and use?
Re: Generically call a function on Variant's payload?
On Sunday, August 19, 2018 6:33:06 PM MDT Nick Sabalausky (Abscissa) via Digitalmars-d-learn wrote: > On 08/19/2018 08:27 PM, Nick Sabalausky (Abscissa) wrote: > > Suppose I've wrapped a Variant in a struct/class which ensures the > > Variant *only* ever contains types which satisfy a particular constraint > > (for example: isInputRange, or hasLength, etc...). > > > > Is there a way to call a function (ex: popFront) on the Variant, > > *without* knowing ahead of time all the possible static types it might > > might contain? > > Maybe something involving using Variant.coerce to convert the payload to > a single common type? Not sure how I would do that though. You could always create a wrapper type. Whatever code you're dealing with is going to need to statically know what type it's using even that's a base type in a class hierarchy. So, you need to present a type which actually has the appropriate API even if internally, it can dynamically handle several types which have that API, and it's not known ahead of time which type that is. - Jonathan M Davis
Re: Generically call a function on Variant's payload?
On 08/19/2018 08:27 PM, Nick Sabalausky (Abscissa) wrote: Suppose I've wrapped a Variant in a struct/class which ensures the Variant *only* ever contains types which satisfy a particular constraint (for example: isInputRange, or hasLength, etc...). Is there a way to call a function (ex: popFront) on the Variant, *without* knowing ahead of time all the possible static types it might might contain? Maybe something involving using Variant.coerce to convert the payload to a single common type? Not sure how I would do that though.
Generically call a function on Variant's payload?
Suppose I've wrapped a Variant in a struct/class which ensures the Variant *only* ever contains types which satisfy a particular constraint (for example: isInputRange, or hasLength, etc...). Is there a way to call a function (ex: popFront) on the Variant, *without* knowing ahead of time all the possible static types it might might contain?
