Re: Re: Vtables organization
At 10:54 AM -0500 1/23/04, Benjamin Kojm Stuhl wrote: Well, that was why I had my suggested sample pseudocode restore the previous vtable pointer before calling down to the next function (and put itself back when that's done). That has reentrancy issues, unfortunately. Potentially threading and DOD issues as well. Keeping the vtable reasonably immutable's in our best interests. -- Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: Re: Vtables organization
-Original Message- > Date: Fri Jan 23 09:27:12 EST 2004 > From: "Dan Sugalski" <[EMAIL PROTECTED]> > At 10:37 PM -0500 1/22/04, Benjamin K. Stuhl wrote: > >Dan Sugalski wrote: > >>In addition to the thread autolocking front end and debugging front > >>end vtable functions, both of which can be generic, there's the > >>potential for tracing and auditing front end functions, input data > >>massaging wrappers, and all manner of Truly Evil front (and back) > >>end wrappers that don't need to actually access the guts of the > >>PMC, but can instead rely on the other vtable functions to get the > >>information that they need to operate. > >> > >>Not that this necessarily mandates passing in the vtable pointer to > >>the functions, but the uses aren't exactly marginal. > > > >Going back to the idea of generating these vtables on the fly (and > >caching them): each instance of a vtable gets a void* closure in the > >vtable itself, > >so at a certain expense in extra vtables, one could hang a structure off > >of that that includes a pointer to the original vtable. > > Which I thought of, but that only allows for one layer of > indirection, and doesn't allow the original vtable to hang any data > off its vtable data pointer. (Which exists, and is there for that > very reason) If you have two or three layers of vtable functions > installed then it becomes difficult and time-consuming to find the > right data pointer--if you allow the same vtable to be layered in > multiple times (and no, I don't know why you'd want to) then it > becomes essentially impossible. > > Unfortunately the layers need to stay separate with separate data > attached, so if we allow layering and don't forbid the same layer in > there twice we have to pass in the pointer to the vtable actually > being called into, so the vtable functions can find the > layer-specific data. Well, that was why I had my suggested sample pseudocode restore the previous vtable pointer before calling down to the next function (and put itself back when that's done). This means that every vtable function knows that PMC->vtable is the vtable _for the current vtable function_, and so any vtable function can be confident that it is accessing the corect layer-specific data. It's a bit more complexity and 2 extra assignments in the wrapper vtable functions versus an extra parameter to _all_ vtable functions. -- BKS
Re: Vtables organization
At 10:37 PM -0500 1/22/04, Benjamin K. Stuhl wrote: Dan Sugalski wrote: In addition to the thread autolocking front end and debugging front end vtable functions, both of which can be generic, there's the potential for tracing and auditing front end functions, input data massaging wrappers, and all manner of Truly Evil front (and back) end wrappers that don't need to actually access the guts of the PMC, but can instead rely on the other vtable functions to get the information that they need to operate. Not that this necessarily mandates passing in the vtable pointer to the functions, but the uses aren't exactly marginal. Going back to the idea of generating these vtables on the fly (and caching them): each instance of a vtable gets a void* closure in the vtable itself, so at a certain expense in extra vtables, one could hang a structure off of that that includes a pointer to the original vtable. Which I thought of, but that only allows for one layer of indirection, and doesn't allow the original vtable to hang any data off its vtable data pointer. (Which exists, and is there for that very reason) If you have two or three layers of vtable functions installed then it becomes difficult and time-consuming to find the right data pointer--if you allow the same vtable to be layered in multiple times (and no, I don't know why you'd want to) then it becomes essentially impossible. Unfortunately the layers need to stay separate with separate data attached, so if we allow layering and don't forbid the same layer in there twice we have to pass in the pointer to the vtable actually being called into, so the vtable functions can find the layer-specific data. -- Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: Vtables organization
Dan Sugalski wrote:
[sniping to reduce verbiage]
The issue is that the PMC's original vtable assumes (and should, IMHO
be _able_ to assume) that it has total control over the PMC's data,
Well... I think I'll disagree here. The *class* vtable can assume this.
However that doesn't mean that any random vtable function can.
Yes, that's what I meant by the original vtable. :-)
In addition to the thread autolocking front end and debugging front end
vtable functions, both of which can be generic, there's the potential
for tracing and auditing front end functions, input data massaging
wrappers, and all manner of Truly Evil front (and back) end wrappers
that don't need to actually access the guts of the PMC, but can instead
rely on the other vtable functions to get the information that they need
to operate.
Not that this necessarily mandates passing in the vtable pointer to the
functions, but the uses aren't exactly marginal.
Going back to the idea of generating these vtables on the fly (and caching
them): each instance of a vtable gets a void* closure in the vtable itself,
so at a certain expense in extra vtables, one could hang a structure off
of that that includes a pointer to the original vtable. E.g. (pseudo-code)
if (we don't have a tracing PerlInt in our cache) {
TracePerlIntVtable = clone_vtable(interp, TraceVtable);
vtable_set_data(interp, TracePerlIntVtable, PerlIntVtable);
cache(TracePerlIntVtable);
}
TraceVtable::get_number(INTERP, self) {
FLOATVAL f;
// I don't have the headers in front of me to get the right field names...
VTABLE *my_vtbl = self->vtable;
VTABLE *old_vtbl = my_vtbl->private;
self->vtable = old_vtbl;
f = self->vtable->get_number(interp, self);
TRACE("%p->get_number() = %f", self, f);
self->vtable = my_vtbl;
return f;
}
With slightly more complicated closures, most of the listed uses can use this
method. It's memory-heavy, but not too bad if we need a fair number of PMCs of
each wrapped type and the cost gets amortized over them. And this does save an
argument to every PMC function and just makes the wrapping classes pay. Where
it's suboptimal is for classes like autolocking that really want a closure of
their own on every PMC. But then again, that would again probably get into
issues of data collisions (except for locking, which has its own reserved area).
Just some thoughts.
-- BKS
Re: Vtables organization
At 8:37 AM -0500 1/19/04, Benjamin K. Stuhl wrote: Luke Palmer wrote: Benjamin K. Stuhl writes: Other than the special case of :readonly, can you give me an example of when you'd need to, rather than simply writing a PMC class that inherits from some base? I'm having trouble thinking of an example of when you'd want to be able to do that... After all, since operator overloading and tying effectively _replace_ the builtin operations, what more does one need? Well, other than Constant, we need to be able to put locking on shared PMCs. We'd like to add a debug trace to a PMC. We could even make any kind of PMC sync itself with an external source on access, though that's a bit of a pathological case. Indeed, all of this can be done, however, by subclassing Ref. I think the reason this isn't sufficient is that we want to change the actual PMC into this new variant when it is possibly already in existence. Like my C was evilly trying to do. Perhaps there's a way to get that working safely... The issue is that the PMC's original vtable assumes (and should, IMHO be _able_ to assume) that it has total control over the PMC's data, Well... I think I'll disagree here. The *class* vtable can assume this. However that doesn't mean that any random vtable function can. In addition to the thread autolocking front end and debugging front end vtable functions, both of which can be generic, there's the potential for tracing and auditing front end functions, input data massaging wrappers, and all manner of Truly Evil front (and back) end wrappers that don't need to actually access the guts of the PMC, but can instead rely on the other vtable functions to get the information that they need to operate. Not that this necessarily mandates passing in the vtable pointer to the functions, but the uses aren't exactly marginal. -- Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: Vtables organization
Luke Palmer wrote: Benjamin K. Stuhl writes: Other than the special case of :readonly, can you give me an example of when you'd need to, rather than simply writing a PMC class that inherits from some base? I'm having trouble thinking of an example of when you'd want to be able to do that... After all, since operator overloading and tying effectively _replace_ the builtin operations, what more does one need? Well, other than Constant, we need to be able to put locking on shared PMCs. We'd like to add a debug trace to a PMC. We could even make any kind of PMC sync itself with an external source on access, though that's a bit of a pathological case. Indeed, all of this can be done, however, by subclassing Ref. I think the reason this isn't sufficient is that we want to change the actual PMC into this new variant when it is possibly already in existence. Like my C was evilly trying to do. Perhaps there's a way to get that working safely... The issue is that the PMC's original vtable assumes (and should, IMHO be _able_ to assume) that it has total control over the PMC's data, so there is nowhere in the PMC to put a lock or a handle to an external source or anything. So you'd either need a Ref of some sort anyway or a global lookup table, which seems to be an even worse idea. Debug tracing, though, is a good question... I hate to pass an extra pointer to every vtable call just for that, though... -- BKS
Re: Vtables organization
Benjamin K. Stuhl writes: > Other than the special case of :readonly, can you give me an example > of when you'd need to, rather than simply writing a PMC class that > inherits from some base? I'm having trouble thinking of an example of > when you'd want to be able to do that... After all, since operator > overloading and tying effectively _replace_ the builtin operations, > what more does one need? Well, other than Constant, we need to be able to put locking on shared PMCs. We'd like to add a debug trace to a PMC. We could even make any kind of PMC sync itself with an external source on access, though that's a bit of a pathological case. Indeed, all of this can be done, however, by subclassing Ref. I think the reason this isn't sufficient is that we want to change the actual PMC into this new variant when it is possibly already in existence. Like my C was evilly trying to do. Perhaps there's a way to get that working safely... Luke > -- BKS
Re: Vtables organization
Thusly did Dan Sugalski inscribe: At 3:33 PM -0500 1/16/04, Benjamin K. Stuhl wrote: Dan Sugalski wrote: I was going to go on about a few ways to do this, but after I did I realized that only one option is viable. So, let's try this on for size: Vtables are chained. That means each vtable has a link to the next in the chain. It *also* means that each call into a vtable function has to pass in a pointer to the vtable the call came from so calls can be delegated properly. If we don't want this to suck down huge amounts of memory it also means that the vtable needs to be split into a vtable header and vtable function table body. Downside there is that we have an extra parameter (somewhat pricey) to all the vtable functions. This is sort of icky. What about dynamically constructing vtables and caching them? How would one wrap a vtable, then? If you do this, there's no way to call back into the original (or earlier wrapping) entry. Other than the special case of :readonly, can you give me an example of when you'd need to, rather than simply writing a PMC class that inherits from some base? I'm having trouble thinking of an example of when you'd want to be able to do that... After all, since operator overloading and tying effectively _replace_ the builtin operations, what more does one need? -- BKS
Re: Vtables organization
At 3:33 PM -0500 1/16/04, Benjamin K. Stuhl wrote: Dan Sugalski wrote: I was going to go on about a few ways to do this, but after I did I realized that only one option is viable. So, let's try this on for size: Vtables are chained. That means each vtable has a link to the next in the chain. It *also* means that each call into a vtable function has to pass in a pointer to the vtable the call came from so calls can be delegated properly. If we don't want this to suck down huge amounts of memory it also means that the vtable needs to be split into a vtable header and vtable function table body. Downside there is that we have an extra parameter (somewhat pricey) to all the vtable functions. This is sort of icky. What about dynamically constructing vtables and caching them? How would one wrap a vtable, then? If you do this, there's no way to call back into the original (or earlier wrapping) entry. -- Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Re: Vtables organization
Dan Sugalski wrote: I was going to go on about a few ways to do this, but after I did I realized that only one option is viable. So, let's try this on for size: Vtables are chained. That means each vtable has a link to the next in the chain. It *also* means that each call into a vtable function has to pass in a pointer to the vtable the call came from so calls can be delegated properly. If we don't want this to suck down huge amounts of memory it also means that the vtable needs to be split into a vtable header and vtable function table body. Downside there is that we have an extra parameter (somewhat pricey) to all the vtable functions. This is sort of icky. What about dynamically constructing vtables and caching them? We'd probably need some sort of mangling scheme, but it wouldn't be too hard; maybe just the ordered list of packages separated by NULs or something. Then again, though, any sort of stacking runs into data collisions in the PMC. So maybe we just need lots of vtable types. Ties should probably be their own vtable (or one each for tied scalars, hashes, and arrays); read-only-ification should maybe be a special case that actually does dynamically create (and cache, if need be) vtables, replacing ->set_* and ->morph with functions that throw an exception. For thread-sharing: proxies seem to be standard? What other attributes or traits or whatever actually expect to still be able to directly access the underlying variable? (Ties and overloads, for instance, _don't_ - they have to provide their own backing storage.) I suppose what I'm getting down to is that the intelligence should just be in the compile-time PMC class generator, not dynamic in the runtime. Except maybe for read-only-ification. -- BKS
Re: Vtables organization
At 11:53 AM +0100 1/16/04, Leopold Toetsch wrote: PMCs use Vtables for almost all their functionality *and* for stuff that in Perl5 term is "magic" (or they should AFAIK). E.g. setting the "_ro" property of a PMC (that supports it[1]) swaps in the Const$PMC vtable, where all vtable methods that would change the PMC thrown an exception. Or: setting a PMC shared, whould swap in a vtable, that locks e.g. internal aggregate state on access. That is a non-shared PMC doesn't suffer of any locking slowdown. Tieing will very likely swap in just another vtable and so on. The questions are: - Where and how should we store these vtables? - Are these PMC variants distinct types (with class_enum and name) - Or are these sub_types (BTW what is vtable->subtype)? E.g. hanging off from the "main" vtable? I was going to go on about a few ways to do this, but after I did I realized that only one option is viable. So, let's try this on for size: Vtables are chained. That means each vtable has a link to the next in the chain. It *also* means that each call into a vtable function has to pass in a pointer to the vtable the call came from so calls can be delegated properly. If we don't want this to suck down huge amounts of memory it also means that the vtable needs to be split into a vtable header and vtable function table body. Downside there is that we have an extra parameter (somewhat pricey) to all the vtable functions. -- Dan --"it's like this"--- Dan Sugalski even samurai [EMAIL PROTECTED] have teddy bears and even teddy bears get drunk
Vtables organization
PMCs use Vtables for almost all their functionality *and* for stuff that in Perl5 term is "magic" (or they should AFAIK). E.g. setting the "_ro" property of a PMC (that supports it[1]) swaps in the Const$PMC vtable, where all vtable methods that would change the PMC thrown an exception. Or: setting a PMC shared, whould swap in a vtable, that locks e.g. internal aggregate state on access. That is a non-shared PMC doesn't suffer of any locking slowdown. Tieing will very likely swap in just another vtable and so on. The questions are: - Where and how should we store these vtables? - Are these PMC variants distinct types (with class_enum and name) - Or are these sub_types (BTW what is vtable->subtype)? E.g. hanging off from the "main" vtable? Comments welcome, leo [1] This still needs more work: Real constant PMCs are allocated in a separate arena which isn't scanned during DOD, *but* all items, that the PMC may refer too have to be constant too, including Buffers it may use. But swapping in the vtable is working.
