Re: Proposed change to ForeignPtr
Alastair Reid wrote: [snip] What you're asking Hugs and NHC to do is: add a function to a list whenever you have a finalizer to run; make sure the interpreter will test that bit whenever it is in a position to perform a context switch. Am I really asking that much? In that paper you wrote you already propose that the Haskell system implement functions which tell C when it enters GC and when it leaves GC. So what it seems you need is for this to set a flag so that (1) When the (Foogle) finalizer runs, it runs normally if this flag is unset (no GC is running, so presumably safe). Otherwise add the action to a queue. (2) When Haskell leaves GC it checks the queue and runs any pending actions, backing up if it's necessary to do so. The flag and queue need to be atomically accessed, but since (for Hugs and NHC) you are assuming only one OS thread, that should be enough. You don't for example have to worry about exported functions being called absolutely everywhere. [snip] The cost is going over all data structures in the system making sure that operations on them are suitably atomic. [snip] If something like that would work, the cost during normal operation would only be the cost of setting and clearing the flag at the start and end of GC, and checking the queue at the end of GHC. I'm sorry, it's frightfully arrogant of me to argue over details of Haskell compilers with their implementors, but what else can I do here? Anyway the point is a general one; can we implement FFI without needing the whole machinery of concurrency? If it's really impossible for NHC or Hugs to implement this, I think I would still rather it was left to the NHC and Hugs documentation to admit that exported Haskell functions basically don't work in some circumstances, rather than to the GHC documentation to say that actually they do. It's a matter of taste how you do these things. If we take it that there is no way for a finalizer to call functions exported from NHC or Hugs, my personal preference would be to regard this as an undesirable implementation glitch which should be documented in the NHC and Hugs documentation. This would then be something users of the FFI with NHC and Hugs have to worry about, rather than users of the FFI in general. I don't think it makes the FFI worthless for NHC and Hugs, any more than the recent discovery that IO is not really a monad on most implementations makes the Haskell standard worthless. ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
RE: Proposed change to ForeignPtr
I confess that I have not followed the twists and turns of this discussion, but it seems to have gotten more complicated than necessary. There are several separate issues. 1. Can a finaliser for a Haskell value be an arbitrary Haskell computation? For GHC, yes. For Hugs, no (and for good reasons).So Hugs can only allow a C procedure as a finaliser. 2. If a garbage collector (whether the Haskell gc or the Foogle gc) finds that it is dropping a pointer to an object, when may it run the finalizer? Answer: it should be allowed to do so at any time: when the pointer is dropped (for ref-count gc) or during gc (as Hugs does) or after gc (as GHC does). 3. If the finalizer, in turn, needs to call some kind of foreign 'free' routine (e.g. the finaliser for a Haskell object encapsulating a Foogle object calls the Foogle free code; or the finalizer for a Foogle object encapsulating a Haskell value calls the Haskell free routine) it should be allowed to do so at any time. BUT DOING SO DOES NOT trigger an immediate GC in the other system. Several people have explained why it would be bad to trigger immediate GC. All the 'free' routine does is record that Foogle no longer holds this pointer (or vice versa). That information (as Alastair puts it) tweaks local data structures (yes, there may need to be a lock if there are separate OS threads involved), and that info is exploited at the next GC. 4. None of this says anything at all about asynchronous calls into (say) Haskell from Foogle. I don't understand what the issues are exactly. Simon ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
RE: Proposed change to ForeignPtr
I'm afraid George's questions have also rekindled my curiosity about whether implementing Haskell finalizers is really as hard as it sounds. Much has been written, but I still don't think we've got to the nub of the issue. On the face of it, if you can implement 'foreign import ccall safe', then you have a re-entrant runtime system. The times at which the program can make one of these foreign calls are limited, i.e. in the IO monad only - but I believe there's nothing particularly special about IO computations in the evaluation models used by nhc98 and Hugs (correct me if I'm wrong). Alastair writes: The way GHC implements preemption is an optimized form of: set a bit when preemption is needed; and make sure that generated code will test that bit whenever it is in a position to perform a context switch. What you're asking Hugs and NHC to do is: add a function to a list whenever you have a finalizer to run; make sure the interpreter will test that bit whenever it is in a position to perform a context switch. It's basically the same. We don't have to mess around with signals to provide a regular timer interrupt but that's the easy bit of the code. Ok so far. We can probably avoid messing around with multiple C stacks. That's a significant saving but, it's the complexity of that is fairly self-contained - we could probably steal some code from some other language implementation. The cost is going over all data structures in the system making sure that operations on them are suitably atomic. One of the issues I remember from old versions of GHC was that some of the primops would do some work, allocate some memory, then finish the job. The classic error to make in that code was for the second half of the code to assume almost anything about what happened in the first half of the code: how much space is on the stack, does a table have free space in it, is this pointer into the middle of an object ok? You certainly can't keep local variables live across a heap check, everything has to be saved on the stack. Hugs is different because it has a conservative GC, so doesn't need to save everything on the stack for a GC. But how does it implement a safe foreign call? Presumably it must save away state on the stack in a way that the computation can be resumed safely, and that's all you need in order to be able to run a Haskell finalizer. The problem is the scope: every single data structure and every bit of code that accesses it has to be vetted and we have to keep it in mind as we maintain the code. It's a high price to pay and I don't think it's necessary (because all you really need is for the runtime systems to talk to each other in very limited ways). I don't quite understand this: could you give a concrete example of some extra invariant that has to be maintained? In GHC, context switches happen at very precise points (i.e. heap checks) so I don't think we have these kind of problems; certainly I don't remember vetting every single data structure. Can't Hugs use a similar approach? Cheers, Simon ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: Proposed change to ForeignPtr
Manuel M T Chakravarty wrote: [snip] BTW, having two languages with separated heaps interact is a big mess as soon as you can have cycles, which you usually cannot exclude. Alastair already pointed that out and Martin Odersky also has nice stories to tell about this. [snip] Yeah yeah I know, indeed I think I pointed it out in the message which started this whole discussion. In the example application I am thinking of, I think cycles can be excluded in the first instance. In general I think there are various rather complicated things one might do in the finalizers to deal with cycles, and frankly I'd rather be able to write the code for this in Haskell rather than in C. ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: Proposed change to ForeignPtr
BTW, having two languages with separated heaps interact is a big
mess as soon as you can have cycles, which you usually cannot
exclude. Alastair already pointed that out and Martin Odersky also
has nice stories to tell about this.
Hmmm, way back in '94, my thought was that the only thing to do in the
presence of these cycles was to run the two GCs at once with a rather
intimate communication between them where one says 'I can reach X' the
other says 'ok, now I can reach Y', and eventually they both run out
of objects to trace and they can discard unreached objects.
The problem with this is that it runs into the same problems Malcolm
and I are so keen to avoid: the other language has to be able to
trigger GCs at more or less arbitrary times.
Since then, I've read a lot about non-stop concurrent GC which doesn't
need the two GCs to run simultaneously. Just thinking aloud, I wonder
if it would be any easier to implement? The communication would be
much the same ('I'm starting', 'I can reach X' and 'I'm done') but
there'd be no need to synchronize the GCs.
--
Alastair Reid [EMAIL PROTECTED]
Reid Consulting (UK) Limited http://www.reid-consulting-uk.ltd.uk/alastair/
___
FFI mailing list
[EMAIL PROTECTED]
http://www.haskell.org/mailman/listinfo/ffi
Re: Proposed change to ForeignPtr
Manuel M T Chakravarty [EMAIL PROTECTED] wrote, We seem to have a consensus on this one. We change the type of the existing functions to newForeignPtr :: Ptr a - FunPtr (Ptr a - IO ()) - IO (ForeignPtr a) addForeignPtrFinalizer :: ForeignPtr a - FunPtr (Ptr a - IO ()) - IO () For GHC, I propose to put the closure-based versions into an extra module (that's easy enough with the hierarchical libraries). This makes changing over old code easier, as it merely requires to alter the import and not all occurences of the functions. Any objections? I have changed this in the spec now. I attach the wording used in the spec. Manuel -=- \item[newForeignPtr ::\ Ptr a - FunPtr (Ptr a - IO ()) - IO (ForeignPtr a)] Turn a plain memory reference into a foreign object by associating a finalizer with the reference. The finalizer is represented by a pointer to an external function, which will be executed after the last reference to the foreign object is dropped. On invocation, the finalizer receives a pointer to the associated foreign object as an argument. Note that there is no guarantee on how soon the finalizer is executed after the last reference was dropped; this depends on the details of the Haskell storage manager. The only guarantee is that the finalizer runs before the program terminates. Whether a finaliser may call back into the Haskell system is system dependent. Portable code may not rely on such call backs. \item[addForeignPtrFinalizer ::\ ForeignPtr a - FunPtr (Ptr a - IO ()) - IO ()] Add another finalizer to the given foreign object. No guarantees are made on the order in which multiple finalizers for a single object are run. ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: Proposed change to ForeignPtr
Manuel wrote (snipped) I have changed this in the spec now. I attach the wording used in the spec. \item[newForeignPtr ::\ Ptr a - FunPtr (Ptr a - IO ()) - IO (ForeignPtr a)] Turn a plain memory reference into a foreign object by associating a finalizer with the reference. The finalizer is represented by a pointer to an external function, which will be executed after the last reference to the foreign object is dropped. On invocation, the finalizer receives a pointer to the associated foreign object as an argument. Note that there is no guarantee on how soon the finalizer is executed after the last reference was dropped; this depends on the details of the Haskell storage manager. The only guarantee is that the finalizer runs before the program terminates. Whether a finaliser may call back into the Haskell system is system dependent. Portable code may not rely on such call backs. \item[addForeignPtrFinalizer ::\ ForeignPtr a - FunPtr (Ptr a - IO ()) - IO ()] Add another finalizer to the given foreign object. No guarantees are made on the order in which multiple finalizers for a single object are run. I think this is all a rather murky area. Consider two systems, let's call them Haskell and Foogle, which both operate heaps and do their own storage allocation, but also communicate over similar FFIs. We might very reasonably have situations where fairly complex inter-language pointers exist, so for example Haskell holds a ForeignPtr to something in the Foogle heap; the pointed-to Foogle object in turn references a Haskell object (presumably provided via StablePtr). Now suppose Haskell wants to drop the ForeignPtr. Then the logical thing for the finalizer to do is to tell Foogle that Haskell is no longer interested in the Foogle object. This then gives Foogle the chance on its own garbage collection to in turn drop the Haskell StablePtr. In turn this means somehow running StablePtr.freeStablePtr. However this scheme I don't know if that's legal, because the Haskell finalizer you need to run freeStablePtr is indirectly provoked by the initial Haskell finalizer. This is a pity, because this might actually be a fairly good way of managing garbage collection between Foogle and Haskell. Of course you would need at least reference counters (if you can guarantee there are no cycles containing both languages) or something more powerful otherwise, but reference counters at least can be provided. Furthermore I do actually have a real case in mind where I might use something like this, though I'd rather not go into details at this time. I'm afraid I haven't been following this thread lately, so I don't know what the big problem is about calling Haskell from a finalizer; I suppose it's something to do with us being in the middle of garbage collection. However wouldn't it be better to allow finalizers to somehow provide an action which may call Haskell, but (a) may be performed at some later date (such as when the GC is over); (b) consequently, may not assume that the pointer finalized still points to anything? ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: Proposed change to ForeignPtr
I think this is all a rather murky area. Consider two systems, let's call them Haskell and Foogle, which both operate heaps and do their own storage allocation, but also communicate over similar FFIs. This is indeed a murky area. Especially tricky when you have two GC's (even if one is nothing more than reference counting) is the issue of cycles that span the two systems: neither side is willing to release its pointer into the other world until the other side releases its pointer. (See section 5.2 of http://www.reid-consulting-uk.ltd.uk/alastair/publications/FP94.ps.gz for more on the problem and a sketch of a fix.) We might very reasonably have situations where fairly complex inter-language pointers exist, so for example Haskell holds a ForeignPtr to something in the Foogle heap; the pointed-to Foogle object in turn references a Haskell object (presumably provided via StablePtr). Now suppose Haskell wants to drop the ForeignPtr. Then the logical thing for the finalizer to do is to tell Foogle that Haskell is no longer interested in the Foogle object. This then gives Foogle the chance on its own garbage collection to in turn drop the Haskell StablePtr. In turn this means somehow running StablePtr.freeStablePtr. However this scheme I don't know if that's legal, because the Haskell finalizer you need to run freeStablePtr is indirectly provoked by the initial Haskell finalizer. We should provide a C function hs_freeStablePtr and explicitly say that it is safe to call this from inside a finalizer. Of course you would need at least reference counters (if you can guarantee there are no cycles containing both languages) or something more powerful otherwise, but reference counters at least can be provided. At least half the times I've used ForeignPtrs (aka ForeignObjs aka MallocPtrs), I've had to implement my own reference counting so, yes, it does seem like it'd be good to include reference counting in the implementation somehow. I don't remember ever finding a good way to do it though. The design seems obvious enough but it seems there's always some odd little wrinkle like the object already having its own reference counting implementation or the other world wanting to talk about foo_t* (i.e., the type the external library provides) instead of hs_stablePtr_t. In particular, if I put a wrapper around an object so that I can attach a reference counter, you somehow always find yourself faced with the problem that a C library hands you a pointer to an unwrapped object and you have to try to track down the reference counter for it. Any ideas welcome. -- Alastair Reid [EMAIL PROTECTED] Reid Consulting (UK) Limited http://www.reid-consulting-uk.ltd.uk/alastair/ ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: Proposed change to ForeignPtr
Malcolm Wallace wrote [snip] I don't see the problem. The Foogle garbage collector runs separately and asynchronously to the Haskell GC. A Foogle object is released by the Haskell collector, then at a later moment in time, a Haskell object is released by the Foogle collector. Where is the conflict? [snip] Does at a later moment in time mean that it is late enough that we can be sure calling Haskell will be OK? Look, suppose for simplicity that Foogle implements an identical FFI to Haskell. So we have (Haskell ForeignPtr A) == (Foogle StablePtr A) (Foogle StablePtr A) points to (Foogle ForeignPtr B) (Foogle ForeignPtr B) == (Haskell StablePtr B) Haskell frees (Haskell ForeignPtr A) thereby causing the finalizer. This presumably does the Foogle equivalent of freeStablePtr on A. This *may* trigger an immediate Foogle garbage collection (I think Foogle's RTS is within its rights if it does), so that Foogle now wants to finalize Foogle ForeignPtr B. The Foogle finalizer action for foreignPtr B involves calling back to Haskell so that (Haskell StablePtr B) can have freeStablePtr applied to it. Now is the FFI specification going to guarantee that however quickly the Foogle garbage collector executes the Haskell finalizer, things will work. It seems to me the wording suggested does not guarantee this. All it says that the finaliser cannot portably call back into the Haskell system. But when the finaliser provokes an immediate GC which calls back into the Haskell system? I'm altogether rather puzzled by this notion of FunPtr's which are allowed to call Haskell back at some times and not others. Nor do I understand how it's supposed to work, say, on truly parallel Haskell implementations. [snip] Why do you suggest a need for reference counts? In the absence of cycles, surely the existing two garbage collectors (howsoever implemented) are sufficient. [snip] Garbage collectors need roots. If I understand the situation correctly, a StablePtr is itself a root, until explicitly freed. If you are handing out the same object via the same StablePtr to several different Foogle things, it seems to me you might attach a reference counter to the StablePtr, so that when Foogle says I am no longer interested in this (Haskell) StablePtr, you decrement the reference counter by 1, and deallocate if it reaches zero. Alternatively of course you could create a fresh StablePtr every time something is passed to Foogle. I think reference counters are sometimes used in things like CORBA, where a similar problem arises. Of course they cannot cope with interlanguage cycles. ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: Proposed change to ForeignPtr
Alastair Reid wrote [snip] We should provide a C function hs_freeStablePtr and explicitly say that it is safe to call this from inside a finalizer. [snip] This would be the simplest solution, but would not permit you to do anything more sophisticated at the Haskell side, such as reference counting or more complicated strategies to check for possible cycles. You might end up having to do all your inter-language GC code in C, ugh. ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: Proposed change to ForeignPtr
Malcolm Wallace wrote [snip] Quite simply, no finaliser (whether in Foogle or Haskell) should be capable of triggering a garbage collection within its call. This condition is absolutely necessary to prevent a cascade effect of cross-language garbage collections, where a finaliser in Haskell could trigger a GC in Foogle which triggers another (nested) GC in Haskell etc. [snip] Unfortunately some sort of cascade is exactly what we want and need when the Haskell finaliser indicates that Haskell is no longer interested in some Foogle object, which means Foogle can run a GC which indicates Foogle is no longer interested in some Haskell object and so on . . . Thus, if Haskell.freeSomething calls Foogle.freeSomething, and Foogle.freeSomething cannot cause a Foogle GC, then no Foogle finalisers are run yet, and so Foogle *cannot* call the Haskell world until the Haskell GC is complete. After that, it doesn't matter when the Foogle finaliser decides to run. [snip] But surely Foogle has no way of knowing when the Haskell GC is over? Suppose Haskell does [enter GC] ... [run finaliser 1] ... [run finaliser 2] ... [leave GC] Then you want Foogle to delay any Haskell calls consequent on finaliser 1 until [leave GC], don't you? How can it? Is it really so difficult to create some queue of delayed functions which can be appended to from C and which nhc checks every time it does leave GHC? ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: Proposed change to ForeignPtr
George Russell [EMAIL PROTECTED] writes: Unfortunately some sort of cascade is exactly what we want and need when the Haskell finaliser indicates that Haskell is no longer interested in some Foogle object, which means Foogle can run a GC which indicates Foogle is no longer interested in some Haskell object and so on . . . Yes. The key thing is just to delay the steps of the cascade so that you are not trying to call Haskell/Foogle code whilst there is no Haskell/Foogle heap available to run it. After that, it doesn't matter when the Foogle finaliser decides to run. But surely Foogle has no way of knowing when the Haskell GC is over? When sequential control returns to the Foogle world, that is when the Haskell GC is guaranteed to be complete. Suppose Haskell does [enter GC] ... [run finaliser 1] ... [run finaliser 2] ... [leave GC] Then you want Foogle to delay any Haskell calls consequent on finaliser 1 until [leave GC], don't you? How can it? Because the Haskell calls don't happen until the Foogle GC invokes them. Ok, you have a Haskell ForeignPtr which is really a Foogle object. It becomes garbage. At the next Haskell GC, its finaliser is run. The finaliser is not Haskell code. The finaliser is Foogle code. This finaliser runs and a Haskell StablePtr contained within the Foogle object becomes garbage. Although the Haskell object is regarded as a StablePtr in Haskell land, it is a Foogle ForeignPtr. Hence, its finaliser does not run yet. The Haskell GC finishes. Computation proceeds. At some later moment, Foogle exhausts its heap and starts a GC. The finaliser for the garbage object is now run. This is not Foogle code, it is Haskell code. In fact, it is the Haskell routine freeStablePtr. This is ok, because we have sufficient heap to run the finaliser. The StablePtr is released. The Foogle GC finishes. Computation proceeds. Is it really so difficult to create some queue of delayed functions which can be appended to from C and which nhc checks every time it does leave GC? I did implement this, once. The problem is that we don't know when to run the delayed finaliser. The moment Haskell GC finishes is *not* a good time. Some arbitrary reduction step is in mid-progress, and running another process at this moment corrupts the context. What you really need is some kind of scheduler that can decide when it is safe to run an independent thread. Regards, Malcolm ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: Proposed change to ForeignPtr
[snip] No, you do not really need separate threads for this problem to occur. All you need is, say, Hugs to call a GHC-exported function as a finalizer, in the same OS thread, GHC to run a garbage collection during this function, and the garbage collection in turn to want to run a Hugs finalizer, before the finalizer called from Hugs has finished. The Hugs and GHC runtimes can talk to each other just fine (or, if they can't it's a simple oversight and well fix it). There's no problem with GHC and Hugs each telling each other that some object they own has one less pointer to it. Next time it is convenient for the runtime, it can run a GC, perhaps recognize that there's a few GHC objects it can release and it tells the GHC runtime that it can release them. The reason we can do this is because it has limited scope: just a few data structures have to be tweaked to avoid GHC coming in when Hugs' data structures are in an inconsistent state. It's quite a different matter to allow arbitrary Haskell code to be run - that means the entire runtime system and libraries have to be made reentrant. Of course one wouldn't normally want to link GHC from Hugs, but if even these two cannot be made to meet, I don't know how you expect Haskell to call anything else with a reasonably flexible GC system; it puts the kybosh on Java for example, which I am fairly sure makes plenty of use of both callbacks and finalizers. That's fine, they can have all the finalizers they want. The finalizers can fiddle with things in the runtime system to tell the GC whatever they want. In any case it seems to me just as dangerous to assume that the implementation does not use OS threads, as to assume it does. The internal structure of your apps is up to you - use locks to avoid using single-threaded code in multithreaded manner. You are effectively writing on top of the FFI document If your program does this perfectly reasonable combination of finalizers, it will fall over in an undefined way should the implementation use OS threads; furthermore there is no way around this. Basically the fact that there is only one OS thread is an implementation detail, not something that the user should have to think about. Programmers are used to dealing with code which is single threaded or not reentrant. It's quite common. Is it really the case that neither NHC nor Hugs can implement a list of actions to be taken at the first convenient point after GC has finished without implementing the whole machinery of preemptive concurrency? I take Malcolm Wallace's word for it that it isn't trivial, but why do you need for example asynchronous interruption of Haskell threads, wait queues, or time slices? Surely what you need is some way of backing up the state upon return from GC in such a way that you can run the queued IO actions, which may be hard but is a long way off preemptive concurrency. The way GHC implements preemption is an optimized form of: set a bit when preemption is needed; and make sure that generated code will test that bit whenever it is in a position to perform a context switch. What you're asking Hugs and NHC to do is: add a function to a list whenever you have a finalizer to run; make sure the interpreter will test that bit whenever it is in a position to perform a context switch. It's basically the same. We don't have to mess around with signals to provide a regular timer interrupt but that's the easy bit of the code. We can probably avoid messing around with multiple C stacks. That's a significant saving but, it's the complexity of that is fairly self-contained - we could probably steal some code from some other language implementation. The cost is going over all data structures in the system making sure that operations on them are suitably atomic. One of the issues I remember from old versions of GHC was that some of the primops would do some work, allocate some memory, then finish the job. The classic error to make in that code was for the second half of the code to assume almost anything about what happened in the first half of the code: how much space is on the stack, does a table have free space in it, is this pointer into the middle of an object ok? The problem is the scope: every single data structure and every bit of code that accesses it has to be vetted and we have to keep it in mind as we maintain the code. It's a high price to pay and I don't think it's necessary (because all you really need is for the runtime systems to talk to each other in very limited ways). If it's really impossible for NHC or Hugs to implement this, I think I would still rather it was left to the NHC and Hugs documentation to admit that exported Haskell functions basically don't work in some circumstances, rather than to the GHC documentation to say that actually they do. It's a matter of taste how you do these things. -- Alastair Reid [EMAIL PROTECTED] Reid Consulting (UK)
RE: Proposed change to ForeignPtr
1) Add these functions: makeForeignPtr :: Ptr a - FunPtr (Ptr a - IO ()) - IO (ForeignPtr a) attachForeignPtrFinalizer :: ForeignPtr a - FunPtr (Ptr a - IO ()) - IO () It is implementation defined whether the free functions are allowed to call Haskell functions. 2) Remove newForeignPtr and addForeignPtrFinalizer [GHC can go ahead and list them as non-standard extensions] There's a minor issue about whether the old function names should be reused (leaving GHC to come up with its own names) or not. I have ceased to care either way. I have a slight preference for re-using the old names, at least for newForeignPtr. The reason is that it follows the naming conventions in http://www.haskell.org/~simonmar/libraries/conventions.html Cheers, Simon ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Proposed change to ForeignPtr
[Now that we've gotten the library specification issue out of the way, I'd like to revive discussion of this topic. Last time we discussed it, there seemed to be concensus that we would make a change but I didn't get much response when I made a concrete proposal. I'd like to resolve this promptly so that the impending Hugs release can match what the spec says (or vice-versa...).] Since requiring ForeignPtr.newForeignPtr would require preemptive concurrency (see previous discussion), I propose the following changes: 1) Add these functions: makeForeignPtr :: Ptr a - FunPtr (Ptr a - IO ()) - IO (ForeignPtr a) attachForeignPtrFinalizer :: ForeignPtr a - FunPtr (Ptr a - IO ()) - IO () It is implementation defined whether the free functions are allowed to call Haskell functions. 2) Remove newForeignPtr and addForeignPtrFinalizer [GHC can go ahead and list them as non-standard extensions] There's a minor issue about whether the old function names should be reused (leaving GHC to come up with its own names) or not. I have ceased to care either way. -- Alastair Reid [EMAIL PROTECTED] Reid Consulting (UK) Limited http://www.reid-consulting-uk.ltd.uk/alastair/ ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: Proposed change to ForeignPtr
Alastair Reid [EMAIL PROTECTED] writes: I'm not sure which position you're preferring here. I lean a bit towards using the old names for the new functions (the ones with free functions) and finding new names for the old functions (the ones with closure arguments). That would be my preference as well. Keep the existing standard names, but change their type signatures to reflect what Hugs and nhc98 can actually implement. Add new names with the old signatures for what ghc can additionally implement via its concurrency model. Regards, Malcolm ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
RE: Proposed change to ForeignPtr
What do you expect to happen if the finaliser calls a foreign exported function? Good question. I do not expect that to work on any platform that has difficulty implementing newForeignPtr (because you could use it to implement newForeignPtr). I don't know if it would be likely to work on GHC. I think the spec should say that it is an error or undefined depending on whether GHC supports reentrant finalizers or not. Yes, it will work in GHC. makeForeignPtr is easily implemented in terms of newForeignPtr, using a foreign import dynamic. That's a tricky one. From the standards point of view, I am actually *very* reluctant to introduce new names. On the other hand, reusing the old names will lead to a couple of unhappy emails from people using the old interface again. But only a couple I conjecture. Heh, I distinctly remember several complaints from *you* in the past when things have changed in GHC! Cheers, Simon ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: Proposed change to ForeignPtr
We seem to have a consensus on this one. We change the type of the existing functions to newForeignPtr :: Ptr a - FunPtr (Ptr a - IO ()) - IO (ForeignPtr a) addForeignPtrFinalizer :: ForeignPtr a - FunPtr (Ptr a - IO ()) - IO () For GHC, I propose to put the closure-based versions into an extra module (that's easy enough with the hierarchical libraries). This makes changing over old code easier, as it merely requires to alter the import and not all occurences of the functions. Any objections? Cheers, Manuel ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: Proposed change to ForeignPtr
Alastair Reid [EMAIL PROTECTED] wrote, What do you expect to happen if the finaliser calls a foreign exported function? Good question. I do not expect that to work on any platform that has difficulty implementing newForeignPtr (because you could use it to implement newForeignPtr). I don't know if it would be likely to work on GHC. SimonM, what do you think? I think the spec should say that it is an error or undefined depending on whether GHC supports reentrant finalizers or not. IMHO, it's a nice feature to have. I understand that the spec can't require it, as systems without preemptive threads can't implement it. However, it would be a pity if the new interfaces would mean that even systems that feature preemptive threads can't have it. That's a tricky one. From the standards point of view, I am actually *very* reluctant to introduce new names. On the other hand, reusing the old names will lead to a couple of unhappy emails from people using the old interface again. But only a couple I conjecture. I read this as you would also (= like me) be in favour of keeping the old names. Right? Other opinions? Cheers, Manuel ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: Proposed change to ForeignPtr
I assume you meant makeForeignPtr :: Ptr a - FunPtr (Ptr a - IO ()) - IO (ForeignPtr a) Oops, yes. What do you expect to happen if the finaliser calls a foreign exported function? Good question. I do not expect that to work on any platform that has difficulty implementing newForeignPtr (because you could use it to implement newForeignPtr). I don't know if it would be likely to work on GHC. I think the spec should say that it is an error or undefined depending on whether GHC supports reentrant finalizers or not. That's a tricky one. From the standards point of view, I am actually *very* reluctant to introduce new names. On the other hand, reusing the old names will lead to a couple of unhappy emails from people using the old interface again. But only a couple I conjecture. -- Alastair Reid [EMAIL PROTECTED] Reid Consulting (UK) Limited http://www.reid-consulting-uk.ltd.uk/alastair/ ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: Proposed change to ForeignPtr
Alastair wrote [snip] makeForeignPtr :: Ptr a - FunPtr (Ptr a - IO ()) - IO ForeignObj [snip] I don't understand this proposal. What is a ForeignObj? Sorry, that was a typo. The result type should be IO (ForeignPtr a) I call a C function, which gives me a cString :: Ptr CChar, and it's my responsibility to free it when I've finished with it. So I convert it to a ForeignPtr: foreignPtr - mkForeignPtr cString and then always refer to the pointer via foreignPtr. When foreignPtr is garbage collected the space is freed. So how do I do this with your proposal? With the existing spec, you would write: foreign import free :: Ptr CChar - IO () foo = do ... foreignPtr - newForeignPtr cString (free cString) ... With my proposal, you would write: foreign import free :: FunPtr (Ptr CChar - IO ()) foo = do ... foreignPtr - newForeignPtr cString free ... All the rest of your code to manipulate ForeignPtrs remains the same. (Well, there's a corresponding change in addForeignFinalizer if you happen to use that.) -- Alastair Reid [EMAIL PROTECTED] Reid Consulting (UK) Limited http://www.reid-consulting-uk.ltd.uk/alastair/ ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
