Re: [Haskell-cafe] STM and FFI
What are you trying to do? (1) Call a foreign function from inside an STM transaction? If the function is pure, this is trivial, just declare it as a pure function in the foreign import statement. You do need to be a bit careful, however, as it is possible the function will get called with invalid arguments, and I believe that GHC won't interrupt a thread inside of a foreign function call. So you need to make sure that the function never fails to terminate, even when given bad input. (There's an example code being called with improper arguments in Simon's STM paper). If the function isn't pure, you need to do a lot more proofs to assure that this is safe. In particular, the function must be able to be called with invalid input. If you are confident that this is the case, you can use unsafeIOToSTM to convert a call to that function into an STM primitive. (2) Have a foreign function use transactional memory primitives? I'm not sure that this is possible. (3) something else? -- ryan On Mon, Sep 8, 2008 at 2:56 PM, Mauricio [EMAIL PROTECTED] wrote: Hi, Is it possible to use foreign function interface with STMs? If so, where can I find examples? Thanks, Maurício ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] STM and FFI
On Tue, Sep 9, 2008 at 11:36, Jules Bean [EMAIL PROTECTED] wrote: ...not only must it be safe to be called with invalid inputs, but it most not have any long-term effects, whether the input is valid or invalid, since I do not believe that there is any way for the function to 'undo' its effect at 'retry' time. Maybe this is an idea for an extension to the STM system, adding something like unsafeIOToSTM, except that in addition to the main IO action, it also takes two more IO actions that are invoked on rollback and commit, respectively. This might allow for integration with transactional systems (e.g. a remote transaction on an rdbms), although to support two-phased commit we'd need a third action for the prepare step. cheers, Arnar ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] STM and FFI
Arnar Birgisson wrote: On Tue, Sep 9, 2008 at 11:36, Jules Bean [EMAIL PROTECTED] wrote: ...not only must it be safe to be called with invalid inputs, but it most not have any long-term effects, whether the input is valid or invalid, since I do not believe that there is any way for the function to 'undo' its effect at 'retry' time. Maybe this is an idea for an extension to the STM system, adding something like unsafeIOToSTM, except that in addition to the main IO action, it also takes two more IO actions that are invoked on rollback and commit, respectively. This might allow for integration with transactional systems (e.g. a remote transaction on an rdbms), although to support two-phased commit we'd need a third action for the prepare step. That would be an absolutely killer feature. A common problem in large systems is that the underlying RDBMS supports transactionality, but then the software layer has to handle its own rollbacks. I've seen some nasty bugs when the DB rolled back and the software didn't. If we could have a transactional RDBMS linked into STM with matching semantics, that would be a very nice thing. Jules ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] STM and FFI
On Tue, Sep 9, 2008 at 11:58, Jules Bean [EMAIL PROTECTED] wrote: Maybe this is an idea for an extension to the STM system, adding something like unsafeIOToSTM, except that in addition to the main IO action, it also takes two more IO actions that are invoked on rollback and commit, respectively. This might allow for integration with transactional systems (e.g. a remote transaction on an rdbms), although to support two-phased commit we'd need a third action for the prepare step. That would be an absolutely killer feature. A common problem in large systems is that the underlying RDBMS supports transactionality, but then the software layer has to handle its own rollbacks. I've seen some nasty bugs when the DB rolled back and the software didn't. If we could have a transactional RDBMS linked into STM with matching semantics, that would be a very nice thing. I think this is entirely doable. For comparison we already have done this with another STM framework, the DSTM2 library for Java. I.e. we hooked into prepare, commit and rollback and integrated with both MySQL transactions and a transactional file system library from Apache Commons. I'm not yet involved enough with the GHC library code, but I guess this would require the addition of a prepare phase to the STM code. There's also the question of what to do when the remote TX system indicates failure, should the transaction be retried or aborted? In the DSTM2 case we make it abort and throws an exception encapsulating the remote error to the code that initiated the TX (in Haskell's case, the caller of atomically). On a related note, we do have a paper on utilizing the STM system for authorization and policy enforcement in general. The paper is to be presented at CCS'08, and has an implementation on top of DSTM2, but we have a technical report in the works that implements this on top of the Haskell STM and gives operational semantics for the whole thing. You can find the conference paper on my website: http://www.hvergi.net/arnar/publications cheers, Arnar ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] STM and FFI
Ryan Ingram wrote: If the function isn't pure, you need to do a lot more proofs to assure that this is safe. In particular, the function must be able to be called with invalid input. If you are confident that this is the case, you can use unsafeIOToSTM to convert a call to that function into an STM primitive. ...not only must it be safe to be called with invalid inputs, but it most not have any long-term effects, whether the input is valid or invalid, since I do not believe that there is any way for the function to 'undo' its effect at 'retry' time. Jules ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] STM and FFI
I've been playing with this, and on top of STM as it exists, managed to neatly interleave it with sqite3 and postgres. To do so with postgres, however, required setting the locking mode to be a bit more restrictive than it is out-of-the-box. Clever use of encapsulation and monad transformers gets you 90% of the way there quite easily. Note, however, that unsafeIOToSTM is *much* more unsafe at the moment than you would expect -- in fact there is no safe way to use it at all, due to the interaction of exceptions and rollbacks at the moment. The thread about this on glasgow-haskell-users[1], along with my initial note, has a very useful reply by Simon Marlow where he both explains some things about the STM implementation and logic behind it that I didn't understand, and also describes how the GHC team intends to fix this at some point in the future. Regards, Sterl. [1] http://www.nabble.com/Where-STM-is-unstable-at-the-moment%2C-and- how-we-can-fix-it-tc19236082.html#a19236082 On Sep 9, 2008, at 6:08 AM, Arnar Birgisson wrote: On Tue, Sep 9, 2008 at 11:58, Jules Bean [EMAIL PROTECTED] wrote: Maybe this is an idea for an extension to the STM system, adding something like unsafeIOToSTM, except that in addition to the main IO action, it also takes two more IO actions that are invoked on rollback and commit, respectively. This might allow for integration with transactional systems (e.g. a remote transaction on an rdbms), although to support two-phased commit we'd need a third action for the prepare step. That would be an absolutely killer feature. A common problem in large systems is that the underlying RDBMS supports transactionality, but then the software layer has to handle its own rollbacks. I've seen some nasty bugs when the DB rolled back and the software didn't. If we could have a transactional RDBMS linked into STM with matching semantics, that would be a very nice thing. I think this is entirely doable. For comparison we already have done this with another STM framework, the DSTM2 library for Java. I.e. we hooked into prepare, commit and rollback and integrated with both MySQL transactions and a transactional file system library from Apache Commons. I'm not yet involved enough with the GHC library code, but I guess this would require the addition of a prepare phase to the STM code. There's also the question of what to do when the remote TX system indicates failure, should the transaction be retried or aborted? In the DSTM2 case we make it abort and throws an exception encapsulating the remote error to the code that initiated the TX (in Haskell's case, the caller of atomically). On a related note, we do have a paper on utilizing the STM system for authorization and policy enforcement in general. The paper is to be presented at CCS'08, and has an implementation on top of DSTM2, but we have a technical report in the works that implements this on top of the Haskell STM and gives operational semantics for the whole thing. You can find the conference paper on my website: http://www.hvergi.net/arnar/publications cheers, Arnar ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] STM and FFI
On Tue, Sep 9, 2008 at 13:58, Sterling Clover [EMAIL PROTECTED] wrote: I've been playing with this, and on top of STM as it exists, managed to neatly interleave it with sqite3 and postgres. To do so with postgres, however, required setting the locking mode to be a bit more restrictive than it is out-of-the-box. Clever use of encapsulation and monad transformers gets you 90% of the way there quite easily. Note, however, that unsafeIOToSTM is *much* more unsafe at the moment than you would expect -- in fact there is no safe way to use it at all, due to the interaction of exceptions and rollbacks at the moment. The thread about this on glasgow-haskell-users[1], along with my initial note, has a very useful reply by Simon Marlow where he both explains some things about the STM implementation and logic behind it that I didn't understand, and also describes how the GHC team intends to fix this at some point in the future. This is very interesting, do you have any code to release? Thanks for the ghu link, registering for that ML now :) cheers, Arnar ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] STM and FFI
briqueabraque: Hi, Is it possible to use foreign function interface with STMs? If so, where can I find examples? Defintely possible. FFI functions are imported as normal functions, so use them wherever the types fit. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
