Re: Race-condition in alternative 'System.Timeout.timeout' implementation
You might want to take a look at https://github.com/alphaHeavy/timeout-control/blob/master/System/Timeout/Control.hs#L72too, though I'd guess it is subject to the same race condition. I have a few other fixes (for dealing with lifted bracket iirc) I still need to merge back from a private branch. On Sun, Feb 24, 2013 at 2:31 PM, Herbert Valerio Riedel h...@gnu.org wrote: Hello *, I've been experimenting with an alternative implementation of 'System.Timeout.timeout'[1] which avoids the overhead of spawning a new thread for each invocation. Part of my motivation is to see if I can implement a faster version of threadWaitReadTimeout :: Int - Fd - IO Bool threadWaitReadTimeout to = liftM (maybe False (const True)) . timeout to . threadWaitRead and thus exploit GHC's event notification system instead of having to reimplement a timeout-manager myself (like popular HTTP server libraries such as Snap or Warp do currently). The following Haskell program shows a proof-of-concept implementation derived directly from 'System.Timeout.timeout' together with a Criterion benchmark comparing the performance between the original and the alternative 'timeout' function wrapping a 'readMVar' call. On a i7-3770 with GHC-7.6.2/Linux/64bit ran with +RTS -A4m -N4, the benchmark shows a 15x improvement for the new implementation (below 1 uS) compared to the original implementation (~13 uS): , | benchmarking id | mean: 22.60933 ns, lb 22.50331 ns, ub 22.73515 ns, ci 0.950 | std dev: 591.0383 ps, lb 509.6189 ps, ub 663.2670 ps, ci 0.950 | found 17 outliers among 100 samples (17.0%) | 17 (17.0%) high mild | variance introduced by outliers: 19.992% | variance is moderately inflated by outliers | | benchmarking timeout_1ms | mean: 13.79584 us, lb 13.62939 us, ub 13.92814 us, ci 0.950 | std dev: 756.3080 ns, lb 524.7628 ns, ub 1.068547 us, ci 0.950 | found 14 outliers among 100 samples (14.0%) | 4 (4.0%) low severe | 5 (5.0%) high mild | 5 (5.0%) high severe | variance introduced by outliers: 52.484% | variance is severely inflated by outliers | | benchmarking timeout2_1ms | mean: 879.8152 ns, lb 874.5223 ns, ub 885.9759 ns, ci 0.950 | std dev: 29.31963 ns, lb 25.65941 ns, ub 32.98116 ns, ci 0.950 | found 9 outliers among 100 samples (9.0%) | 9 (9.0%) high mild | variance introduced by outliers: 28.734% | variance is moderately inflated by outliers | ... ` Alas there's a race-condition hidden somewhere I'm struggling with; When the timeout is set low enough, the internal 'Timeout2' exceptions leaks outside the 'timeout2' wrapper: , | ... | benchmarking timeout2_10us | newtimeout: timeout2 ` I've tried rewriting the code but couldn't figure out a way to keep the exception from escaping 'timeout2'. Does the race-condition actually lie in the 'timeout2' implementation -- and if so, is it possible to rewrite 'timeout2' to solve it? [1]: http://hackage.haskell.org/packages/archive/base/latest/doc/html/System-Timeout.html#v:timeout cheers, hvr ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Race-condition in alternative 'System.Timeout.timeout' implementation
Dear Herbert, I've been experimenting with an alternative implementation of 'System.Timeout.timeout'[1] which avoids the overhead of spawning a new thread for each invocation. Be warned that timeouts are very intricate. We had a lengthy discussion on the topic 2 years ago, starting at http://www.haskell.org/pipermail/libraries/2011-February/015876.html There was even an IO manager based proposal similar to yours: http://hackage.haskell.org/trac/ghc/ticket/4963 (What's the busyWontTimeout benchmark mentioned there?) http://www.haskell.org/pipermail/libraries/2011-February/015953.html The main trouble with the IO manager based approach is that even after unregisterTimeout finished, the timeout may still be invoked, and additional work is needed to protect against that. (I have more to say on this, but will postpone it until later. A lot of it has already been said in the earlier thread anyway.) Best regards, Bertram ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: base package -- goals
Hi, Am Samstag, den 23.02.2013, 10:27 + schrieb Simon Peyton-Jones: I’d like to be very clear about goals, though. I have not been following this thread closely enough, but is there a Wiki page that explains what the goals of the base-package break-up is? I added a Goals section to http://hackage.haskell.org/trac/ghc/wiki/SplitBase and besides I believe that the driving goal is: · to allow changes to internals without forcing a version-bump on ‘base’, on which every package depends added these two goals, which I find worthwhile to pursue: To allow packages to be explictly about what they need A library that does not use the IO monad could communicate that just by not depending on some base-io package. Similar with the Foreign Function Interface or unsafe operations. To allow alternative implementations/targets A Haskell-to-Javascript compiler will not support File IO, or maybe not even IO at all. It would be desirable such an implementation has a chance to at least provide a complete and API compatible base-pure package, and that one can hence reasonably assume that packages and libraries depending only on base-pure will indeed work without modification. This might be subsumed by fulfilling the previous goal. Just by looking at the goals, the variant with a big base package that uses all kinds of “uglyness” (FFI for pure stuff, cyclic dependencies between API-wise unrelated stuff, lots of GHC internals used) and re-exporting packages that have a more specific and possibly more stable API sounds like it can provide the mentioned goals. Iain commented this idea earlier this thread¹ with three points: * No-one would use the new packages unless they come with GHC; e.g. not a perfect analogy, but compare the number of rev-deps according to http://packdeps.haskellers.com/reverse of the various *prelude* packages vs base: 4831 base 6 basic-prelude 8 classy-prelude 4 classy-prelude-conduit 2 custom-prelude 1 general-prelude 1 modular-prelude 17 numeric-prelude 2 prelude-extras Hopefully the problem here (often-changing base) is big enough and the alternative (more purpose-oriented and more stable) packages are attractive enough to make people use the new set. * If it comes with GHC, it would mean us permanently maintaining the two levels True. I’m not convinced that it will be too much a burden, at least if the reexporting packages do that on the module level. * base would still be an opaque blob, with too many modules and cyclic imports, which makes development tricky Does it really make development tricky? I’d rather expect it to make development easier, because you _can_ mix, say, IO and Foreign stuff easily and even use some of that in lower levels. If it were less tricky to separate it, then my experiment at https://github.com/nomeata/packages-base/tree/base-split would have been less work... In any case there is still the problem: What and where is the Prelude... but maybe let’s postpone this. Greetings, Joachim ¹ http://www.haskell.org/pipermail/glasgow-haskell-users/2013-February/023774.html -- Joachim nomeata Breitner Debian Developer nome...@debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata signature.asc Description: This is a digitally signed message part ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: base package -- goals
| I added a Goals section to | http://hackage.haskell.org/trac/ghc/wiki/SplitBase Thanks. But the first goal, which is the dominant one, is very unclear to me as my comments mentioned. A description of what the problem is, and why a simple API wrapper approach would not solve it, would be useful. SImon | -Original Message- | From: glasgow-haskell-users-boun...@haskell.org [mailto:glasgow-haskell- | users-boun...@haskell.org] On Behalf Of Joachim Breitner | Sent: 25 February 2013 13:32 | To: glasgow-haskell-users@haskell.org | Subject: Re: base package -- goals | | Hi, | | Am Samstag, den 23.02.2013, 10:27 + schrieb Simon Peyton-Jones: | I’d like to be very clear about goals, though. I have not been | following this thread closely enough, but is there a Wiki page that | explains what the goals of the base-package break-up is? | | I added a Goals section to | http://hackage.haskell.org/trac/ghc/wiki/SplitBase | | and besides | | I believe that the driving goal is: | | · to allow changes to internals without forcing a version-bump | on ‘base’, on which every package depends | | added these two goals, which I find worthwhile to pursue: | | To allow packages to be explictly about what they need | | A library that does not use the IO monad could communicate that | just by not depending on some base-io package. Similar with the | Foreign Function Interface or unsafe operations. | | To allow alternative implementations/targets | | A Haskell-to-Javascript compiler will not support File IO, or | maybe not even IO at all. It would be desirable such an | implementation has a chance to at least provide a complete and | API compatible base-pure package, and that one can hence | reasonably assume that packages and libraries depending only on | base-pure will indeed work without modification. This might be | subsumed by fulfilling the previous goal. | | | Just by looking at the goals, the variant with a big base package that | uses all kinds of “uglyness” (FFI for pure stuff, cyclic dependencies | between API-wise unrelated stuff, lots of GHC internals used) and re- | exporting packages that have a more specific and possibly more stable | API sounds like it can provide the mentioned goals. | | Iain commented this idea earlier this thread¹ with three points: | | * No-one would use the new packages unless they come with GHC; |e.g. not a perfect analogy, but compare the number of rev-deps |according to http://packdeps.haskellers.com/reverse of the various |*prelude* packages vs base: |4831 base | 6 basic-prelude | 8 classy-prelude | 4 classy-prelude-conduit | 2 custom-prelude | 1 general-prelude | 1 modular-prelude | 17 numeric-prelude | 2 prelude-extras | | Hopefully the problem here (often-changing base) is big enough and the | alternative (more purpose-oriented and more stable) packages are | attractive enough to make people use the new set. | | * If it comes with GHC, it would mean us permanently maintaining the | two |levels | | True. I’m not convinced that it will be too much a burden, at least if | the reexporting packages do that on the module level. | | * base would still be an opaque blob, with too many modules and cyclic |imports, which makes development tricky | | Does it really make development tricky? I’d rather expect it to make | development easier, because you _can_ mix, say, IO and Foreign stuff | easily and even use some of that in lower levels. If it were less tricky | to separate it, then my experiment at | https://github.com/nomeata/packages-base/tree/base-split would have been | less work... | | | In any case there is still the problem: What and where is the Prelude... | but maybe let’s postpone this. | | Greetings, | Joachim | | ¹ http://www.haskell.org/pipermail/glasgow-haskell-users/2013- | February/023774.html | | | -- | Joachim nomeata Breitner | Debian Developer | nome...@debian.org | ICQ# 74513189 | GPG-Keyid: 4743206C | JID: nome...@joachim-breitner.de | http://people.debian.org/~nomeata ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: base package -- goals
On Mon, Feb 25, 2013 at 02:25:03PM +, Simon Peyton-Jones wrote: | I added a Goals section to | http://hackage.haskell.org/trac/ghc/wiki/SplitBase Thanks. But the first goal, which is the dominant one, is very unclear to me as my comments mentioned. A description of what the problem is, and why a simple API wrapper approach would not solve it, would be useful. On the wiki page you say: SPJ: But that goal needs a bit of unpacking. Suppose we divided base into six, base1, base2, base3, etc, but each was a vertical silo and every other package depended on all six. Then nothing would be gained; bumping any of them would cause a ripple of bumps down the line. but even if we did just divide base up into vertical silos then I don't think most packages would depend on them all; for example, most packages would probably not depend on file-io or concurrency. But in any case, I'd hope we would also make some horizontal cuts, and I expect very few packages would need to depend on ghc-io-manager etc. Thanks Ian ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: base package -- goals
On Mon, Feb 25, 2013 at 02:31:56PM +0100, Joachim Breitner wrote: Hopefully the problem here (often-changing base) is big enough and the alternative (more purpose-oriented and more stable) packages are attractive enough to make people use the new set. I'm pretty confident that most packages won't do more than the minimal base bumping while importing base continues to work. * base would still be an opaque blob, with too many modules and cyclic imports, which makes development tricky Does it really make development tricky? I’d rather expect it to make development easier, because you _can_ mix, say, IO and Foreign stuff easily and even use some of that in lower levels. If it were less tricky to separate it, then my experiment at https://github.com/nomeata/packages-base/tree/base-split would have been less work... It's tricky to make changes to the core modules, because that generally requires changing imports, and it's hard to see how to actually do that without making an import loop (or without making more import loops than are necessary). In general there's actually a fair amount of flexibility in which way module imports go (e.g. you can move a Cl Ty instance from the Cl module to the Ty module or vice-versa), but in base it's hard to see how to structure things best: there are approaching 200 modules, half of which are tied up in a recursive knot, with 13 hs-boot modules (2 of which import other hs-boot modules). In any case there is still the problem: What and where is the Prelude... but maybe let’s postpone this. I'd put it in its own package for now, and then look at whether/what it should be merged with later. I'm in 2 minds about it. On the one hand, I'm sure that lots of people won't like a single-module package that essentially everything depends on. But on the other hand, Prelude is both magic and broad, so it would make some sense to have it in its own package. Thanks Ian ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: base package -- goals
Somebody claiming to be Ian Lynagh wrote: On Mon, Feb 25, 2013 at 02:31:56PM +0100, Joachim Breitner wrote: In any case there is still the problem: What and where is the Prelude... but maybe let’s postpone this. I'd put it in its own package for now, and then look at whether/what it should be merged with later. Why shouldn't Prelude (and other really stable, standard modules) just live in the `haskell2010` package? -- Stephen Paul Weber, @singpolyma See http://singpolyma.net for how I prefer to be contacted edition right joseph signature.asc Description: Digital signature ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: base package -- goals
* Stephen Paul Weber singpol...@singpolyma.net [2013-02-25 11:29:42-0500] Why shouldn't Prelude (and other really stable, standard modules) just live in the `haskell2010` package? Because then we can't make changes to it without producing a new language standard. Roman ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: base package -- goals
Somebody claiming to be Roman Cheplyaka wrote: * Stephen Paul Weber singpol...@singpolyma.net [2013-02-25 11:29:42-0500] Why shouldn't Prelude (and other really stable, standard modules) just live in the `haskell2010` package? Because then we can't make changes to it without producing a new language standard. That sounds like a good thing. Very in line with the goal of making stable modules more stable. -- Stephen Paul Weber, @singpolyma See http://singpolyma.net for how I prefer to be contacted edition right joseph signature.asc Description: Digital signature ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: base package -- goals
On Mon, Feb 25, 2013 at 11:29:42AM -0500, Stephen Paul Weber wrote: Somebody claiming to be Ian Lynagh wrote: On Mon, Feb 25, 2013 at 02:31:56PM +0100, Joachim Breitner wrote: In any case there is still the problem: What and where is the Prelude... but maybe let’s postpone this. I'd put it in its own package for now, and then look at whether/what it should be merged with later. Why shouldn't Prelude (and other really stable, standard modules) just live in the `haskell2010` package? If we did that then every package would depend on haskell2010, which is fine until haskell2013 comes along and they all need to be changed (or miss out on any improvements that were made). Even the really stable modules change, incidentally. For example, since Haskell 2010, the Show superclass of Prelude.Num was removed, Prelude no longer exports catch, and Data.List gained a function dropWhileEnd. Thanks Ian ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: base package -- goals
Ian Lynagh i...@well-typed.com writes: [...] If we did that then every package would depend on haskell2010, which is fine until haskell2013 comes along and they all need to be changed (or miss out on any improvements that were made). ...wouldn't there also be the danger of type(class)-incompatible (e.g. the superclass breakages for startes) changes between say haskell2010 and haskell2013, that would cause problems when trying to mix libraries depending on different haskell20xx library versions? ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: base package -- goals
On Mon, Feb 25, 2013 at 06:38:46PM +0100, Herbert Valerio Riedel wrote: Ian Lynagh i...@well-typed.com writes: [...] If we did that then every package would depend on haskell2010, which is fine until haskell2013 comes along and they all need to be changed (or miss out on any improvements that were made). ...wouldn't there also be the danger of type(class)-incompatible (e.g. the superclass breakages for startes) changes between say haskell2010 and haskell2013, that would cause problems when trying to mix libraries depending on different haskell20xx library versions? I think that actually, for the Num/Show change, the hasell98/haskell2010 packages just incorrectly re-export the new class. Personally, I don't think the language report should be specifying the content of libraries at all, and I doubt anyone really uses the haskell* packages. A separate library specification, perhaps based on the Haskell Platform, would make more sense IMO. But that's another debate :-) Thanks Ian ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: base package -- goals
Hi all, Let me add the goals I had in mind last time I considered trying to split base. 1. I'd like to have text Handles use the Text type and binary Handles use the ByteString type. Right now we have this somewhat awkward setup where the I/O APIs are spread out and bundled with pure types. Splitting base would let us fix this and write a better I/O layer. 2. The I/O manager currently has a copy of IntMap inside its implementation because base cannot use containers. Splitting base would let us get rid of this code duplication. I'm less interested in having super fine-grained dependencies in my libraries. More packages usually means more busy-work managing dependencies. Taken to its extreme you could imagine having base-maybe, base-bool, and whatnot. I don't think this is an improvement. Splitting base into perhaps 3-5 packages (e.g. GHC.*, IO, pure types) should let us get a bunch of benefits without too many downsides. -- Johan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Race-condition in alternative 'System.Timeout.timeout' implementation
I accidentally replied to Herbert privately. I'm forwarding the message to the list. - Takano Akio -- Forwarded message -- From: Akio Takano tkn.a...@gmail.com Date: Mon, Feb 25, 2013 at 6:15 PM Subject: Re: Race-condition in alternative 'System.Timeout.timeout' implementation To: Herbert Valerio Riedel h...@gnu.org Hi, I think the problem is that E.unregisterTimeout doesn't guarantee that no timeout will be delivered after it returns. This allows an execution sequence like: 0. Thread A calls timeout2 1. Thread A registers a callback to the event manager 2. Thread A unregisters the callback 3. Thread A exits from the handleJust 4. The callback is triggered, killing thread A If I understand correctly this can be worked around with an extra mutex in timeout2. I'll attach my implementation. It's called timeout3 and is a bit slower than timeout2, but I haven't seen a leaking exception with it. On Mon, Feb 25, 2013 at 7:31 AM, Herbert Valerio Riedel h...@gnu.org wrote: Hello *, I've been experimenting with an alternative implementation of 'System.Timeout.timeout'[1] which avoids the overhead of spawning a new thread for each invocation. Part of my motivation is to see if I can implement a faster version of threadWaitReadTimeout :: Int - Fd - IO Bool threadWaitReadTimeout to = liftM (maybe False (const True)) . timeout to . threadWaitRead and thus exploit GHC's event notification system instead of having to reimplement a timeout-manager myself (like popular HTTP server libraries such as Snap or Warp do currently). The following Haskell program shows a proof-of-concept implementation derived directly from 'System.Timeout.timeout' together with a Criterion benchmark comparing the performance between the original and the alternative 'timeout' function wrapping a 'readMVar' call. On a i7-3770 with GHC-7.6.2/Linux/64bit ran with +RTS -A4m -N4, the benchmark shows a 15x improvement for the new implementation (below 1 uS) compared to the original implementation (~13 uS): , | benchmarking id | mean: 22.60933 ns, lb 22.50331 ns, ub 22.73515 ns, ci 0.950 | std dev: 591.0383 ps, lb 509.6189 ps, ub 663.2670 ps, ci 0.950 | found 17 outliers among 100 samples (17.0%) | 17 (17.0%) high mild | variance introduced by outliers: 19.992% | variance is moderately inflated by outliers | | benchmarking timeout_1ms | mean: 13.79584 us, lb 13.62939 us, ub 13.92814 us, ci 0.950 | std dev: 756.3080 ns, lb 524.7628 ns, ub 1.068547 us, ci 0.950 | found 14 outliers among 100 samples (14.0%) | 4 (4.0%) low severe | 5 (5.0%) high mild | 5 (5.0%) high severe | variance introduced by outliers: 52.484% | variance is severely inflated by outliers | | benchmarking timeout2_1ms | mean: 879.8152 ns, lb 874.5223 ns, ub 885.9759 ns, ci 0.950 | std dev: 29.31963 ns, lb 25.65941 ns, ub 32.98116 ns, ci 0.950 | found 9 outliers among 100 samples (9.0%) | 9 (9.0%) high mild | variance introduced by outliers: 28.734% | variance is moderately inflated by outliers | ... ` Alas there's a race-condition hidden somewhere I'm struggling with; When the timeout is set low enough, the internal 'Timeout2' exceptions leaks outside the 'timeout2' wrapper: , | ... | benchmarking timeout2_10us | newtimeout: timeout2 ` I've tried rewriting the code but couldn't figure out a way to keep the exception from escaping 'timeout2'. Does the race-condition actually lie in the 'timeout2' implementation -- and if so, is it possible to rewrite 'timeout2' to solve it? [1]: http://hackage.haskell.org/packages/archive/base/latest/doc/html/System-Timeout.html#v:timeout cheers, hvr ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users timeout3.hs Description: Binary data ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
