Re: [Haskell-cafe] Ann: Win32-notify 0.1
Conal Elliott: I'd love to see this functionality available cross-platform. Are there plans for a unified library with a single API? +1 It would be nice if the two inotify packages and this one could be combined. And if someone would add a kqueue backend for the BSD's and OSX then most platforms would be supported. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Ann: Win32-notify 0.1
I'd love to see this functionality available cross-platform. Are there plans for a unified library with a single API? Yep, that's the plan (not too far) down the line, me and Lennart (who wrote the hinotify bindings) have already discussed it to some extent. I think the work order will be - get a decent API for Win32-notify up. - create a cross-platform single-API package on top of Win32-notify and hinotify. - go back to Win32-notify and make the implementation less naive. So API suggestions would be most welcome! Cheers, /Niklas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] I hate Haskell's typeclasses
On Sun, Apr 20, 2008 at 4:46 AM, Jonathan Cast [EMAIL PROTECTED] wrote: On 19 Apr 2008, at 5:02 AM, David MacIver wrote: Independently of the rant... On Sat, Apr 19, 2008 at 6:01 AM, Jonathan Cast [EMAIL PROTECTED] wrote: But why do I need to jump through these hoops for a perfectly safe commonly desired operation? It's called a proof obligation. Haskell is not here to stop you from jumping through hoops. In fact, it is here precisely to force you to jump through hoops. That's why it's called a bondage and discipline language. Surely it's there to lovingly assist you through the hoops? You can't just force people not to do the wrong thing and expect to get a good statically typed language out of it - you have to make it easier for them to do the right thing. I think going through the hoop is paramount in Haskell. That's why Haskell is pure, for example, even though it (still) requires awkward code on occasion. Haskell is certainly designed to make getting through the hoops as easy as possible, but never by providing a general way around them. (unsafePerformIO notwithstanding). Sure. I'm just saying, it's more of a Jump through this hoop and you shall have moist, delicious cake. And by the way, here's a leg up set up. There are rewards for the hoop jumping, and assistance on the way there (which is more than can be said for a lot of languages which make you jump through hoops) :-) I think I might be stretching the analogy slightly. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re[4]: [Haskell-cafe] C++ interface with Haskell
Hello Don, Saturday, April 19, 2008, 12:08:11 AM, you wrote: Would someone like to summarise the current approaches to combining Haskell C++ on the Haskell wiki, even if just in bullet points? started at http://haskell.org/haskellwiki/IO_inside#Interfacing_with_foreign_evil -- Best regards, Bulatmailto:[EMAIL PROTECTED] ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] I hate Haskell's typeclasses
On 20 Apr 2008, at 3:05 AM, David MacIver wrote: On Sun, Apr 20, 2008 at 4:46 AM, Jonathan Cast [EMAIL PROTECTED] wrote: On 19 Apr 2008, at 5:02 AM, David MacIver wrote: Independently of the rant... On Sat, Apr 19, 2008 at 6:01 AM, Jonathan Cast [EMAIL PROTECTED] wrote: But why do I need to jump through these hoops for a perfectly safe commonly desired operation? It's called a proof obligation. Haskell is not here to stop you from jumping through hoops. In fact, it is here precisely to force you to jump through hoops. That's why it's called a bondage and discipline language. Surely it's there to lovingly assist you through the hoops? You can't just force people not to do the wrong thing and expect to get a good statically typed language out of it - you have to make it easier for them to do the right thing. I think going through the hoop is paramount in Haskell. That's why Haskell is pure, for example, even though it (still) requires awkward code on occasion. Haskell is certainly designed to make getting through the hoops as easy as possible, but never by providing a general way around them. (unsafePerformIO notwithstanding). Sure. I'm just saying, it's more of a Jump through this hoop and you shall have moist, delicious cake. And by the way, here's a leg up set up. There are rewards for the hoop jumping, and assistance on the way there (which is more than can be said for a lot of languages which make you jump through hoops) :-) Absolutely. But I think the original rant strayed into the realm of wanting the cake and the leg up /without/ the hoop. I think I might be stretching the analogy slightly. There is nothing the least bit wrong with that. jcc ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] I hate Haskell's typeclasses
David MacIver wrote: Sure. I'm just saying, it's more of a Jump through this hoop and you shall have moist, delicious cake. And by the way, here's a leg up set up. There are rewards for the hoop jumping, and assistance on the way there (which is more than can be said for a lot of languages which make you jump through hoops) :-) I think I might be stretching the analogy slightly. THE CAKE IS A LIE!! (Sorry, couldn't resist... I'll try not to do that again.) ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Ann: Win32-notify 0.1
Niklas Broberg wrote: So API suggestions would be most welcome! Your challenge will lie in finding an API that can be implemented efficiently on all concerned platforms. I don't know what the characteristics of the Windows or OS X notification APIs are, but if you use inotify on Linux, and you immediately issue a read every time select tells you there's something to be read, you'll easily consume 70% of a CPU when the filesystem gets busy. With a little guile, you can reduce that overhead by two orders of magnitude, to a fraction of a percent. It's not all that easy to do this well, but using inotify naively is so crippling that you must do it. Any higher-level API must account for factors like this. b ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Ann: Win32-notify 0.1
On Sun, Apr 20, 2008 at 5:22 PM, Bryan O'Sullivan [EMAIL PROTECTED] wrote: Niklas Broberg wrote: So API suggestions would be most welcome! Your challenge will lie in finding an API that can be implemented efficiently on all concerned platforms. I don't know what the characteristics of the Windows or OS X notification APIs are, but if you use inotify on Linux, and you immediately issue a read every time select tells you there's something to be read, you'll easily consume 70% of a CPU when the filesystem gets busy. With a little guile, you can reduce that overhead by two orders of magnitude, to a fraction of a percent. It's not all that easy to do this well, but using inotify naively is so crippling that you must do it. Any higher-level API must account for factors like this. Factor has a file system notification API that wraps windows, linux and mac ones. It might be worth taking a look at that to see what it does. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Parallel weirdness [new insights]
OK, well I now have so much data sitting in from of me I don't even know *what* I'm seeing any more. I have made several significant discoveries though... Consider the following: msort [] = [] msort [x] = [x] msort xs = let xs0 = msort (split0 xs) xs1 = msort (split1 xs) in merge xs0 xs1 This takes roughly 14 seconds to sort a list of one million Word32 values. If I now change the final line to read in listSeq rwhnf xs0 `seq` listSeq rwhnf xs1 `seq` merge xs0 xs1 it now takes 8 seconds to do the same job. Notice that this is still completely sequential execution. It's just executing in a different order. (And, at first glance, doing slightly more work.) Of all the benchmarks I've performed, I have yet to find anything that goes faster than this. If I make it so that xs0 is computed in parallel with xs1 instead of in series, then it goes at roughly the same speed (but with more variation) if the number of real threads is 1. If you add more real threads, execution slows down. (Go figure!) I was expecting running parallel at just the top few levels and then switching to pure sequential for the lower levels to give the best performance. But the numbers I have seem to say that more parallel = slower, with 100% sequential giving me the fastest time of all. The next insight happens when you look at the GC statistics. Both the unmarked and the explicitly sequential program are giving me roughly 55% GC time and 45% user time. (!!) Obviously this is a Very Bad Thing. I discovered that simply by adding -H200m to the command line, I can make both programs speed up by about 20% or so. (They then drop down to roughly 25% GC time. Adding more RAM doesn't seem to make any difference.) I had assumed that the explicitly sequential program was faster because it was somehow demanding less GC time, but that doesn't appear to be the case - both GC time and user time are lower for the explicitly sequential version. And adding more heap space doesn't make the (large) speed difference go away. Is the strictness of the seq operator making GHC come up with different a Core implementation for this function or something? I have no idea. With the extra heap space, the speed difference between the sequential and parallel programs becomes smaller. The sequential version *is* still faster, however. I have no explanation for why that might be. Adding more heap also seems to make the runtimes more variable. (I run each test 8 times. One test, the fastest run was 7 seconds and the slowest was 11 seconds. That's quite a variation. The sequential algorithm only varies by a few milliseconds each time.) In short, it seems my little sorting algorithm test is *actually* just stressing out the GC engine, and I'm really benchmarking different GC settings. :-( Questions: 1. Does running the GC force all threads to stop? I know some GC designs do this, but I have no idea how the one GHC implements works. 2. Is the GC still single-threaded? (GHC 6.8.2 here.) 3. Is there any way for a running Haskell program to find out how much heap space is currently allocated / used / free? I know you can find out how much wall time and CPU time you've used, but I couldn't find anything for RAM usage. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Parallel weirdness [new insights]
On Apr 20, 2008, at 15:41 , Andrew Coppin wrote: 1. Does running the GC force all threads to stop? I know some GC designs do this, but I have no idea how the one GHC implements works. 2. Is the GC still single-threaded? (GHC 6.8.2 here.) Full GC is single-threaded and stops the entire program, yes. IIRC GHC's runtime tries to do incremental GC to minimize the need for a full GC. 3. Is there any way for a running Haskell program to find out how much heap space is currently allocated / used / free? I know you can find out how much wall time and CPU time you've used, but I couldn't find anything for RAM usage. You're looking for heap profiling in the GHC manual. -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] [EMAIL PROTECTED] system administrator [openafs,heimdal,too many hats] [EMAIL PROTECTED] electrical and computer engineering, carnegie mellon universityKF8NH ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Parallel weirdness [new insights]
On Apr 20, 2008, at 15:51 , Brandon S. Allbery KF8NH wrote: On Apr 20, 2008, at 15:41 , Andrew Coppin wrote: 3. Is there any way for a running Haskell program to find out how much heap space is currently allocated / used / free? I know you can find out how much wall time and CPU time you've used, but I couldn't find anything for RAM usage. You're looking for heap profiling in the GHC manual. Wait, no, I misread. I don't know if or how a running program could introspect its own heap usage. -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] [EMAIL PROTECTED] system administrator [openafs,heimdal,too many hats] [EMAIL PROTECTED] electrical and computer engineering, carnegie mellon universityKF8NH ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re[2]: [Haskell-cafe] Parallel weirdness [new insights]
Hello Andrew, Sunday, April 20, 2008, 11:41:52 PM, you wrote: yes, GC behavior has significant impact on any new language (i mean java, c#, f# and so on) 1. Does running the GC force all threads to stop? I know some GC designs do this, but I have no idea how the one GHC implements works. yes 2. Is the GC still single-threaded? (GHC 6.8.2 here.) yes. multi-threaded GC is planned gor next ghc version, afair 3. Is there any way for a running Haskell program to find out how much heap space is currently allocated / used / free? i think it's possible by asking internal RTS vars. SM once suggested to add to GHC library that provides official way to ask this info but no volunteer was happen :) -- Best regards, Bulatmailto:[EMAIL PROTECTED] ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Parallel weirdness [new insights]
Bulat Ziganshin wrote: 3. Is there any way for a running Haskell program to find out how much heap space is currently allocated / used / free? i think it's possible by asking internal RTS vars. SM once suggested to add to GHC library that provides official way to ask this info but no volunteer was happen :) The RTS can spit out aggregate data just with a CLI switch (and it doesn't appear to affect runtime noticably). You don't even need to compile with profiling enabled. This seems to indicate that the data is easy to collect, there's just no path for accessing it yet. I'm no GHC developer, but from the outside it appears to be a fairly simple problem. If I knew anything about the RTS, I'd volunteer myself. But I suspect this is one of those jobs that requires knowledge of C... :-( ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] letting go of file handles and Data.Binary
hi all -- using binary 0.4.1 on ghc 6.8.2, vista 64 sp1. consider the following program: import System.Directory import Data.Binary main = do let dat = [1..10]::[Int] fname = foo.dat encodeFile fname dat dat2 - decodeFile fname print (dat == dat2) removeFile fname this throws a permission denied exception, presumably because the file is still open when the removeFile is called. i've grovelled the source of Data.Bytestring.Lazy and all but i can't seem to understand the right way to make this work. note that i've forced the evaluation of dat2 and presumably therefore the filehandle is at least half-closed. any suggestions? take care, Ben ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Parallel weirdness [new insights]
Bulat Ziganshin wrote: yes. multi-threaded GC is planned gor next ghc version, afair To be clear, it'll be a parallel GC, not a concurrent one. The former still stops all threads, but runs the collector on multiple cores at once. The latter performs collection while mutator threads are still running, and is a lot trickier to implement. (For a fine knee-slapping time, try reading a Java GC tuning guide.) b ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] letting go of file handles and Data.Binary
Ben wrote: this throws a permission denied exception, presumably because the file is still open when the removeFile is called. Yes. The file handle opened by encodeFile won't be closed until its finalizer is run. There is no guarantee that the finalizer will be run immediately. In fact, you can usually rely on the finalizer *not* running immediately, because the GC has to kick in, notice that the handle is dead, and hand it to the finalizer thread. It's actually possible that your finalizer could not run at all. i've grovelled the source of Data.Bytestring.Lazy and all but i can't seem to understand the right way to make this work. Don't use encodeFile at all. Instead, write your own: import Data.Binary (Binary, encode) import Control.Exception (bracket) import qualified Data.ByteString.Lazy as L import System.IO (IOMode(..), hClose, openFile) encodeFile :: Binary a = FilePath - a - IO () encodeFile path = bracket (openFile path WriteMode) hClose . flip L.hPut . encode ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] letting go of file handles and Data.Binary
On Sun, 2008-04-20 at 14:02 -0700, Ben wrote: hi all -- using binary 0.4.1 on ghc 6.8.2, vista 64 sp1. consider the following program: import System.Directory import Data.Binary main = do let dat = [1..10]::[Int] fname = foo.dat encodeFile fname dat dat2 - decodeFile fname print (dat == dat2) removeFile fname this throws a permission denied exception, presumably because the file is still open when the removeFile is called. i've grovelled the source of Data.Bytestring.Lazy and all but i can't seem to understand the right way to make this work. note that i've forced the evaluation of dat2 and presumably therefore the filehandle is at least half-closed. any suggestions? There was a bug in bytestring-0.9.0.1 where the file handle was not closed when Data.Bytestring.Lazy.hGetContents reached the end of the file. The fix is certainly in version 0.9.0.4. So the above program should work there. If it still does not then the culprit is probably that your binary deserialiser is not consuming the whole input. Remember if you have access to a Handle you can always hClose it explicitly. Duncan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] letting go of file handles and Data.Binary
Doh! For all that I wrote about encodeFile, substitute decodeFile. You'll need to write something to force the value that you're decoding. Something like this ought to do the trick. import Data.Binary (Binary, decode) import Control.Exception (bracket) import qualified Data.ByteString.Lazy as L import System.IO (IOMode(..), hClose, openFile) strictDecodeFile :: Binary a = FilePath - (a - b) - IO () strictDecodeFile path force = bracket (openFile path ReadMode) hClose $ \h - do c - L.hGetContents h force (decode c) `seq` return () b ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] letting go of file handles and Data.Binary
On Sun, 2008-04-20 at 14:24 -0700, Bryan O'Sullivan wrote: Doh! For all that I wrote about encodeFile, substitute decodeFile. Indeed the version of encodeFile you wrote should be exactly identical to the original because the lazy bytestring writeFile already uses bracket like that: writeFile :: FilePath - ByteString - IO () writeFile f txt = bracket (openBinaryFile f WriteMode) hClose (\hdl - hPut hdl txt) strictDecodeFile :: Binary a = FilePath - (a - b) - IO () strictDecodeFile path force = bracket (openFile path ReadMode) hClose $ \h - do c - L.hGetContents h force (decode c) `seq` return () Yes, the problem with Ben's program was that decodeFile is lazily reading the file and lazily decoding. If the decoding consumes all the input then it should be possible to avoid rewriting decodeFile and use: dat2 - decodeFile fname evaluate dat2 removeFile fname It's not immediately clear to me if we can make the decodeFile behave like your version. I'd have to go think about whether running the Get monad can lazily return values or if it always consumes all the input it'll ever need. Duncan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] ANN: HXQ: An XQuery Compiler/Interpreter for Haskell
Dear Haskell cafe members, I would like to announce the release of an XQuery compiler/interpreter for Haskell, called HXQ. You can get more info and download it at: http://lambda.uta.edu/HXQ/ HXQ is a fast and space-efficient translator from XQuery (the standard query language for XML) to embedded Haskell code. The translation is based on Haskell templates. HXQ takes full advantage of Haskell's lazy evaluation to keep in memory only those parts of XML data needed at each point of evaluation, thus performing stream-based evaluation for forward queries (queries that do not contain backward steps). This results to an implementation that is as fast and space-efficient as any stream-based implementation based on SAX filters or finite state machines. Furthermore, the coding is far simpler and extensible since its based on XML trees, rather than SAX events. For example, a complex XQuery against the DBLP bibliography (420MB XML) runs in 39 seconds on my laptop (using 18MB of max heap space in ghc). To contrast this, Qexo, which compiles XQueries to Java bytecode, took 1 minute 17 seconds (using no less than 1400MB of heap space). Also XQilla, which is written in C++, took 1 minute and 10 secs (using 1150MB heap). Current Status: HXQ supports most essential XQuery features, although some system functions are missing (but are easy to add). HXQ has little support for backward step axes, such as /.. (parent). Some, but not all, parent axis steps are removed using optimization rules; all others cause a compilation error. Finally, HXQ does not comply to the XQuery semantics that requires duplicate elimination and sorting by document order for every XPath step, which is very expensive and unnecessary in most cases. Please email me any suggestions for improvements or extensions you'd like to see. My plan is to use it as a base for various XQuery projects, such as adding SQL connectivity, as well as native storage and indexing. Best, Leonidas Fegaras U. of Texas at Arlington http://lambda.uta.edu/ ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Announce: bytestring 0.9.1.0
Hey all, I'm pleased to announce a new major release of bytestring, the efficient string library for Haskell, suitable for high-performance scenarios. This release is primarily an (incremental) performance improvement release, though with some notable significant improvements, along with long term test coverage and quality control changes. Highlights: * a long term performance bug with Ord instances, involving very small strings, and Data.Map has been squashed. * everything's a little faster -- shootout problems showed a 1-5% speedup just by switching to the new library. Thanks goes to the Hac4 Haskell Hackathon organisers, in Gothenburg, Sweden, where the majority of this work to create this release took place. Key changes: * Data.Map short key performance greatly improved: - 'words Map' running time: 6.310s bytestring 0.9.0.1 1.071s bytestring 0.9.1.0 * Uses cheaper unsafeDupablePerformIO for allocation. - tail recursive tight loops (fixes obscure stack overflow) * Generally faster: - Shootout sum-file: 1.218s to 1.190s - Shooout fasta: 9.210s to 8.811s * 4-5x faster small substring search (breakSubstring/findSubstring/isInfixOf). * Extensive QuickCheck coverage reporting and improvements: - http://code.haskell.org/~dons/tests/bytestring/hpc_index.html Get the code: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/bytestring Note that if you upgrade to the new bytestring release, older packages built against previous releases will still require the old bytestring package. For best results, rebuild any bytestring-depending packages against the new library only. Cheers, Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Announce: bytestring 0.9.1.0
On 2008.04.20 15:09:33 -0700, Don Stewart [EMAIL PROTECTED] scribbled 1.7K characters: Hey all, I'm pleased to announce a new major release of bytestring, the efficient string library for Haskell, suitable for high-performance scenarios. This release is primarily an (incremental) performance improvement release, though with some notable significant improvements, along with long term test coverage and quality control changes. Highlights: * a long term performance bug with Ord instances, involving very small strings, and Data.Map has been squashed. * everything's a little faster -- shootout problems showed a 1-5% speedup just by switching to the new library. Thanks goes to the Hac4 Haskell Hackathon organisers, in Gothenburg, Sweden, where the majority of this work to create this release took place. Key changes: * Data.Map short key performance greatly improved: - 'words Map' running time: 6.310s bytestring 0.9.0.1 1.071s bytestring 0.9.1.0 * Uses cheaper unsafeDupablePerformIO for allocation. - tail recursive tight loops (fixes obscure stack overflow) * Generally faster: - Shootout sum-file: 1.218s to 1.190s - Shooout fasta: 9.210s to 8.811s * 4-5x faster small substring search (breakSubstring/findSubstring/isInfixOf). * Extensive QuickCheck coverage reporting and improvements: - http://code.haskell.org/~dons/tests/bytestring/hpc_index.html Get the code: http://hackage.haskell.org/cgi-bin/hackage-scripts/package/bytestring Note that if you upgrade to the new bytestring release, older packages built against previous releases will still require the old bytestring package. For best results, rebuild any bytestring-depending packages against the new library only. Cheers, Don That's all good news; will this release of ByteString be used for GHC 6.8.3? I'm a little tired of linking everything against 0.9.0.1 just so I can use Yi (since GHC/the-GHC-API links against it). :) -- gwern NSDM USP Edens SAS kibo quarter NSES Gamma MP5k threat pgp4XMG2ELfSd.pgp Description: PGP signature ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] letting go of file handles and Data.Binary
FWIW, installed bytestring-0.9.1.0, ran ghc-pkg hide bytestring-0.9.0.1, recompiled and reinstalled binary-0.4.1. then i played around with all that you suggested, and came to the conclusion that i don't understand seq! import Control.Exception (bracket) import System.Directory import System.IO import Data.Binary import Data.ByteString.Lazy as B strictDecodeFile :: Binary a = FilePath - (a - b) - IO () strictDecodeFile path force = bracket (openFile path ReadMode) hClose $ \h - do c - B.hGetContents h force (decode c) `seq` return () strictDecodeFile' :: Binary a = FilePath - (a - IO b) - IO () strictDecodeFile' path force = bracket (openFile path ReadMode) hClose $ \h - do c - B.hGetContents h force (decode c) return () main = do let dat = [1..10]::[Int] fname = foo.dat encodeFile fname dat h - openFile fname ReadMode c - B.hGetContents h let dat2 = decode c print (dat == dat2) hClose h removeFile fname encodeFile fname dat strictDecodeFile fname (\x - do print strict 1 print (x == dat)) removeFile fname encodeFile fname dat strictDecodeFile' fname (\x - do print strict 2 print (x == dat)) removeFile fname encodeFile fname dat dat4 - decodeFile fname print (dat == dat4) removeFile fname running main outputs True strict 2 True True *** Exception: foo.dat: removeFile: permission denied (Permission denied) e.g. the handle version works, Bryan's original strictDecodeFile appears to not run force, the modified strictDecodeFile' does run force (i didn't use seq, just an additional line in the monad), and the encodeFile / decodeFile / removeFile appears to still not work with the latest bytestring. what's the difference between the seq and non-seq versions? for now i can use strictDecodeFile' but at least something should be said in the docs about decodeFile et al holding handles. (i understand this is not the fault of binary per se as much as haskell's non-strict semantics, but a reminder for noobs like me would be helpful.) and finally something like strictDecodeFile' might be useful in the library? thanks for the help, ben On Sun, Apr 20, 2008 at 2:34 PM, Duncan Coutts [EMAIL PROTECTED] wrote: On Sun, 2008-04-20 at 14:24 -0700, Bryan O'Sullivan wrote: Doh! For all that I wrote about encodeFile, substitute decodeFile. Indeed the version of encodeFile you wrote should be exactly identical to the original because the lazy bytestring writeFile already uses bracket like that: writeFile :: FilePath - ByteString - IO () writeFile f txt = bracket (openBinaryFile f WriteMode) hClose (\hdl - hPut hdl txt) strictDecodeFile :: Binary a = FilePath - (a - b) - IO () strictDecodeFile path force = bracket (openFile path ReadMode) hClose $ \h - do c - L.hGetContents h force (decode c) `seq` return () Yes, the problem with Ben's program was that decodeFile is lazily reading the file and lazily decoding. If the decoding consumes all the input then it should be possible to avoid rewriting decodeFile and use: dat2 - decodeFile fname evaluate dat2 removeFile fname It's not immediately clear to me if we can make the decodeFile behave like your version. I'd have to go think about whether running the Get monad can lazily return values or if it always consumes all the input it'll ever need. Duncan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Announce: bytestring 0.9.1.0
That's all good news; will this release of ByteString be used for GHC 6.8.3? I'm a little tired of linking everything against 0.9.0.1 just so I can use Yi (since GHC/the-GHC-API links against it). :) Indeed; this is the biggest issue I have with bytestring right now as it's interfered with my work with hs-plugins/GHC-api, especially considering I think the new cabal and ghc 6.8.3 should fix or at least warn about the library-version-mismatch issues (from what I've heard.) The only probable fix I can think of (other than doing compile-time hackery on both the C and haskell side to make symbol names differ across releases, which is fairly infeasible i'd think and would regardless kill portability since template-haskell is the only viable option on the haskell note,) is to factor out the usage of bytestring in the GHC-API, or just stick the source code to bytestring into the GHC source tree so it's not built as a package (so you wouldn't see it in, e.g. 'ghc-pkg describe ghc', although I would think the names would still show up in the symbol table, regardless.) But whatever route you choose, I'm not sure of the ramifications in general and it seems to be a tough cookie to solve properly, so we don't end up breaking things as we try and fix them. Don probably has better ideas. -- It was in the days of the rains that their prayers went up, not from the fingering of knotted prayer cords or the spinning of prayer wheels, but from the great pray-machine in the monastery of Ratri, goddess of the Night. Roger Zelazny ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Announce: bytestring 0.9.1.0
mad.one: That's all good news; will this release of ByteString be used for GHC 6.8.3? I'm a little tired of linking everything against 0.9.0.1 just so I can use Yi (since GHC/the-GHC-API links against it). :) Indeed; this is the biggest issue I have with bytestring right now as it's interfered with my work with hs-plugins/GHC-api, especially considering I think the new cabal and ghc 6.8.3 should fix or at least warn about the library-version-mismatch issues (from what I've heard.) The only probable fix I can think of (other than doing compile-time hackery on both the C and haskell side to make symbol names differ across releases, which is fairly infeasible i'd think and would regardless kill portability since template-haskell is the only viable option on the haskell note,) is to factor out the usage of bytestring in the GHC-API, or just stick the source code to bytestring into the GHC source tree so it's not built as a package (so you wouldn't see it in, e.g. 'ghc-pkg describe ghc', although I would think the names would still show up in the symbol table, regardless.) But whatever route you choose, I'm not sure of the ramifications in general and it seems to be a tough cookie to solve properly, so we don't end up breaking things as we try and fix them. Don probably has better ideas. The use of bytestring inside GHC is limited only to a little bit in the GHCi modules -- and could easily be replaced, I suspect. Doing so would remove one dependency from GHC's core, as well as making it easier to upgrade bytestring versions. Ian, have you looked at this? -- Don ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Announce: bytestring 0.9.1.0
On Sun, 2008-04-20 at 19:05 -0500, Austin Seipp wrote: That's all good news; will this release of ByteString be used for GHC 6.8.3? I'm a little tired of linking everything against 0.9.0.1 just so I can use Yi (since GHC/the-GHC-API links against it). :) Indeed; this is the biggest issue I have with bytestring right now as it's interfered with my work with hs-plugins/GHC-api, especially considering I think the new cabal and ghc 6.8.3 should fix or at least warn about the library-version-mismatch issues (from what I've heard.) Cabal-1.4 does warn at configure time if a build is going to use inconsistent versions of dependent libraries. Trying to come up with an installation plan that avoids the problem is a good deal harder and in general isn't possible without having to rebuild lots of unrelated packages. The only probable fix I can think of (other than doing compile-time hackery on both the C and haskell side to make symbol names differ across releases, All haskell code does work this way, ghc puts the package name and version into the symbol name so it is possible to link several versions of a package into one program. The particular problem for bytestring is the C code that it uses. So while Data.ByteString.foo get's mapped to a symbol name something like bytestring_0_9_1_0_DatazByteStringzfoo the embedded C code gets no such mangling so fps_reverse from bytestring-0.9.0.1 clashes with fps_reverse from bytestring-0.9.0.4. Actually the gnu ld linker doesn't mind about the duplicate symbols and just picks ones of them. The ghci linker is a bit more paranoid and rejects the duplicates. I guess we could try and adjust the names of the C symbols to include the package name and version. Duncan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] letting go of file handles and Data.Binary
Am Montag, 21. April 2008 01:35 schrieb Ben: FWIW, installed bytestring-0.9.1.0, ran ghc-pkg hide bytestring-0.9.0.1, recompiled and reinstalled binary-0.4.1. then i played around with all that you suggested, and came to the conclusion that i don't understand seq! import Control.Exception (bracket) import System.Directory import System.IO import Data.Binary import Data.ByteString.Lazy as B strictDecodeFile :: Binary a = FilePath - (a - b) - IO () strictDecodeFile path force = bracket (openFile path ReadMode) hClose $ \h - do c - B.hGetContents h force (decode c) `seq` return () This means that force (decode c) is reduced to head normal form, not fully evaluated. So if the result type of force were e.g. Either String Int, it would be evaluated just far enough to determine if force (decode c) is bottom, Left something or Right somethingelse; Left undefined and Right undefined won't cause an exception. In the case below, force (decode c) is an IO action, it will be evaluated as far as necessary to see it's (IO whatever) and not bottom, for that it need not be run, therefore you don't see strict 1 printed out. strictDecodeFile' :: Binary a = FilePath - (a - IO b) - IO () strictDecodeFile' path force = bracket (openFile path ReadMode) hClose $ \h - do c - B.hGetContents h force (decode c) return () Here you say you want the IO action force (decode c) to be run, so it is run and strict 2 is printed out. Hope that helps. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] letting go of file handles and Data.Binary
one more piece of email pollution: import Control.Exception (bracket) import System.IO import Data.Binary import qualified Data.ByteString.Lazy as B strictDecodeFile :: Binary a = FilePath - IO a strictDecodeFile path = bracket (openBinaryFile path ReadMode) hClose $ \h - do c - B.hGetContents h return $! decode c seems to work like Data.Binary.decodeFile but explicitly closes the handle. take care, ben On Sun, Apr 20, 2008 at 4:35 PM, Ben [EMAIL PROTECTED] wrote: FWIW, installed bytestring-0.9.1.0, ran ghc-pkg hide bytestring-0.9.0.1, recompiled and reinstalled binary-0.4.1. then i played around with all that you suggested, and came to the conclusion that i don't understand seq! import Control.Exception (bracket) import System.Directory import System.IO import Data.Binary import Data.ByteString.Lazy as B strictDecodeFile :: Binary a = FilePath - (a - b) - IO () strictDecodeFile path force = bracket (openFile path ReadMode) hClose $ \h - do c - B.hGetContents h force (decode c) `seq` return () strictDecodeFile' :: Binary a = FilePath - (a - IO b) - IO () strictDecodeFile' path force = bracket (openFile path ReadMode) hClose $ \h - do c - B.hGetContents h force (decode c) return () main = do let dat = [1..10]::[Int] fname = foo.dat encodeFile fname dat h - openFile fname ReadMode c - B.hGetContents h let dat2 = decode c print (dat == dat2) hClose h removeFile fname encodeFile fname dat strictDecodeFile fname (\x - do print strict 1 print (x == dat)) removeFile fname encodeFile fname dat strictDecodeFile' fname (\x - do print strict 2 print (x == dat)) removeFile fname encodeFile fname dat dat4 - decodeFile fname print (dat == dat4) removeFile fname running main outputs True strict 2 True True *** Exception: foo.dat: removeFile: permission denied (Permission denied) e.g. the handle version works, Bryan's original strictDecodeFile appears to not run force, the modified strictDecodeFile' does run force (i didn't use seq, just an additional line in the monad), and the encodeFile / decodeFile / removeFile appears to still not work with the latest bytestring. what's the difference between the seq and non-seq versions? for now i can use strictDecodeFile' but at least something should be said in the docs about decodeFile et al holding handles. (i understand this is not the fault of binary per se as much as haskell's non-strict semantics, but a reminder for noobs like me would be helpful.) and finally something like strictDecodeFile' might be useful in the library? thanks for the help, ben On Sun, Apr 20, 2008 at 2:34 PM, Duncan Coutts [EMAIL PROTECTED] wrote: On Sun, 2008-04-20 at 14:24 -0700, Bryan O'Sullivan wrote: Doh! For all that I wrote about encodeFile, substitute decodeFile. Indeed the version of encodeFile you wrote should be exactly identical to the original because the lazy bytestring writeFile already uses bracket like that: writeFile :: FilePath - ByteString - IO () writeFile f txt = bracket (openBinaryFile f WriteMode) hClose (\hdl - hPut hdl txt) strictDecodeFile :: Binary a = FilePath - (a - b) - IO () strictDecodeFile path force = bracket (openFile path ReadMode) hClose $ \h - do c - L.hGetContents h force (decode c) `seq` return () Yes, the problem with Ben's program was that decodeFile is lazily reading the file and lazily decoding. If the decoding consumes all the input then it should be possible to avoid rewriting decodeFile and use: dat2 - decodeFile fname evaluate dat2 removeFile fname It's not immediately clear to me if we can make the decodeFile behave like your version. I'd have to go think about whether running the Get monad can lazily return values or if it always consumes all the input it'll ever need. Duncan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] letting go of file handles and Data.Binary
Am Montag, 21. April 2008 02:34 schrieb Daniel Fischer: This means that force (decode c) is reduced to head normal form, not fully evaluated. Ooops, _weak_ head normal form, of course. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] letting go of file handles and Data.Binary
Ben wrote: i played around with all that you suggested, and came to the conclusion that i don't understand seq! That's certainly possible, but you also got the type of your first forcing function wrong :-) strictDecodeFile :: Binary a = FilePath - (a - b) - IO () encodeFile fname dat strictDecodeFile fname (\x - do print strict 1 print (x == dat)) removeFile fname You provided a function that returns an IO action. Note that strictDecodeFile ensures that the result of force is evaluated to WHNF, but it doesn't know what the *type* of the result is. If you return an IO action, there's no way it can be run, because the caller can't even tell that you returned an IO action: it just knows that you returned something of an unknown type b. If the IO action can't be run, the comparison inside can't be performed, and so nothing useful actually happens. Instead, just provide a pure function to force the decoding: something as simple as (==dat) will do. Evaluating this to WHNF will reduce it to the constructor True or False. In order to produce a constructor, enough of the decoded data should be demanded to suit your needs. Developing a good enough mental model of how laziness works is a very useful way to spend some time. b ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
