d: > Hi all, > > I'm going to start a project where I'll have to do some data analysis > (statistics about product orders) based on database entries; it will > mostly be some very basic stuff like grouping by certain rules and > finding averages as well as summing up and such. It will however be > more than what can be done directly in the database using SQL, so there > will be some processing in my program. > > I'm thinking about trying to do this in Haskell (because I like this > language a lot); however, it is surely not my most proficient language > and I tried to do some number crunching (real one that time) before in > Haskell where I had to deal with some 4 million integer lists, and this > failed; the program took a lot more memory than would have been > necessary and ran for several minutes (kept swapping all the time, too). > A rewrite in Fortran did give the result in 6s and didn't run out of > space.
**Don't use lists when you mean to use arrays** E.g. multiple two 4M element arrays, map over the result and sum that. import Data.Array.Vector main = print . sumU . mapU (+7) $ zipWithU (*) (enumFromToU 1 (4000000 :: Int)) (enumFromToU 2 (4000001 :: Int)) Compile it: $ ghc -O2 -fvia-C -optc-O3 -funbox-strict-fields --make $ time ./A 2886605259654448384 ./A 0.03s user 0.00s system 97% cpu 0.034 total Not the end of the world at all. > This was probably my fault at that time, because I surely did something > completely wrong for the Haskell style. However, I fear I could run > into problems like that in the new project, too. So I want to ask for > your opinions, do you think Haskell is the right language to do data You want to compile Haskell DB queries into SQL? > analysis of this kind? And do you think it is hard for still beginner > Haskell programmer to get this right so Haskell does not use up a lot of > memory for thunks or list-overhead or things like that? And finally, > are there database bindings for Haskell I could use for the queries? There are lots of database bindings. Very popular ones are HDBC and Takusen. Check on hackage.haskell.org -- Donnn _______________________________________________ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe