Send Beginners mailing list submissions to
beginners@haskell.org
To subscribe or unsubscribe via the World Wide Web, visit
http://www.haskell.org/mailman/listinfo/beginners
or, via email, send a message with subject or body 'help' to
beginners-requ...@haskell.org
You can reach the person managing the list at
beginners-ow...@haskell.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Beginners digest..."
Today's Topics:
1. problem with System.Directory.Tree (Anand Mitra)
2. Re: problem with System.Directory.Tree (Daniel Fischer)
3. Trying around with Functional Reactive Programming
(Nathan Huesken)
4. Re: Trying around with Functional Reactive Programming
(Jorden M)
5. web server availability (Paul Higham)
6. Re: lazy IO in readFile (Andrew Sackville-West)
7. Best way to stop listening on a port. (aditya siram)
8. Re: Best way to stop listening on a port. (Patrick LeBoutillier)
9. Re: problem with System.Directory.Tree (Drew Haven)
----------------------------------------------------------------------
Message: 1
Date: Mon, 7 Jun 2010 17:36:22 +0530
From: Anand Mitra <anand.mi...@gmail.com>
Subject: [Haskell-beginners] problem with System.Directory.Tree
To: beginners@haskell.org
Message-ID:
<aanlktikpor3w1t-ij6laompnz6khiy3j6kj88q4ct...@mail.gmail.com>
Content-Type: text/plain; charset="iso-8859-1"
Hello All,
I want to build a program which will recursively scan a directory and
build md5sum for all the files. The intent is to do something similar
to unison but more specific to my requirements. I am having trouble in
the initial part of building the md5sums.
I did some digging around and found that "System.Directory.Tree" is a
very close match for what I want to do. In fact after a little poking
around I could do exactly what I wanted.
,----
| import Monad
| import System.Directory.Tree
| import System.Directory
| import Data.Digest.Pure.MD5
| import qualified Data.ByteString.Lazy.Char8 as L
|
| calcMD5 =
| readDirectoryWith (\x-> liftM md5 (L.readFile x))
`----
This work perfectly for small directories. readDirectoryWith is
already defined in the library and exactly what we want
,----
| *Main> calcMD5 "/home/mitra/Desktop/"
|
| "/home/mitra" :/ Dir {name = "Desktop", contents = [File {name =
| "060_LocalMirror_Workflow.t.10.2.62.9.log", file =
| f687ad04bc64674134e55c9d2a06902a},File {name = "cmd_run", file =
| 6f334f302b5c0d2028adeff81bf2a0d9},File {name = "cmd_run~",
`----
However when ever I give it something more challenging it gets into
trouble.
,----
| *Main> calcMD5 "/home/mitra/laptop/"
| *** Exception: /home/mitra/laptop/ell/calc-2.02f/calc.info-27:
| openFile: resource exhausted (Too many open files)
| *Main> 29~
`----
If I understand what is happening it seems to be doing all the opens
before consuming them via md5. This works fine for small directories
but for any practical setup this could potentially be very large. I
tried forcing the md5 evaluation in the hope that the file descriptor
will be freed once the entire file is read. That did not help, either
because I could not get it right or there is some more subtle I am
missing.
I also had a look at the code in module "System.Directory.Tree" and
although it gave me some understanding of how it works I am no closer
to a solution.
regards
--
Anand Mitra
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://www.haskell.org/pipermail/beginners/attachments/20100607/28ee5ff9/attachment-0001.html
------------------------------
Message: 2
Date: Mon, 7 Jun 2010 22:01:33 +0200
From: Daniel Fischer <daniel.is.fisc...@web.de>
Subject: Re: [Haskell-beginners] problem with System.Directory.Tree
To: beginners@haskell.org
Cc: Anand Mitra <anand.mi...@gmail.com>
Message-ID: <201006072201.33621.daniel.is.fisc...@web.de>
Content-Type: text/plain; charset="utf-8"
On Monday 07 June 2010 14:06:22, Anand Mitra wrote:
> Hello All,
>
> I want to build a program which will recursively scan a directory and
> build md5sum for all the files. The intent is to do something similar
> to unison but more specific to my requirements. I am having trouble in
> the initial part of building the md5sums.
>
> I did some digging around and found that "System.Directory.Tree" is a
> very close match for what I want to do. In fact after a little poking
> around I could do exactly what I wanted.
>
> ,----
>
> | import Monad
> | import System.Directory.Tree
> | import System.Directory
> | import Data.Digest.Pure.MD5
> | import qualified Data.ByteString.Lazy.Char8 as L
> |
> | calcMD5 =
> | readDirectoryWith (\x-> liftM md5 (L.readFile x))
Does
calcMD5 =
readDirectoryWith (\x -> do
txt <- readFile x
return $! md5 txt)
help?
>
> `----
>
> This work perfectly for small directories. readDirectoryWith is
> already defined in the library and exactly what we want
>
> ,----
>
> | *Main> calcMD5 "/home/mitra/Desktop/"
> |
> | "/home/mitra" :/ Dir {name = "Desktop", contents = [File {name =
> | "060_LocalMirror_Workflow.t.10.2.62.9.log", file =
> | f687ad04bc64674134e55c9d2a06902a},File {name = "cmd_run", file =
> | 6f334f302b5c0d2028adeff81bf2a0d9},File {name = "cmd_run~",
>
> `----
>
> However when ever I give it something more challenging it gets into
> trouble.
>
> ,----
>
> | *Main> calcMD5 "/home/mitra/laptop/"
> | *** Exception: /home/mitra/laptop/ell/calc-2.02f/calc.info-27:
> | openFile: resource exhausted (Too many open files)
> | *Main> 29~
>
> `----
>
> If I understand what is happening it seems to be doing all the opens
> before consuming them via md5. This works fine for small directories
> but for any practical setup this could potentially be very large. I
> tried forcing the md5 evaluation in the hope that the file descriptor
> will be freed once the entire file is read. That did not help, either
> because I could not get it right or there is some more subtle I am
> missing.
>
> I also had a look at the code in module "System.Directory.Tree" and
> although it gave me some understanding of how it works I am no closer
> to a solution.
>
> regards
------------------------------
Message: 3
Date: Sun, 6 Jun 2010 13:42:10 -0400
From: Nathan Huesken <hask...@lonely-star.org>
Subject: [Haskell-beginners] Trying around with Functional Reactive
Programming
To: Biginners Haskell Mailinglist <beginners@haskell.org>
Message-ID: <20100606134210.1e0bd...@samzwo>
Content-Type: text/plain; charset=US-ASCII
Hi,
I tried to read the article "Simple efficient functional reactivity".
Being a beginner to haskell, I had my problems. I decided to try with
around with some implementations first. I found this:
http://conal.net/fran/
Unfortantlly it is windows only, I am on gentoo linux.
Is there a linux compatible implementation I can experiment with?
Thanks!
Nathan
------------------------------
Message: 4
Date: Mon, 7 Jun 2010 16:15:36 -0400
From: Jorden M <jrm8...@gmail.com>
Subject: Re: [Haskell-beginners] Trying around with Functional
Reactive Programming
To: Nathan Huesken <hask...@lonely-star.org>
Cc: Biginners Haskell Mailinglist <beginners@haskell.org>
Message-ID:
<aanlktinonzxfft_vspfoilbjooqhren3cxk7onhyi...@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
On Sun, Jun 6, 2010 at 1:42 PM, Nathan Huesken <hask...@lonely-star.org> wrote:
> Hi,
>
> I tried to read the article "Simple efficient functional reactivity".
> Being a beginner to haskell, I had my problems. I decided to try with
> around with some implementations first. I found this:
>
> http://conal.net/fran/
>
> Unfortantlly it is windows only, I am on gentoo linux.
>
> Is there a linux compatible implementation I can experiment with?
Grapefruit is one.
>
> Thanks!
> Nathan
> _______________________________________________
> Beginners mailing list
> Beginners@haskell.org
> http://www.haskell.org/mailman/listinfo/beginners
>
------------------------------
Message: 5
Date: Sun, 6 Jun 2010 15:43:10 -0700
From: Paul Higham <polyg...@mac.com>
Subject: [Haskell-beginners] web server availability
To: beginners@haskell.org
Message-ID: <73eb03ff-0bbc-4eff-8a74-4840b2b48...@mac.com>
Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
Is anyone else having trouble reaching the haskell.org web site? I
have been unsuccessful in trying to reach it for over a week. Does
anybody know what's going on?
::paul
------------------------------
Message: 6
Date: Sat, 5 Jun 2010 14:12:26 -0700
From: Andrew Sackville-West <and...@swclan.homelinux.org>
Subject: Re: [Haskell-beginners] lazy IO in readFile
To: beginners@haskell.org
Message-ID: <20100605211225.gk17...@basement.swclan.homelinux.org>
Content-Type: text/plain; charset="iso-8859-1"
On Wed, Jun 02, 2010 at 06:40:15PM +0200, Chaddaï Fouché wrote:
> On Thu, May 20, 2010 at 3:17 AM, Andrew Sackville-West
> <and...@swclan.homelinux.org> wrote:
> > thanks for this. it helps a lot. hmmm... I wonder why it is I never
> > have a problem returning functions in Scheme, but it never occurs to
> > me as I learn Haskell?
>
> Maybe you're already doing it without realizing it ? For instance for
> the same kind of problem but without the IO part, the type of the
> function could be :
> > getIsNewItemPredicate :: stuff -> (String -> Bool)
> but in normal Haskell, you wouldn't write this last pair of
> parenthesis (since they're implicit) :
> > getIsNewItemPredicate :: stuff -> String -> Bool
> And so it is pretty likely that you would write this function just as
> if it had two parameters :
> > getIsNewItemPredicate stuff str = .... str `isMember` set
> And later on use the fact that the function is curried to get a
> predicate on String :
> > let isNewItem = getIsNewItemPredicate someStuff
>
> In this case, you're "returning" a function but it may not be as
> obvious as in Scheme (where curryfication is not an idiom encouraged
> by the language).
very insightful observation. I do indeed do it, but not conciously as
would be the case in a language with Scheme.
A
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
Url :
http://www.haskell.org/pipermail/beginners/attachments/20100607/c4f8149b/attachment-0001.bin
------------------------------
Message: 7
Date: Fri, 4 Jun 2010 13:59:22 -0500
From: aditya siram <aditya.si...@gmail.com>
Subject: [Haskell-beginners] Best way to stop listening on a port.
To: beginners <beginners@haskell.org>
Message-ID:
<aanlktimaixuketobuwsadjvyo9uj31fc_w0zgp_-d...@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Hi all,
I've been writing some server code like the following:
Control.Exception.bracketOnError (listenOn port) sClose procRequests
Sometimes I run this server in GHCI and interrupt it with C-c. But
when I try and rerun the server it tells me that the port is already
bound meaning that sClose either doesn't get called or doesn't
complete. Terminating the interpreter seems to work.
Is there a better way to correctly stop listening on a port?
-deech
------------------------------
Message: 8
Date: Mon, 7 Jun 2010 18:09:28 -0400
From: Patrick LeBoutillier <patrick.leboutill...@gmail.com>
Subject: Re: [Haskell-beginners] Best way to stop listening on a port.
To: aditya siram <aditya.si...@gmail.com>
Cc: beginners <beginners@haskell.org>
Message-ID:
<aanlktil4-xxkt3wflqjs5g4ohsn1cbyi1iffegwtj...@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
Hi,
Could this have something to do with SO_REUSEADDR?
This could be the case if you interrupt the server while some clients
are connected to it.
If so, I think the Haskell way to set this is: setSocketOption socket
ReuseAddr 1
Patrick
On Fri, Jun 4, 2010 at 2:59 PM, aditya siram <aditya.si...@gmail.com> wrote:
> Hi all,
> I've been writing some server code like the following:
> Control.Exception.bracketOnError (listenOn port) sClose procRequests
>
> Sometimes I run this server in GHCI and interrupt it with C-c. But
> when I try and rerun the server it tells me that the port is already
> bound meaning that sClose either doesn't get called or doesn't
> complete. Terminating the interpreter seems to work.
>
> Is there a better way to correctly stop listening on a port?
> -deech
> _______________________________________________
> Beginners mailing list
> Beginners@haskell.org
> http://www.haskell.org/mailman/listinfo/beginners
>
--
=====================
Patrick LeBoutillier
Rosemère, Québec, Canada
------------------------------
Message: 9
Date: Mon, 7 Jun 2010 16:15:21 -0700
From: Drew Haven <drew.ha...@gmail.com>
Subject: Re: [Haskell-beginners] problem with System.Directory.Tree
To: Anand Mitra <anand.mi...@gmail.com>
Cc: beginners@haskell.org
Message-ID:
<aanlktina-cdsgszuomxcgb2hnpl4q8mgwrbh1excq...@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
I did something similar where I built up an md5sum of all the files in
a directory for comparing whether two directories were identical (I
was cleaning up some server storage). One difference is that I only
read the first 4096 bytes of the file because if files are going to
differ they will likely differ in those bytes (and definitely would in
my case) and that is the default page read size is I recall, so even
if you use hGet handle 512, the system still reads 4192 bytes into
memory anyway, so why not use them.
I think I had a similar problem to yours with open file handles until
i used `withFile` from System.IO. This handy function took care of
closing up file resources for me so I wouldn't have a ton of open file
handles. My getFileHash function is as follows:
getFileHash :: FilePath -> IO (Maybe String)
getFileHash path =
(do
contents <- withFile path ReadMode (\h -> hGet h 4096)
return . Just $! md5sum contents)
`catch` (\e -> printFileError e
>> return Nothing)
printFileError is just a function for printing out pretty errors
related to files.
You can see that it reads some contents of the file through withFile
and then md5sums them. I have the $! to force evaluation so it will
compute as we go, otherwise it builds a huge tree of sums waiting to
be computed before computing the result for display at the root.
There are other $! operators in the tree operations to collapse at that level,
and now the program runs in constant memory space.
--
Drew Haven
drew.ha...@gmail.com
On Mon, Jun 7, 2010 at 5:06 AM, Anand Mitra <anand.mi...@gmail.com> wrote:
> Hello All,
>
> I want to build a program which will recursively scan a directory and
> build md5sum for all the files. The intent is to do something similar
> to unison but more specific to my requirements. I am having trouble in
> the initial part of building the md5sums.
>
> I did some digging around and found that "System.Directory.Tree" is a
> very close match for what I want to do. In fact after a little poking
> around I could do exactly what I wanted.
>
> ,----
> | import Monad
> | import System.Directory.Tree
> | import System.Directory
> | import Data.Digest.Pure.MD5
> | import qualified Data.ByteString.Lazy.Char8 as L
> |
> | calcMD5 =
> | readDirectoryWith (\x-> liftM md5 (L.readFile x))
> `----
>
> This work perfectly for small directories. readDirectoryWith is
> already defined in the library and exactly what we want
>
> ,----
> | *Main> calcMD5 "/home/mitra/Desktop/"
> |
> | "/home/mitra" :/ Dir {name = "Desktop", contents = [File {name =
> | "060_LocalMirror_Workflow.t.10.2.62.9.log", file =
> | f687ad04bc64674134e55c9d2a06902a},File {name = "cmd_run", file =
> | 6f334f302b5c0d2028adeff81bf2a0d9},File {name = "cmd_run~",
> `----
>
> However when ever I give it something more challenging it gets into
> trouble.
>
> ,----
> | *Main> calcMD5 "/home/mitra/laptop/"
> | *** Exception: /home/mitra/laptop/ell/calc-2.02f/calc.info-27:
> | openFile: resource exhausted (Too many open files)
> | *Main> 29~
> `----
>
> If I understand what is happening it seems to be doing all the opens
> before consuming them via md5. This works fine for small directories
> but for any practical setup this could potentially be very large. I
> tried forcing the md5 evaluation in the hope that the file descriptor
> will be freed once the entire file is read. That did not help, either
> because I could not get it right or there is some more subtle I am
> missing.
>
> I also had a look at the code in module "System.Directory.Tree" and
> although it gave me some understanding of how it works I am no closer
> to a solution.
>
> regards
> --
> Anand Mitra
>
>
> _______________________________________________
> Beginners mailing list
> Beginners@haskell.org
> http://www.haskell.org/mailman/listinfo/beginners
>
>
------------------------------
_______________________________________________
Beginners mailing list
Beginners@haskell.org
http://www.haskell.org/mailman/listinfo/beginners
End of Beginners Digest, Vol 24, Issue 5
****************************************
