On Tue, Feb 24, 2009 at 11:36 AM, Gwern Branwen <[email protected]> wrote: > On Tue, Feb 24, 2009 at 2:43 AM, Jean-Philippe Bernardy > <[email protected]> wrote: >> >> On Tue, Feb 24, 2009 at 5:00 AM, Gwern Branwen <[email protected]> wrote: >> >>> After a fair bit of hacking, that turned out to be a workable >>> solution. I just stored ArticleDB in it (don't need anything else). >>> Had a bit of trouble working out the monads and was shortly flummoxed >>> by the Initializable stuff >> >>>(until I realized that that typeclass was >>> how the case of no-defined-variable was handled, by calling the >>> 'initial' function) >> >> It would help if you documented the parts that were opaque to you; >> a patch with haddock comments would be great. > > I'll see if I can do some documenting this week. It's not all that > clear where the Haddocks would go, though - one can't really use the > Buffer.Misc functions without putA/getA, and I'm not really sure what > they do without those functions. Do I put the comment about what > Initializable means by the typeclass or somewhere else? > >>> But the initial load is still several seconds. How would Data.Binary >>> help there? Is it intended to be a faster version of '(read $ readFile >>> foo) :: ArticleDB)'? >> >> Yep. >> >> Cheers, >> -- JP. > > I've been looking, and it looks like that might work. At least, > Data.Binary claims > http://hackage.haskell.org/packages/archive/binary/0.5/doc/html/Data-Binary.html > to have a 'Binary e => Binary (Seq e)', and I'm sure there's an > instance for String/[Char]. I wonder whether binary-strict might not > be better though?
Think some sort of strictness may be required. I tried switching to
Data.Binary, and made the following changes:
hunk ./Yi/IReader.hs 11
+import Data.Binary
hunk ./Yi/IReader.hs 22
-import qualified Data.ByteString.Char8 as B (pack, unpack, readFile, writeFile)
hunk ./Yi/IReader.hs 51
-writeDB adb = do io . forkIO . join . fmap (flip B.writeFile $ B.pack
$ show adb) $ dbLocation
+writeDB adb = do io . forkIO . join . fmap (flip encodeFile adb) $ dbLocation
hunk ./Yi/IReader.hs 56
-readDB = io $ rddb `catch` (const $ return empty) -- May seem silly
to read in as bytestring
- where rddb = fmap B.readFile dbLocation >>= fmap (read .
B.unpack) -- and then unpack it, $
-
-- but - is strict
+readDB = io (dbLocation >>= decodeFile) `catch` (\_ -> return empty))
This compiles without issue, but when one recompiles yi and runs M-x
iread, one gets a horrible error:
'M-x ireadyi-i386-linux: /home/gwern/.yi/articles.db: openBinaryFile:
resource busy (file is locked)'
I would think this just a lazy IO issue but for the fact that Yi seems
to loop and begin eating up as much memory as possible until the OS
kills it. (First screenshot.)
Then I reasoned - perhaps Data.Binary has a different on-disk format
than Show, and the issue is that yi is trying to read a show-file as a
binary-file. So I went into GHCi, manually read in and parsed
articles.db, and wrote it back out with encodeFile. Sure enough, the
format was indeed different. I restarted yi, and tried again. No dice!
I get a locked error and then a 'error: stack overflow'. Oy.
So I began bugging #haskell and Saizan gave me an ugly ByteString
recipe which worked. No idea why - my extensive tinkering in GHCi
didn't give me much of a intuition why I could do something like
'encodeFile foo; bar <- decodeFile "foo"; foo == bar' and get a stack
overflow.
It's incorrect - there's still an issue with file locking (the
background writes don't seem to go through). But I will admit
data.binary does seem to read articles.db pretty darn quickly.
--
gwern
--~--~---------~--~----~------------~-------~--~----~
Yi development mailing list
[email protected]
http://groups.google.com/group/yi-devel
-~----------~----~----~----~------~----~------~--~---
<<inline: xwd-123553332212150.png>>
