Re: [Haskell-cafe] Re: Efficient way to edit a file

[Bulat Ziganshin]

Hello Maurício,

Tuesday, June 20, 2006, 12:59:47 AM, you wrote:

  I need to edit big text files (5 to 500 Mb). But I just need to
change one or two small lines, and save it. What is the best way to do 
that in Haskell, without creating copies of the whole files?
I like very much the idea of memory mapping the file. In some
 situations, random access to the file would help a lot. Can I do that on
 Windows?

i wanted to write what my Streams 0.2 library (see previous letter)
also supports memory-mapped files on Windows (see Examples/wc4MMFile.hs),
but i'm not sure what you really need it. if your edit don't involve
moving of data (i.e. new lines will have the same size as overwritten
ones), you don't need to use mmfile because it's enough to open file
in ReadWriteMode (see openBinaryFile call description). if you need
to change file size, then you can just read it into large buffer,
change all that you want and write it back.


-- 
Best regards,
 Bulatmailto:[EMAIL PROTECTED]

___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe

[Haskell-cafe] Re: Efficient way to edit a file

[Maurício]

David Roundy wrote:

On Fri, Jun 02, 2006 at 12:34:51PM +1000, Donald Bruce Stewart wrote:


dons:


briqueabraque:


 Hi,

 I need to edit big text files (5 to 500 Mb). But I just need to 
change one or two small lines, and save it. What is the best way to do 
that in Haskell, without creating copies of the whole files?




Thinking further, since you want to avoid copying on the disk, you need
to be able to keep the edited version in memory. So the strict
bytestring would be best, for example:



dons is right here, but I'd add that it's hard to safely edit a big
file without creating a copy, if you want your program to leave the
file in a consistent state even if it crashes (power failure, kill,
file server failure).  dons' suggestion could leave you with a deleted
file (if power goes down at the beginning of a write).  If you aren't
changing the size of the file, opening it ReadWrite will allow you to
modify it reasonably safeily.  If you *are* changing its size, then
doing something explicit would probably be the way to go (and I'd
probably actually use mmap and memmove to make the change, if you do
need to modify the file size).  But then, I'm thinking posix (as I
generally do), which may not be the case for you.  And perhaps you
don't need to be careful.  I've found that if bad things can happen,
they do.  But that's largely because darcs has lots of users...


  I like very much the idea of memory mapping the file. In some 
situations, random access to the file would help a lot. Can I do that on 
Windows?
  Also: safety is not a concern. I can delete those big files as much 
as I need, they are only temporary data.


  Maurício

___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe