Mark Waddingham wrote:
open file ...
repeat forever
read from file ... until return
if the result is not empty then
exit repeat
end if
if *it is a new message boundary* then
... finish processing current message ...
... start processing new boundary ...
else
... append line to current message ...
end if
end repeat
What is the size of the read buffer used when reading until <char>?
I'm assuming it isn't reading a single char per disk access, probably at
least using the file system's block size, no?
I ask because some months ago I wrote a needed to parse a 6GB file and
"read...until CR" was slower than I preferred so I experimented with a
complicated routine that reads into a buffer of about 128k and then
parses the buffer.
If I can turn up the code it may be mildly interesting, but the main
question it raised for me was:
Given that the engine is probably already doing pretty much the same
thing, would it make sense to consider a readBufferSize global property
which would govern the size of the buffer the engine uses when executing
"read...until <char>"?
In my experiments I was surprised to find that larger buffers (>10MB)
were slower than "read...until <char>", but the sweet spot seemed to be
around 128k. Presumably this has to do with the overhead of allocating
contiguous memory, and if you have any insights on that it would be
interesting to learn more.
I recognize this sort of things may seem like mere performance
fetishism, but I believe this has useful application for making LC an
ever better solution for working with large amounts of data.
Pretty much any program will read big files in chunks, and if LC can do
so optimally with all the grace and ease of "read...until <char>" it
makes one more strong set of use cases where choosing LC isn't a
tradeoff but an unquestionable advantage.
--
Richard Gaskin
Fourth World Systems
Software Design and Development for the Desktop, Mobile, and the Web
____________________________________________________________________
ambassa...@fourthworld.com http://www.FourthWorld.com
_______________________________________________
use-livecode mailing list
use-livecode@lists.runrev.com
Please visit this url to subscribe, unsubscribe and manage your subscription
preferences:
http://lists.runrev.com/mailman/listinfo/use-livecode
