It depends on how you are accessing it. If you are processing a small area of the file at a time, then moving to other areas, then paging can handle it quite nicely. But if you are hitting all parts of the file randomly then the paging can get out of hand.
The file simply becomes an extension of virtual memory. It is not necessary for the file to be copied to the swap file, but the pages of the file have to reside in real memory to be processed. The file can be considered as split into pages to be moved to real memory to be processed. If you hit lots of pages of the file then you will need lots of real memory to handle that, hitting a few memory addresses in each page. To keep down the real memory requirements, which if the requirements become large result in thrashing, try to keep references to the data in the mapped file close together. On Mon, Aug 1, 2011 at 4:50 PM, bill lam <[email protected]> wrote: > The statement in Chris' article, > > Memory-mapped files can be much larger than system RAM, yet they can be > accessed without page thrashing. > > needs further investigation in 64-bit J. You can write a dummy 10G file > using linux command, > > dd if=/dev/zero of=foo bs=1M count=10000 > > and play with it (increase file size if you get more than 10G RAM). > > Пнд, 01 Авг 2011, Steven Taylor писал(а): > > > Hi Bill, > > > > so in J does the OS still manage pulling parts of the file in / out of > > memory? > > > > optimistic case: > > Does 'size of viewport' mean that J takes x amount of the addressable > > memory, yet the OS only loads in parts relevant to the current operation? > > i.e. so long as you stay away from some safe threshold (say 1e9 * 0.66), > you > > *should* be okay? In a 32 bit OS it's about 4 GB (1e_9 * 2^32 GB)? > > > > Thanks for the info Bill, Raul, and Lettow. > > > > re: JDB. I'll come back to this when I get a spare moment. I'm not sure > I > > want to tackle a new database just yet :) > > > > re: vector article, is that available on the vector.org.uk website > > somewhere, or in the printed periodical? > > > > thanks, > > -Steven > > ---------------------------------------------------------------------- > > For information about J forums see http://www.jsoftware.com/forums.htm > > -- > regards, > ==================================================== > GPG key 1024D/4434BAB3 2008-08-24 > gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3 > ---------------------------------------------------------------------- > For information about J forums see http://www.jsoftware.com/forums.htm ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
