Re: [Flightgear-devel] RAM disk / Unix
Jon S Berndt [EMAIL PROTECTED] wrote: On Tue, 10 Feb 2004 15:06:41 -0800 Andy Ross [EMAIL PROTECTED] wrote: Jon S. Berndt wrote: It is thought that the use of a RAM disk might help. My interpretation was that their problem was latency, not I/O throughput. The program cooks along using 100% of the CPU, then it needs to load a giant data set off the disk, [...] Yes, it seems to be a latency issue. There are still 'silicon disks' that attach to a SCSI channel - very expensive but really nice. You might want to prefetch your data from the hard disk to such a silicon disk before you really use it. If the hard disk read operations are predictabe, there is an alternative in employing a CacheFS on top of XFS and increase the read-ahead number in /var/sysgen/stune (see /var/sysgen/mtune/cachefs for explanation). On the other hand it might be easier to 'preload' the data into memory and prevent this memory area to being swapped out (there should be some operating system call), Martin. -- Unix _IS_ user friendly - it's just selective about who its friends are ! -- ___ Flightgear-devel mailing list [EMAIL PROTECTED] http://mail.flightgear.org/mailman/listinfo/flightgear-devel
Re: [Flightgear-devel] RAM disk / Unix
Martin Spott wrote: On the other hand it might be easier to 'preload' the data into memory and prevent this memory area to being swapped out (there should be some operating system call), There certainly is. It's called read(). :) Andy ___ Flightgear-devel mailing list [EMAIL PROTECTED] http://mail.flightgear.org/mailman/listinfo/flightgear-devel
Re: [Flightgear-devel] RAM disk / Unix
On Dienstag, 10. Februar 2004 23:01, Jon S Berndt wrote: Is anyone aware of a RAM disk utility or feature under Unix (specifically, IRIX)? When running a simulation on IRIX we are finding that the disk access is taking too much time at various phase boundaries. It is thought that the use of a RAM disk might help. Don't know irix well enough. With Linux you can have ramdisks. You can create them a runtime or at boottime and desroy them whenever you like. Also there is the ramfs filesystem which is a kind of better ramdisk. It is a ramdisk which is not based on real disk images held in ram but it is pure dynamic. This means that it takes not more ram from main memory then required for storing the files in the ramdisk plus some small metadata. A more flexible version of the ramfs is called tmpfs. This is a ramfs where the memory pages containing the data could be paged out into the swap partition. Not a good idea to avoid disk latency, but such a tmpfs is equivalent to the ramfs stuff if there is no swap. And If you have such a big machine where you think about placing some data into a ramdisk you can propably live without swap. And much more important, I believe that such a tmpfs also exists in IRIX. I am shure Solaris and *Bsd have such a tmpfs. Other than that. there is a UNIX system call mlock (I believe provided by some POSIX standard, and I expect that it is available on IRIX) which is able to prevent paging for a given area of the address space. Also there is a possibility to map a file or parts of them into virtual memory of your process via the mmap call. This way you can access the content of the file by simply reading from memory through a pointer. And you can mlock this area. The mlock call then guarantees that the data in question is in main memory when it returns. So this is also a kind of prefetch algorithm with the advance that you can prevent throwing away important cached data by disk caching code when caching other less important files later. Hope this helps Greetings -- Mathias Fröhlich, email: [EMAIL PROTECTED] ___ Flightgear-devel mailing list [EMAIL PROTECTED] http://mail.flightgear.org/mailman/listinfo/flightgear-devel
RE: [Flightgear-devel] RAM disk / Unix
Jon S Berndt writes: Is anyone aware of a RAM disk utility or feature under Unix (specifically, IRIX)? When running a simulation on IRIX we are finding that the disk access is taking too much time at various phase boundaries. It is thought that the use of a RAM disk might help. On Windows I have found that increading disk cache size and / or using memory mapped files is more productive then a DAM disk Maybe the same tricks would work for IRIX. The main drawback to RAM disks is that you end up needing twice the RAM, one for the Disk Image and once for the Memory Image HTH Norman ___ Flightgear-devel mailing list [EMAIL PROTECTED] http://mail.flightgear.org/mailman/listinfo/flightgear-devel
Re: [Flightgear-devel] RAM disk / Unix
Jon S. Berndt wrote: Is anyone aware of a RAM disk utility or feature under Unix (specifically, IRIX)? When running a simulation on IRIX we are finding that the disk access is taking too much time at various phase boundaries. It is thought that the use of a RAM disk might help. RAM disks are ancient history; they date from a time before OS-managed caching of disk access. If you are sure you have the RAM to support both your application and the data file contents simultaneously, then you can prefetch the file contents into memory with a simple: cat datafile /dev/null Try this before running your program and see how that works. If it turns out you do need the memory for the running process, you might try spawning it off as a background process shortly (you'll have to measure that time delay for an appropriate value) before you start hitting the disk. Andy ___ Flightgear-devel mailing list [EMAIL PROTECTED] http://mail.flightgear.org/mailman/listinfo/flightgear-devel
Re: [Flightgear-devel] RAM disk / Unix
Norman Vine wrote: Jon S. Berndt wrote: It is thought that the use of a RAM disk might help. On Windows I have found that increading disk cache size and / or using memory mapped files is more productive then a DAM disk My interpretation was that their problem was latency, not I/O throughput. The program cooks along using 100% of the CPU, then it needs to load a giant data set off the disk, so it has to wait (making 0% use of the CPU) while the I/O system does its job. If they could ensure that the dataset was in memory they could avoid this delay. Andy ___ Flightgear-devel mailing list [EMAIL PROTECTED] http://mail.flightgear.org/mailman/listinfo/flightgear-devel
Re: [Flightgear-devel] RAM disk / Unix
On Tue, 10 Feb 2004 15:06:41 -0800 Andy Ross [EMAIL PROTECTED] wrote: Norman Vine wrote: Jon S. Berndt wrote: It is thought that the use of a RAM disk might help. On Windows I have found that increading disk cache size and / or using memory mapped files is more productive then a DAM disk My interpretation was that their problem was latency, not I/O throughput. The program cooks along using 100% of the CPU, then it needs to load a giant data set off the disk, so it has to wait (making 0% use of the CPU) while the I/O system does its job. If they could ensure that the dataset was in memory they could avoid this delay. Andy Yes, it seems to be a latency issue. Thanks for all the inputs - at least we have somewhere to start, if not viable solutions. Jon ___ Flightgear-devel mailing list [EMAIL PROTECTED] http://mail.flightgear.org/mailman/listinfo/flightgear-devel
