On Monday 18 September 2006 03:36, PFC wrote: > Windows already does this. I am familiar with this, it was in part the inspiration for this idea.
> > It has a service which monitors filesystem usage, and writes data to > disk > ; the defragmenter uses this data to lay the files on disk so that boot is > very fast. Yep, my tests suggest for Ubuntu boot that typical improvements will be on the order of 5-15s (substantially dependent on how fragmented the file system is and complexity of boot). The system needs about 70MB of data to boot. Typical fragmentation on a fresh filesystem will cause 2-3x slowdown over perfect packing. 70MB at 20MB/s is 2.5s so our gain would be 5s. The improvement wasn't as good as I was originally expecting but still very measurable. I am not sure this is enough of a win to justify integrating this idea in a major distro. But without a > > However think it optimizes only the time to the login screen ; so > windows > boot is extremely fast ; of course, once you have logged in, you have to > wait forever until all the crap system tray apps launch themselves and eat > all your RAM... In practice what I am doing improves system responsiveness. This is because I am measuring all file access to an idle state then reallocating and preloading all that data. Gnome still has a bit of data to page in after the menus are available (like menu icons) > > My own repacker is very simple, and it handles any filesystem ! > > - boot from Kanotix CD > - tar cv /mnt/my_disk | lzop -c | ssh -c blowfish other_machine "cat > > >backup.tar.lzo" > > - umount, mkfs, mount > - ssh -c blowfish other_machine "cat backup.tar.lzo" | lzop -cd | tar xv > > You can also use an USB disk, or other disks in the machine. The effects > are pretty visible. > I do the first part often to make a full disk backup to a USB harddrive. > > Anyway, IMHO the best way to have a super responsive system would be : > Are you essentially copying all the files off a file system, recreating a fresh fs and writing them back? This allows the file system to reconstruct the files in a good defraged order. Problem is it takes along time and takes the system down (or at least the fs). The Reiser4 repacker would have the same effect but without any downtime. Windows defrag does the same. This doesn't place files used at boot right next to each other on disk like what I am doing, but both our approaches reduces fragmentation. My script can do its thing in about 10s (for just one thing like better bootup) and without disrupting the running system. (Actually I have seen it break some things, but I expect this can be mitigated by avoiding repacking open files) > Have a daemon which monitors which files, or parts of files, are read, and > in what context : > > - boot to fully loaded KDE/Gnome > - launching an application > > You already have this apparently... > > The repacker would then use this information to lay files on disk (just > like windows does). The python script I am using to measure fragmentation shows that what I am doing results in packing that is within 1% of idea in terms of disk read time. This only works well on a fresh filesystem with large contiguous unallocated space but not so well on a heavily fragmented fs. This suggests for reiser (maybe other fs), a special kernel or defrag tool for this type of reallocation is unnecessary. > > Then, the daemon would trigger readahead, when booting or detecting the > launch of an application, and read everything in (which would be a nice > sequential read)... I hope in a good time to write a python script that allows the user to do something like "reallocate.py oowriter". It will profile Open Office writer on startup using inotify to generating a readahead log. Then reallocate with the copy trick. If the command is run again, it will readahead the files for oowriter and launch the app. I think this could double cold start times. Part from just the readahead and part from the reallocation. It is evident that the inotify file monitoring (using Ubuntu readahead-watch) is not catching everything. If I readahead for oowriter then lanuch it, it takes 6-7s. But it takes 3.6 to start after immediately closing it. In principal, I would think the readahead should be able to load all files oowriter needs into cache getting the 3.6s time after readahead. Also this doesn't monitor access patterns within a file. How much of the data in a file is really needed? I am not sure I would be able to pack only parts of files, but I might with sparse files.
