Re: Announcement: Kernel with automatic boot tracing and prefetching available for testing (GSoC 2007)

2007-08-14 Thread Krzysztof Lichota
Phillip Susi napisał(a):
 Krzysztof Lichota wrote:
 How it works?
 - During boot file accesses are recorded.
 - During subsequent boots this trace is used to prefetch files before
 they are used in order to speed up boot.
 
 How is the data prefetched?  I know that the readahead package used the
 readahead() system call to load entire files that were accessed during
 boot.  This is sub optimal because often times the entire file is not
 needed, only certain pages of it.  

Prefetch works on page-size chunks, so only used parts of files are
prefetched.

As for reading, prefetch uses force_page_cache_readahead(), so it is the
same function as used by readahead() call.

 Also the readahead was done
 synchronously one file at a time, which does not keep the disk at full
 utilization.  Ideally you just want to read in the required pages, in
 the order in which they are required, with several asynchronous requests
 in the queue at a time.  

This is good idea, it might improve results a bit. I will try it in the
future.
However, room for improvement is quite small, as you can see on this
bootchart:
http://prefetch.googlecode.com/svn/trunk/results/boot-prefetching/testmachine-kl1/test-9-versus-plain/bootchart-prefetch-sync.png

During readahead disk is almost 100% utilized.

 You also want to wait until some of the data is
 read before continuing with the boot process, but you don't want to wait
 until ALL of it is read.

It depends. I have done some experiments with asynchronous prefetching
and in most cases it is better to wait for readahead to finish before
proceeding than to let readahead go in parallel with execution of apps.
This is consistent with results of other prefetching systems.

The exception is when you can prefetch some files in advance, before
execution of apps starts. This approach is used in prefetch to prefetch
GUI in advance. The boot is split into 3 phases:
1. boot - since root partition is mounted until all filesystems are mounted
2. system - when all partitions are mounted till KDM/GDM is started
3. gui - when desktop environment is started

Each phase is traced separately. For boot and system phase files are
prefetched synchronously at start of the phase, but GUI phase files are
prefetched in background right after system phase files are prefetched.

I have done quite a lot of experiments to find the proper combination of
parameters. See the page with descriptions of experiments:
http://code.google.com/p/prefetch/wiki/InitialBootPrefetchingResults

Of course there is always room for improvements. This implementation of
prefetch is just the beginning :)

Thanks for insightful comments.

Krzysztof Lichota





signature.asc
Description: OpenPGP digital signature
-- 
Ubuntu-devel-discuss mailing list
Ubuntu-devel-discuss@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss


Re: Announcement: Kernel with automatic boot tracing and prefetching available for testing (GSoC 2007)

2007-08-13 Thread Krzysztof Lichota
(``-_-´´) -- Fernando napisał(a):
 Isnt this the same as puting profile on the grub when you get a new kernel?!

No, because with readahead you have to remember to do it upon each
change (not only new kernel, but also installing libraries, enabling
services, adding autostart apps, etc.).

With prefetch it is done completely transparently for user, it just
adapts to new environment.

Also, as I mentioned in announcement, with prefetch it is possible to do
tracing and prefetching at the same time.

Prefetch tracing facility also makes possible adding more sophisticated
optimizations: prefetching during application start and reordering files
on disk. Both are in the works and I will announce them when they are ready.

Krzysztof Lichota




signature.asc
Description: OpenPGP digital signature
-- 
Ubuntu-devel-discuss mailing list
Ubuntu-devel-discuss@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss


Re: Announcement: Kernel with automatic boot tracing and prefetching available for testing (GSoC 2007)

2007-08-13 Thread Phillip Susi
Krzysztof Lichota wrote:
 Hello everyone.
 I am working this summer on Automatic boot and application start file
 prefetching project as part of Google Summer of Code for Ubuntu.
 
 I am glad to announce that first version of kernel with automatic boot
 tracing and prefetching is available for widespread testing :)
 
 How it works?
 - During boot file accesses are recorded.
 - During subsequent boots this trace is used to prefetch files before
 they are used in order to speed up boot.

How is the data prefetched?  I know that the readahead package used the 
readahead() system call to load entire files that were accessed during 
boot.  This is sub optimal because often times the entire file is not 
needed, only certain pages of it.  Also the readahead was done 
synchronously one file at a time, which does not keep the disk at full 
utilization.  Ideally you just want to read in the required pages, in 
the order in which they are required, with several asynchronous requests 
in the queue at a time.  You also want to wait until some of the data is 
read before continuing with the boot process, but you don't want to wait 
until ALL of it is read.

Does your approach address any of these issues?


-- 
Ubuntu-devel-discuss mailing list
Ubuntu-devel-discuss@lists.ubuntu.com
Modify settings or unsubscribe at: 
https://lists.ubuntu.com/mailman/listinfo/ubuntu-devel-discuss