Re: Relocating files for faster boot/start-up on reiser(fs/4)
On Friday 15 September 2006 16:27, David Masover wrote: > > Not sure if I would be better of trying initng or waiting for upstart > > (Ubuntus new init) to get scripts that actually parallel boot. The code > > for upstart is very clean and it has the backing of a major distro, so I > > have high hopes. > > Hmm. That sounds kind of cool, but I wonder how it compares to Gentoo's > init scripts? I guess I'll have to wait till it hits the one Ubuntu box > I have... Gentoo default init doesn't paralize well. Not when compared to initng which is realitivly easy to get to work on Gentoo. The Ubuntu people decided initng wansn't powerfull enough (let alone the existing sysvinit). They thought it needed a better way to define the bootup sequence during boot. In addition to integrate running any task like ACPI events, hotplut, CRON into one consistent tool. http://www.netsplit.com/blog/work/canonical/upstart.html > > > Much like before, I was able to improve a 16.5s oowriter cold start to > > 14s with this reallocate script, with a cold start of 4.8s (OO 2.0.2, was > > using 2.0.3 before). > > Wait -- cold start is 14s, but it's also 4.8s? Did you mean warm/hot > start for that last number? OOPS its 4.8s warm and was initially 16.5s cold then 14s cold after reallocationg. > > > I think Python will be the best language for this because its become > > relatively universal and its easy to understand for the uninitiated. > > This really isn't black magic so transparency is good. I personally > > prefer Ruby though. > > Wait... Python is more universal than Ruby of Ruby on Rails? Both Gentoo and Ubuntu install Python by default but not Ruby. And more people at least in the US are familiar with Python. Finally I might use Python inotify code (to replace readahead-watch) and the Ruby version is a bit alpha and I don't think availible in Gentoo or Ubuntu packages. I came to Ruby through RoR. I think the language has an unmatched pragmatic eligance. This isn't appreciated until one addresses a few problem domains with it. I don't know of anything Python does reasonably well that Ruby can't do reasonably well (- the performance problem). On the other hand, I doubt Python could make for something as slick as Rake http://www.martinfowler.com/articles/rake.html. And Ruby provides a wealth of conviences and shortcuts without being the lexical mess that is Perl. I could be missing something though. But for this particular problem Python isn't bad. > > Python is faster, anyway... I'm waiting for someone to do a decent > implementation of Ruby on something like .NET before I start using it > for anything I want to perform well.
Re: Re: v3 rebuild-tree left system in unusable state because of space shortage
Just curious, could it also be fixed by mounting the FS, freeing up some space, then retrying the FSCK? Or is the FS unusable? can't mount it either RO or RW. It says the process of rebuilding tree was incomplette and I have no options to take. Corner case really. -- GJ
Re: Relocating files for faster boot/start-up on reiser(fs/4)
Quinn Harris wrote: On Thursday 14 September 2006 23:15, Toby Thain wrote: On 14-Sep-06, at 6:23 PM, David Masover wrote: Quinn Harris wrote: On Thursday 14 September 2006 13:55, David Masover wrote: ... That is a good point. Recording the disk layout before and after to compare relative fragmentation would be a good idea. As well as randomizing the sequence as a sanity check. Also note that during boot I was using readahead on all 3885 files. So the kernel has a good opportunity to rearrange the reads. And the read sequence doesn't necessary match the order its needed (though I tried to get that). Speaking of which, did you parallize the boot process at all? Just off the top of my head, wouldn't that make the access sequence asynchronous & thereby less predictable? (Although I'm sure it's a net win.) It could, but the kernel will try to reorder the outstanding block requests to reduce seek. If that is an overall win I don't know. In addition early in the boot, readahead-list or similar will tell the kernel to start reading most of the files need for the complete boot so they are already in memory when needed. Ubuntu does the readahead now and all my tests where with readahead. That's interesting. I think either parallizing or a very aggressive readahead will perform similarly, except in cases where you have a script blocking on something other than disk or CPU, like, say, network. I'd estimate my system easily spent more than 50% of its boot time not touching the disk at all before I did that. Gentoo can do this, I'm not sure what else, as it kind of needs your init system to understand dependencies. ... The current Ubuntu boot waits for hardware probing, DHCP and other things giving the disk readahead a chance to work. I think this reallocation might help a parallel boot more as the data will be needed sooner. So I changed my mind, I think parallel boot will highlight the reallocate advantage. Now I just need to test the hypothesis. Hmm. That's possible. But again, even with the parallel boot, there was still a bit of time spent not touching the disk, so I wouldn't expect much more of a speedup than what you already have. Which also means, by the way, that I wouldn't use it much -- my system takes more like 20 seconds from Grub to a login prompt, and from then on, the only things that take more than 5 seconds to load are games. Since I know Quake 4 uses zipfiles (probably compressed) for its storage, and I watched the HD LED while it loads, I don't think I can speed that up at all short of buying a faster CPU. Well, that and the Portage tree, but you say I shouldn't expect much from that. Maybe the portage cache? Not sure if I would be better of trying initng or waiting for upstart (Ubuntus new init) to get scripts that actually parallel boot. The code for upstart is very clean and it has the backing of a major distro, so I have high hopes. Hmm. That sounds kind of cool, but I wonder how it compares to Gentoo's init scripts? I guess I'll have to wait till it hits the one Ubuntu box I have... Much like before, I was able to improve a 16.5s oowriter cold start to 14s with this reallocate script, with a cold start of 4.8s (OO 2.0.2, was using 2.0.3 before). Wait -- cold start is 14s, but it's also 4.8s? Did you mean warm/hot start for that last number? I think Python will be the best language for this because its become relatively universal and its easy to understand for the uninitiated. This really isn't black magic so transparency is good. I personally prefer Ruby though. Wait... Python is more universal than Ruby of Ruby on Rails? Python is faster, anyway... I'm waiting for someone to do a decent implementation of Ruby on something like .NET before I start using it for anything I want to perform well.
Re: v3 rebuild-tree left system in unusable state because of space shortage
Vladimir V. Saveliev wrote: while there is no fix currently for this problem you can solve the problem by expanding underlaying device. Just curious, could it also be fixed by mounting the FS, freeing up some space, then retrying the FSCK? Or is the FS unusable?
Re: v3 rebuild-tree left system in unusable state because of space shortage
Scratch that last command use find -type f -printf "%k\t%p\n" | sort -n | tail -n 100 its much faster. On Friday 15 September 2006 11:06, Grzegorz Jaśkiewicz wrote: > On 9/15/06, Vladimir V. Saveliev <[EMAIL PROTECTED]> wrote: > > Could it be that the file was opened when you deleted it? > > nope, I know that issue. I rebooted the system twice before trying any > other way.
Re: v3 rebuild-tree left system in unusable state because of space shortage
Are you still able to mount the fs read write? If so, you might see if you
can find files to delete, or possibly the 4G file was hardlinked to somewhere
else. This should list all files on the partition with the largest last.
find -type f -exec du {} \; | sort -n
Then wait a long time.
Does that turn up anything interesting?
KDE has a nice "file size view" graphic tool as a kpart for konqueror.
On Friday 15 September 2006 11:06, Grzegorz Jaśkiewicz wrote:
> On 9/15/06, Vladimir V. Saveliev <[EMAIL PROTECTED]> wrote:
> > Could it be that the file was opened when you deleted it?
>
> nope, I know that issue. I rebooted the system twice before trying any
> other way.
Re: Relocating files for faster boot/start-up on reiser(fs/4)
On Thursday 14 September 2006 23:15, Toby Thain wrote: > On 14-Sep-06, at 6:23 PM, David Masover wrote: > > Quinn Harris wrote: > >> On Thursday 14 September 2006 13:55, David Masover wrote: > >>> ... > >> > >> That is a good point. Recording the disk layout before and after > >> to compare relative fragmentation would be a good idea. As well > >> as randomizing the sequence as a sanity check. > >> Also note that during boot I was using readahead on all 3885 > >> files. So the kernel has a good opportunity to rearrange the > >> reads. And the read sequence doesn't necessary match the order > >> its needed (though I tried to get that). > > > > Speaking of which, did you parallize the boot process at all? > > Just off the top of my head, wouldn't that make the access sequence > asynchronous & thereby less predictable? (Although I'm sure it's a > net win.) It could, but the kernel will try to reorder the outstanding block requests to reduce seek. If that is an overall win I don't know. In addition early in the boot, readahead-list or similar will tell the kernel to start reading most of the files need for the complete boot so they are already in memory when needed. Ubuntu does the readahead now and all my tests where with readahead. > > > I'd estimate my system easily spent more than 50% of its boot time > > not touching the disk at all before I did that. Gentoo can do > > this, I'm not sure what else, as it kind of needs your init system > > to understand dependencies. > > ... My first test turned out to be on a heavily fragmented file system. I reinstalled Ubuntu Dapper with a fresh reiserfs file system and it booted in 1:07 (grub to desktop background appearing). After extending the time readahead-watch monitors files and running the reallocate script it now boots in 0:50. I wrote a little python script that uses the FIBMAP ioctl to check the blocks the files are using. From this I know the relocate script on this fresh file system is doing exactly what it was intended to do. I am also able to estimate how much it will improve performance by comparing the fragmentation before and after its run. I have learned that the delays on disk io for Ubuntu boot are dominated by rotational latency and not head seeks. The current readahead implementation orders the files by on disk location, substantially mitigating head seek time. But the latency is can easily double the time needed to load the same data. Subjectively (and objectively by about 6s) relocation and extending readahead-watch substantially improved Gnome boot and initial responsiveness. But, I need to measure how much of this was caused by just extending how much is read ahead vs. the reallocation. The current Ubuntu boot waits for hardware probing, DHCP and other things giving the disk readahead a chance to work. I think this reallocation might help a parallel boot more as the data will be needed sooner. So I changed my mind, I think parallel boot will highlight the reallocate advantage. Now I just need to test the hypothesis. Not sure if I would be better of trying initng or waiting for upstart (Ubuntus new init) to get scripts that actually parallel boot. The code for upstart is very clean and it has the backing of a major distro, so I have high hopes. Much like before, I was able to improve a 16.5s oowriter cold start to 14s with this reallocate script, with a cold start of 4.8s (OO 2.0.2, was using 2.0.3 before). It is evident to me that the readahead-watch is missing something on Open Office startup. It seems very possible to get OO to cold start in under 8s with the uses of reallocation and readahead right when it starts. My current scripts are at http://www.quinnh.org/reallocate.py (27 line reallocate script, expects dir /tmp/refrag to exist and takes the readahead-watch log as a paramater) http://www.quinnh.org/measure.py (uses FIBMAP to estimate the time needed to load the files in the passed readahead-watch log, uses average seek and and latency for estimate) http://www.quinnh.org/readahead-watch-time-order.patch (Patch against Ubuntu readahead-watch to add an order by access time option.) I will try to write a nice unified script that will profile, reallocate and do readahead for an application to speed it up. e.g. "# reallocate.py oowriter". Run it once to profile and reallocate. drop_caches, Run it again and oowriter loads faster. I think Python will be the best language for this because its become relatively universal and its easy to understand for the uninitiated. This really isn't black magic so transparency is good. I personally prefer Ruby though.
Re: v3 rebuild-tree left system in unusable state because of space shortage
On 9/15/06, Vladimir V. Saveliev <[EMAIL PROTECTED]> wrote: > Could it be that the file was opened when you deleted it? nope, I know that issue. I rebooted the system twice before trying any other way. -- GJ
Re: v3 rebuild-tree left system in unusable state because of space shortage
Hello On Friday 15 September 2006 14:34, Grzegorz Jaśkiewicz wrote: > okay, seems to makes sens. > So you guys are working on it, or is the problem not quite fixable ? > Substantial changes are needed to fix this problem. > The underlying problem is what worries me more - the file that was > deleted, but space never got free. > Could it be that the file was opened when you deleted it? > I am about to buy larger HD today (this one was 160, I am going to get > 250 one), so I can do it in hardware. We will see, I'll report later > how it went along. > > Ta for help. > >
Re: v3 rebuild-tree left system in unusable state because of space shortage
okay, seems to makes sens. So you guys are working on it, or is the problem not quite fixable ? The underlying problem is what worries me more - the file that was deleted, but space never got free. I am about to buy larger HD today (this one was 160, I am going to get 250 one), so I can do it in hardware. We will see, I'll report later how it went along. Ta for help.
Re: v3 rebuild-tree left system in unusable state because of space shortage
Hello On Friday 15 September 2006 12:25, Grzegorz Jaśkiewicz wrote: > I had little problem, deleted a 4GB file, but this space was never > freed , but file was gone. So I decided to run -check - no problems > found, next step was to rebuild tree: > > [EMAIL PROTECTED]:~# fsck.reiserfs --rebuild-tree -y /dev/hda3 > reiserfsck 3.6.19 (2003 www.namesys.com) > > * > ** Do not run the program with --rebuild-tree unless ** > ** something is broken and MAKE A BACKUP before using it. ** > ** If you have bad sectors on a drive it is usually a bad ** > ** idea to continue using it. Then you probably should get ** > ** a working hard drive, copy the file system from the bad ** > ** drive to the good one -- dd_rescue is a good tool for ** > ** that -- and only then run this program. ** > ** If you are using the latest reiserfsprogs and it fails ** > ** please email bug reports to reiserfs-list@namesys.com, ** > ** providing as much information as possible -- your ** > ** hardware, kernel, patches, settings, all reiserfsck ** > ** messages (including version), the reiserfsck logfile, ** > ** check the syslog file for any related information. ** > ** If you would like advice on using this program, support ** > ** is available for $25 at www.namesys.com/support.html. ** > * > > Will rebuild the filesystem (/dev/hda3) tree > Will put log info to 'stdout' > > Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes > Replaying journal.. > Reiserfs journal '/dev/hda3' in blocks [18..8211]: 0 transactions replayed > ### > reiserfsck --rebuild-tree started at Fri Sep 15 09:14:59 2006 > ### > > Pass 0: > ### Pass 0 ### > Loading on-disk bitmap .. ok, 31232367 blocks marked used > Skipping 9164 blocks (super block, journal, bitmaps) 31223203 blocks > will be read > 0%20%40%60%80%100% left 0, 10725 > /sec > 383576 directory entries were hashed with "r5" hash. > "r5" hash is selected > Flushing..finished > Read blocks (but not data blocks) 31223203 > Leaves among those 214478 > Objectids found 383578 > > Pass 1 (will try to insert 214478 leaves): > ### Pass 1 ### > Looking for allocable blocks .. finished > 0%20%40%60%80%Not enough allocable blocks, > checking bitmap...there are 1 allocable blocks, btw > > out of disk space > Aborted > > > [EMAIL PROTECTED]:~# cat /proc/cpuinfo > processor : 0 > vendor_id : GenuineIntel > cpu family : 6 > model : 8 > model name : Pentium III (Coppermine) > stepping: 6 > cpu MHz : 797.453 > cache size : 256 KB > fdiv_bug: no > hlt_bug : no > f00f_bug: no > coma_bug: no > fpu : yes > fpu_exception : yes > cpuid level : 2 > wp : yes > flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge > mca cmov pat pse36 mmx fxsr sse > bogomips: 1581.05 > > [EMAIL PROTECTED]:~# lspci > :00:00.0 Host bridge: Intel Corp. 82815 815 Chipset Host Bridge > and Memory Controller Hub (rev 02) > :00:02.0 VGA compatible controller: Intel Corp. 82815 CGC [Chipset > Graphics Controller] (rev 02) > :00:1e.0 PCI bridge: Intel Corp. 82801 PCI Bridge (rev 01) > :00:1f.0 ISA bridge: Intel Corp. 82801BA ISA Bridge (LPC) (rev 01) > :00:1f.1 IDE interface: Intel Corp. 82801BA IDE U100 (rev 01) > :00:1f.4 USB Controller: Intel Corp. 82801BA/BAM USB (Hub #2) (rev 01) > :02:08.0 Ethernet controller: Intel Corp. 82801BA/BAM/CA/CAM > Ethernet Controller (rev 01) > :02:09.0 FireWire (IEEE 1394): VIA Technologies, Inc. IEEE 1394 > Host Controller (rev 80) > [EMAIL PROTECTED]:~# dmesg|grep hda > [4294673.965000] ide0: BM-DMA at 0x2020-0x2027, BIOS settings: > hda:DMA, hdb:pio > [4294674.229000] hda: ST3160812A, ATA DISK drive > [4294677.878000] hda: max request size: 1024KiB > [4294677.921000] hda: 312581808 sectors (160041 MB) w/8192KiB Cache, > CHS=19457/255/63, UDMA(100) > > fsck.reiserfs -V > reiserfsck 3.6.19 (2003 www.namesys.com) > > Linux gj1box 2.6.12-10-386 #1 Thu Sep 14 09:34:39 UTC 2006 i686 GNU/Linux > > what shall I do? > while there is no fix currently for this problem you can solve the problem by expanding underlaying device. One possible solution is to setup a linear array. In the example below I had "out of disk space" with rebuild-tree /dev/loop0, created 1000 block extention file and losetup-ed it as /dev/loop1, created linear array /dev/md0 of /dev/loop0 and /dev/loop1 and ran reiserfsck --rebuild-sb /dev/md0 to fix number of blocks in the filesystem and reiserfsck --rebuild-tree /dev/md0. 1) dd if=/dev/zero of=ext-1000 count=1000 bs=4096 loseup /dev/loop1 ext-1000 2) create linear raid mdadm -B
v3 rebuild-tree left system in unusable state because of space shortage
I had little problem, deleted a 4GB file, but this space was never freed , but file was gone. So I decided to run -check - no problems found, next step was to rebuild tree: [EMAIL PROTECTED]:~# fsck.reiserfs --rebuild-tree -y /dev/hda3 reiserfsck 3.6.19 (2003 www.namesys.com) * ** Do not run the program with --rebuild-tree unless ** ** something is broken and MAKE A BACKUP before using it. ** ** If you have bad sectors on a drive it is usually a bad ** ** idea to continue using it. Then you probably should get ** ** a working hard drive, copy the file system from the bad ** ** drive to the good one -- dd_rescue is a good tool for ** ** that -- and only then run this program. ** ** If you are using the latest reiserfsprogs and it fails ** ** please email bug reports to reiserfs-list@namesys.com, ** ** providing as much information as possible -- your ** ** hardware, kernel, patches, settings, all reiserfsck ** ** messages (including version), the reiserfsck logfile, ** ** check the syslog file for any related information. ** ** If you would like advice on using this program, support ** ** is available for $25 at www.namesys.com/support.html. ** * Will rebuild the filesystem (/dev/hda3) tree Will put log info to 'stdout' Do you want to run this program?[N/Yes] (note need to type Yes if you do):Yes Replaying journal.. Reiserfs journal '/dev/hda3' in blocks [18..8211]: 0 transactions replayed ### reiserfsck --rebuild-tree started at Fri Sep 15 09:14:59 2006 ### Pass 0: ### Pass 0 ### Loading on-disk bitmap .. ok, 31232367 blocks marked used Skipping 9164 blocks (super block, journal, bitmaps) 31223203 blocks will be read 0%20%40%60%80%100% left 0, 10725 /sec 383576 directory entries were hashed with "r5" hash. "r5" hash is selected Flushing..finished Read blocks (but not data blocks) 31223203 Leaves among those 214478 Objectids found 383578 Pass 1 (will try to insert 214478 leaves): ### Pass 1 ### Looking for allocable blocks .. finished 0%20%40%60%80%Not enough allocable blocks, checking bitmap...there are 1 allocable blocks, btw out of disk space Aborted [EMAIL PROTECTED]:~# cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 8 model name : Pentium III (Coppermine) stepping: 6 cpu MHz : 797.453 cache size : 256 KB fdiv_bug: no hlt_bug : no f00f_bug: no coma_bug: no fpu : yes fpu_exception : yes cpuid level : 2 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr sse bogomips: 1581.05 [EMAIL PROTECTED]:~# lspci :00:00.0 Host bridge: Intel Corp. 82815 815 Chipset Host Bridge and Memory Controller Hub (rev 02) :00:02.0 VGA compatible controller: Intel Corp. 82815 CGC [Chipset Graphics Controller] (rev 02) :00:1e.0 PCI bridge: Intel Corp. 82801 PCI Bridge (rev 01) :00:1f.0 ISA bridge: Intel Corp. 82801BA ISA Bridge (LPC) (rev 01) :00:1f.1 IDE interface: Intel Corp. 82801BA IDE U100 (rev 01) :00:1f.4 USB Controller: Intel Corp. 82801BA/BAM USB (Hub #2) (rev 01) :02:08.0 Ethernet controller: Intel Corp. 82801BA/BAM/CA/CAM Ethernet Controller (rev 01) :02:09.0 FireWire (IEEE 1394): VIA Technologies, Inc. IEEE 1394 Host Controller (rev 80) [EMAIL PROTECTED]:~# dmesg|grep hda [4294673.965000] ide0: BM-DMA at 0x2020-0x2027, BIOS settings: hda:DMA, hdb:pio [4294674.229000] hda: ST3160812A, ATA DISK drive [4294677.878000] hda: max request size: 1024KiB [4294677.921000] hda: 312581808 sectors (160041 MB) w/8192KiB Cache, CHS=19457/255/63, UDMA(100) fsck.reiserfs -V reiserfsck 3.6.19 (2003 www.namesys.com) Linux gj1box 2.6.12-10-386 #1 Thu Sep 14 09:34:39 UTC 2006 i686 GNU/Linux what shall I do? -- GJ
