Bug#683773: btrfs-write-performance rechecked, downgrading the severity to 'wishlist'
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On Tue, 07 Aug 2012 13:31:28 +0200 Gaudenz Steinlin gaud...@debian.org wrote: Hi Andreas Andreas Glaeser bugs.andreas.glae...@freenet.de writes: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 To check if btrfs is really slow I tried the following: - -# aptitude install btrfs-tools - -created a btrfs-partition as /dev/sdb14 with gparted and aligned it to sector, not to mbr, because the harddisk is an advanced format model with 4096k blocks. - -# mkfs -t btrfs /dev/sdb14 - -# mkdir /mnt/test - -# mount /dev/sdb14 /mnt/test - -# exit andreas@g4d:~$ cd /mnt/test andreas@g4d:/mnt/test$ mkdir fs-root-c-arc andreas@g4d:/mnt/test$ time cp -a /* fs-root-c-arc/ c-arc.txt 2c-err.txt real7m48.020s user0m5.304s sys 1m22.868s andreas@g4d:/mnt/test$ ls -l total 2775172 - -rw-r--r-- 1 andreas andreas 0 Aug 7 08:20 c-arc.txt - -rw-r--r-- 1 andreas andreas1145749 Aug 7 08:27 c-err.txt drwxr-xr-x 1 andreas andreas136 Aug 7 08:27 fs-root-c-arc andreas@g4d:/mnt/test$ du -hs fs-root-c-arc/ 3.6Gfs-root-c-arc/ andreas@g4d:/mnt/test$ chmod 000 fs-root-c-arc/ andreas@g4d:/mnt/test$ time tar -cvf t-arc.tar /* t-out.txt 2t-err.txt real6m25.904s user0m6.016s sys 0m47.936s andreas@g4d:/mnt/test$ ls -l total 2784108 - -rw-r--r-- 1 andreas andreas 0 Aug 7 08:20 c-arc.txt - -rw-r--r-- 1 andreas andreas1145749 Aug 7 08:27 c-err.txt drwxr--r-- 1 andreas andreas136 Aug 7 08:27 fs-root-c-arc - -rw-r--r-- 1 andreas andreas 2841907200 Aug 7 08:47 t-arc.tar - -rw-r--r-- 1 andreas andreas1348292 Aug 7 08:47 t-err.txt - -rw-r--r-- 1 andreas andreas6513194 Aug 7 08:47 t-out.txt This were two tests, first created an archive of the root filesystem using cp below the folder /mnt/test/fs-root-c-arc/. This issued a lot of errors and warning because of missing permissions or files, which changed while being read, but in the end after 7m48s there were 151869 items in that folder, totalling 3.6 GB. Next the mode of the folder was set to 000, because else the content of the folder would be taken into the newly created .tar-archive recursively. Then doing basically the same thing, but putting all readable and accessable files into a single uncompressed .tar-archive instead of just copying them. this was even faster with 6m25s and the archive was 2.6 Gb in size. This is not the same as installing from DVD and via network over http, but big files and many small files are both written fast enough from xfs to btrfs, given that this is a green-labeled harddisk, which is not supposed to break any velocity-records. So I downgraded the installation-report to 'wishlist'. I consider the problems were due to some kind of strange IRQ-conflict or the like. A software-upgrade was not done since installation, just some additional packages installed. No, your test did not really simulate the situation during installation. The problem with btrfs is not poor write performance in general, but very poor fsync performance. dpkg does a lot of fsync's and is therefore heavily affected by this. You could verify this by running debootstrap on a btrfs filesstem (debootstrap wheezy /mnt). This will be incredibly slow. On the other hand if you use the eatmydata utility which turns all fsync calls into noops, it will be fast: eatmydata debootstrap wheezy /mnt. But beware, it's called eatmydata for a reason... Gaudenz Now I retried installing the debian base-system onto my previously created test-partition, selected the smp-kernel, which has no effect until next reboot, so everythin is done on a single core only, and chose the default generic initrd.img. This took about 19m 05s. I retried this with a newly created partition, created during the installation-process using the d-i-partitioning-tool, the result was almost identical, taking 19m 25s, the difference probably being due to my slow resposiveness on interactive questions. When I looked at the two partitions with gparted, there was no information about their alignment, the tool only allows to select this while creating partitions, but can not display information seemingly about existing partitions. cfdisk was not able to show any partitions on the harddrive, but claimed, the disk was empty. It is a bit strange, might actually be another bug. So overall it was right to set the bug-severity to wishlist, because the weak performance was obviously due to faulty hardware and maybe due to some fault-correction or -avoidance mechanism in btrfs. the file system probably tries to preserve data integrity when disk-problems occur. Reading data did go at near to normal speed actually. It is safe now to close this report in my opinion. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux)
Bug#683773: btrfs-write-performance rechecked, downgrading the severity to 'wishlist'
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 To check if btrfs is really slow I tried the following: - -# aptitude install btrfs-tools - -created a btrfs-partition as /dev/sdb14 with gparted and aligned it to sector, not to mbr, because the harddisk is an advanced format model with 4096k blocks. - -# mkfs -t btrfs /dev/sdb14 - -# mkdir /mnt/test - -# mount /dev/sdb14 /mnt/test - -# exit andreas@g4d:~$ cd /mnt/test andreas@g4d:/mnt/test$ mkdir fs-root-c-arc andreas@g4d:/mnt/test$ time cp -a /* fs-root-c-arc/ c-arc.txt 2c-err.txt real7m48.020s user0m5.304s sys 1m22.868s andreas@g4d:/mnt/test$ ls -l total 2775172 - -rw-r--r-- 1 andreas andreas 0 Aug 7 08:20 c-arc.txt - -rw-r--r-- 1 andreas andreas1145749 Aug 7 08:27 c-err.txt drwxr-xr-x 1 andreas andreas136 Aug 7 08:27 fs-root-c-arc andreas@g4d:/mnt/test$ du -hs fs-root-c-arc/ 3.6Gfs-root-c-arc/ andreas@g4d:/mnt/test$ chmod 000 fs-root-c-arc/ andreas@g4d:/mnt/test$ time tar -cvf t-arc.tar /* t-out.txt 2t-err.txt real6m25.904s user0m6.016s sys 0m47.936s andreas@g4d:/mnt/test$ ls -l total 2784108 - -rw-r--r-- 1 andreas andreas 0 Aug 7 08:20 c-arc.txt - -rw-r--r-- 1 andreas andreas1145749 Aug 7 08:27 c-err.txt drwxr--r-- 1 andreas andreas136 Aug 7 08:27 fs-root-c-arc - -rw-r--r-- 1 andreas andreas 2841907200 Aug 7 08:47 t-arc.tar - -rw-r--r-- 1 andreas andreas1348292 Aug 7 08:47 t-err.txt - -rw-r--r-- 1 andreas andreas6513194 Aug 7 08:47 t-out.txt This were two tests, first created an archive of the root filesystem using cp below the folder /mnt/test/fs-root-c-arc/. This issued a lot of errors and warning because of missing permissions or files, which changed while being read, but in the end after 7m48s there were 151869 items in that folder, totalling 3.6 GB. Next the mode of the folder was set to 000, because else the content of the folder would be taken into the newly created .tar-archive recursively. Then doing basically the same thing, but putting all readable and accessable files into a single uncompressed .tar-archive instead of just copying them. this was even faster with 6m25s and the archive was 2.6 Gb in size. This is not the same as installing from DVD and via network over http, but big files and many small files are both written fast enough from xfs to btrfs, given that this is a green-labeled harddisk, which is not supposed to break any velocity-records. So I downgraded the installation-report to 'wishlist'. I consider the problems were due to some kind of strange IRQ-conflict or the like. A software-upgrade was not done since installation, just some additional packages installed. -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.10 (GNU/Linux) iEYEARECAAYFAlAg6AIACgkQ5+rBHyUt5wsPRQCfU4TTdu5f7JhfICqUI9mhKZ1q 1vwAnR890y+p6/eiRPRLktXSLp4KH1PG =v3lz -END PGP SIGNATURE-
Bug#683773: btrfs-write-performance rechecked, downgrading the severity to 'wishlist'
Hi Andreas Andreas Glaeser bugs.andreas.glae...@freenet.de writes: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 To check if btrfs is really slow I tried the following: - -# aptitude install btrfs-tools - -created a btrfs-partition as /dev/sdb14 with gparted and aligned it to sector, not to mbr, because the harddisk is an advanced format model with 4096k blocks. - -# mkfs -t btrfs /dev/sdb14 - -# mkdir /mnt/test - -# mount /dev/sdb14 /mnt/test - -# exit andreas@g4d:~$ cd /mnt/test andreas@g4d:/mnt/test$ mkdir fs-root-c-arc andreas@g4d:/mnt/test$ time cp -a /* fs-root-c-arc/ c-arc.txt 2c-err.txt real 7m48.020s user 0m5.304s sys 1m22.868s andreas@g4d:/mnt/test$ ls -l total 2775172 - -rw-r--r-- 1 andreas andreas 0 Aug 7 08:20 c-arc.txt - -rw-r--r-- 1 andreas andreas1145749 Aug 7 08:27 c-err.txt drwxr-xr-x 1 andreas andreas136 Aug 7 08:27 fs-root-c-arc andreas@g4d:/mnt/test$ du -hs fs-root-c-arc/ 3.6G fs-root-c-arc/ andreas@g4d:/mnt/test$ chmod 000 fs-root-c-arc/ andreas@g4d:/mnt/test$ time tar -cvf t-arc.tar /* t-out.txt 2t-err.txt real 6m25.904s user 0m6.016s sys 0m47.936s andreas@g4d:/mnt/test$ ls -l total 2784108 - -rw-r--r-- 1 andreas andreas 0 Aug 7 08:20 c-arc.txt - -rw-r--r-- 1 andreas andreas1145749 Aug 7 08:27 c-err.txt drwxr--r-- 1 andreas andreas136 Aug 7 08:27 fs-root-c-arc - -rw-r--r-- 1 andreas andreas 2841907200 Aug 7 08:47 t-arc.tar - -rw-r--r-- 1 andreas andreas1348292 Aug 7 08:47 t-err.txt - -rw-r--r-- 1 andreas andreas6513194 Aug 7 08:47 t-out.txt This were two tests, first created an archive of the root filesystem using cp below the folder /mnt/test/fs-root-c-arc/. This issued a lot of errors and warning because of missing permissions or files, which changed while being read, but in the end after 7m48s there were 151869 items in that folder, totalling 3.6 GB. Next the mode of the folder was set to 000, because else the content of the folder would be taken into the newly created .tar-archive recursively. Then doing basically the same thing, but putting all readable and accessable files into a single uncompressed .tar-archive instead of just copying them. this was even faster with 6m25s and the archive was 2.6 Gb in size. This is not the same as installing from DVD and via network over http, but big files and many small files are both written fast enough from xfs to btrfs, given that this is a green-labeled harddisk, which is not supposed to break any velocity-records. So I downgraded the installation-report to 'wishlist'. I consider the problems were due to some kind of strange IRQ-conflict or the like. A software-upgrade was not done since installation, just some additional packages installed. No, your test did not really simulate the situation during installation. The problem with btrfs is not poor write performance in general, but very poor fsync performance. dpkg does a lot of fsync's and is therefore heavily affected by this. You could verify this by running debootstrap on a btrfs filesstem (debootstrap wheezy /mnt). This will be incredibly slow. On the other hand if you use the eatmydata utility which turns all fsync calls into noops, it will be fast: eatmydata debootstrap wheezy /mnt. But beware, it's called eatmydata for a reason... Gaudenz -- Ever tried. Ever failed. No matter. Try again. Fail again. Fail better. ~ Samuel Beckett ~ -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
