Bug#635993: dpkg is very slow with btrfs filesystem
Hi! On Sat, 2011-07-30 at 08:15:38 -0400, Nicolas Stransky wrote: On 07/30/2011 04:02 AM, Raphael Hertzog wrote: On Fri, 29 Jul 2011, Nicolas STRANSKY wrote: Simplistic tests involving tar show that the disk is able to produce throughputs that are about 30x higher than the ones I observe when dpkg is unpacking files (60MB/s vs. 2MB/s). I have tried a couple workarounds but they didn't change anything: using dpkg with --force-unsafe-io or mounting / with nodatacow. Did you try running your apt-get under eatmydata? If it solves your issue it means the slowness is the result of the fsync() that dpkg is doing. And I heard it multiple times that btrfs was not very efficient when you are doing lots of fsync() but I am afraid there's not much we can do to improve this if we want to keep the reliabily of dpkg in general. I didn't know before about eatmydata but it indeed completely solves my issue! dist-upgrades are like a hundred times faster. So I guess the problem is really that the multiple fsync() in dpkg won't work efficiently with btrfs, which is a shame! It makes it almost impossible to use btrfs on / with Debian. Well I've said this before in the previous discussions about the fsync() issues; the dpkg database is sacred and it must never get damaged, providing an interface in dpkg to even allow this possibility is not acceptable, but then if the users use something like eatmydata then they are on their own. In the btrfs specific case, I think we should either close or wontfix this bug report (we already have enother one stating we are not going to remove the db fsync()s), and you should probably take it with the btrfs developers, as it seems extremely bad the file system degrades so much when confronted with several fsync()s. They should probably look at implementing the sync_file_range() interfaces for it if they haven't yet, or checking why they are not efficient. regards, guillem -- To UNSUBSCRIBE, email to debian-dpkg-bugs-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#635993: dpkg is very slow with btrfs filesystem
On Fri, 29 Jul 2011, Nicolas STRANSKY wrote: Simplistic tests involving tar show that the disk is able to produce throughputs that are about 30x higher than the ones I observe when dpkg is unpacking files (60MB/s vs. 2MB/s). I have tried a couple workarounds but they didn't change anything: using dpkg with --force-unsafe-io or mounting / with nodatacow. Did you try running your apt-get under eatmydata? If it solves your issue it means the slowness is the result of the fsync() that dpkg is doing. And I heard it multiple times that btrfs was not very efficient when you are doing lots of fsync() but I am afraid there's not much we can do to improve this if we want to keep the reliabily of dpkg in general. Cheers, -- Raphaël Hertzog ◈ Debian Developer Follow my Debian News ▶ http://RaphaelHertzog.com (English) ▶ http://RaphaelHertzog.fr (Français) -- To UNSUBSCRIBE, email to debian-dpkg-bugs-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#635993: dpkg is very slow with btrfs filesystem
On 07/30/2011 04:02 AM, Raphael Hertzog wrote: On Fri, 29 Jul 2011, Nicolas STRANSKY wrote: Simplistic tests involving tar show that the disk is able to produce throughputs that are about 30x higher than the ones I observe when dpkg is unpacking files (60MB/s vs. 2MB/s). I have tried a couple workarounds but they didn't change anything: using dpkg with --force-unsafe-io or mounting / with nodatacow. Did you try running your apt-get under eatmydata? If it solves your issue it means the slowness is the result of the fsync() that dpkg is doing. And I heard it multiple times that btrfs was not very efficient when you are doing lots of fsync() but I am afraid there's not much we can do to improve this if we want to keep the reliabily of dpkg in general. I didn't know before about eatmydata but it indeed completely solves my issue! dist-upgrades are like a hundred times faster. So I guess the problem is really that the multiple fsync() in dpkg won't work efficiently with btrfs, which is a shame! It makes it almost impossible to use btrfs on / with Debian. Thanks, Nicolas -- Nico GPG FBFA4781 -- To UNSUBSCRIBE, email to debian-dpkg-bugs-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#635993: dpkg is very slow with btrfs filesystem
Package: dpkg Version: 1.16.0.3 Severity: normal Hello, I have a btrfs filesystem on /, and I experience an extreme slowness when updating or installing packages, presumably due to dpkg's way to write or unpack files. As a consequence, any kind of dist-upgrade takes a very long time and the system's load goes pretty high during that process. Simplistic tests involving tar show that the disk is able to produce throughputs that are about 30x higher than the ones I observe when dpkg is unpacking files (60MB/s vs. 2MB/s). I have tried a couple workarounds but they didn't change anything: using dpkg with --force-unsafe-io or mounting / with nodatacow. I know that there has been some debate about dpkg / btrfs in the past, but what I experience would tend to show that things are not solved. On the other hand, I'd be happy to help testing a possible workaround. Thanks, Nicolas -- System Information: Debian Release: wheezy/sid APT prefers unstable APT policy: (500, 'unstable'), (500, 'testing'), (101, 'experimental') Architecture: i386 (i686) Kernel: Linux 3.0.0 (SMP w/2 CPU cores; PREEMPT) Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8) Shell: /bin/sh linked to /bin/bash Versions of packages dpkg depends on: ii coreutils 8.5-1GNU core utilities ii libbz2-1.0 1.0.5-6 high-quality block-sorting file co ii libc6 2.13-11 Embedded GNU C Library: Shared lib ii libselinux1 2.0.98-1.1 SELinux runtime shared libraries ii xz-utils5.0.0-2 XZ-format compression utilities ii zlib1g 1:1.2.3.4.dfsg-3 compression library - runtime dpkg recommends no packages. Versions of packages dpkg suggests: ii apt 0.8.15.5 Advanced front-end for dpkg -- Configuration Files: /etc/dpkg/dpkg.cfg changed: no-debsig log /var/log/dpkg.log force-unsafe-io -- no debconf information -- To UNSUBSCRIBE, email to debian-dpkg-bugs-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
