Your message dated Thu, 25 Nov 2010 07:02:13 +0000
with message-id <[email protected]>
and subject line Bug#584254: fixed in dpkg 1.15.8.6
has caused the Debian Bug report #584254,
regarding please allow to disable extremely slow fsync/sync at runtime
to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact [email protected]
immediately.)
--
584254: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=584254
Debian Bug Tracking System
Contact [email protected] with problems
--- Begin Message ---
Package: dpkg
Version: 1.15.7.2
Severity: normal
Hello,
first of all, I'm fully aware of [1] and [2]. However, solution implemented in
#578635 is not good enough and slows down dpkg considerably on ext4 and even
twice as much on btrfs [3]. So it is pretty clear that dpkg is not ready for
the age of modern file systems. I personally find performance after #578635
almost acceptable on my main / where I definitely don't want it to produce
zero-length files but dpkg has other use cases where such a slow down is not
acceptable for the benefit it brings.
For example, sometimes zero-length files after unclean shutdown are not critical
but barely minor problem. One of them is chroots, esp. buildd ones. sbuild and
pbuilder install the same packages over and over again in the (copies of) the
chroot and slow dpkg performance takes much developer time even on new fast
machines. However, in case of power failure, it is not important what happens
to that (copy of) the chroot and how many zero-length files dpkg is going to
leave it with. The (copy of) the chroot will either be `rm -rf` or fixed
manually anyway. So huge dpkg slowdown is for no benefit at all in this case,
it only wastes developer time and considerably prolongs package build time.
What is more, frequent sync()s (#578635), obviously, affect ext3 and other file
systems as well esp. when there is much other parallel disk I/O going on at the
same time.
Therefore, I would like to be able to effectively disable fsync()/sync() at
runtime via dpkg command line switch and alternatively, via entry in
/etc/dpkg/dpkg.conf (yes, this includes fsync() calls on database updates, as
they are the major source of slow down as well). Of course, it would be even
better if dpkg itself were fixed to be crash-tolerant and fast on the file
systems with delayed allocation, but it seems that it is not going to be easy
to implement.
[1] http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=578635
[2] https://bugzilla.kernel.org/show_bug.cgi?id=15910
[3] nosync.c:
void sync() { }
int fsync(int fildes) { return 0; }
$ gcc -shared -fPIC -O2 -g nosync.c -o libnosync.so
--------------------------------------------------------------------
On the BTRFS file system:
# unset LD_PRELOAD
# time apt-get -y build-dep kde4libs
The following NEW packages will be installed:
adduser automoc bison bsdmainutils ca-certificates cdbs cmake cmake-data
comerr-dev debhelper emacsen-common file flex fontconfig
fontconfig-config gettext gettext-base groff-base hspell html2text
intltool-debian krb5-multidev libacl1-dev libasound2 libasound2-dev
libaspell-dev libaspell15 libasyncns0 libattica-dev libattica0 libattr1-dev
libaudio2 libavahi-client3 libavahi-common-data
libavahi-common-dev libavahi-common3 libbz2-dev libcap2 libclucene0ldbl
libcroco3 libcups2 libcups2-dev libcurl3-gnutls libdbus-1-3
libdbus-1-dev libdbus-glib-1-2 libdrm2 libeggdbus-1-0 libenchant-dev
libenchant1c2a libexiv2-6 libexpat1 libfam-dev libfam0 libflac8
libfontconfig1 libfreetype6 libgcrypt11 libgcrypt11-dev libgif-dev libgif4
libgl1-mesa-dev libgl1-mesa-glx libglib2.0-0 libglib2.0-dev
libglu1-mesa libglu1-mesa-dev libgnutls-dev libgnutls26 libgpg-error-dev
libgpg-error0 libgssapi-krb5-2 libgssrpc4 libhunspell-1.2-0
libice-dev libice6 libidn11 libilmbase-dev libilmbase6 libiodbc2
libjasper-dev libjasper1 libjpeg62 libjpeg62-dev libk5crypto3
libkadm5clnt-mit7 libkadm5srv-mit7 libkdb5-4 libkeyutils-dev libkeyutils1
libkrb5-3 libkrb5-dev libkrb5support0 liblcms1 libldap-2.4-2
liblzma-dev libmagic1 libmng1 libncursesw5 libogg0 libopenexr-dev libopenexr6
libpcre3 libpcre3-dev libpcrecpp0 libphonon-dev libphonon4
libpng12-0 libpng12-dev libpolkit-agent-1-0 libpolkit-gobject-1-0
libpolkit-qt-1-0 libpolkit-qt-1-dev libpopt0 libpthread-stubs0
libpthread-stubs0-dev libpulse-mainloop-glib0 libpulse0 libqca2 libqca2-dev
libqt4-assistant libqt4-dbus libqt4-designer libqt4-dev
libqt4-help libqt4-multimedia libqt4-network libqt4-opengl libqt4-opengl-dev
libqt4-qt3support libqt4-script libqt4-scripttools
libqt4-sql libqt4-svg libqt4-test libqt4-webkit libqt4-xml libqt4-xmlpatterns
libqtcore4 libqtgui4 libraptor1 librasqal2 librdf0
libsasl2-2 libsm-dev libsm6 libsndfile1 libsoprano-dev libsoprano4
libsqlite3-0 libssl-dev libssl0.9.8 libstreamanalyzer-dev
libstreamanalyzer0 libstreams-dev libstreams0 libtasn1-3 libtasn1-3-dev
libtiff4 libunistring0 libutempter-dev libutempter0 libvorbis0a
libvorbisenc2 libwrap0 libx11-6 libx11-data libx11-dev libxau-dev libxau6
libxcb1 libxcb1-dev libxcursor-dev libxcursor1 libxdamage1
libxdmcp-dev libxdmcp6 libxext-dev libxext6 libxfixes-dev libxfixes3
libxi-dev libxi6 libxml2 libxml2-dev libxml2-utils libxrender-dev
libxrender1 libxslt1-dev libxslt1.1 libxss-dev libxss1 libxt6 libxtst-dev
libxtst6 libxxf86vm1 m4 man-db mesa-common-dev openssl
pkg-config pkg-kde-tools po-debconf procps qt4-qmake
shared-desktop-ontologies shared-mime-info soprano-daemon ttf-dejavu-core ucf
x11-common x11proto-core-dev x11proto-fixes-dev x11proto-input-dev
x11proto-kb-dev x11proto-record-dev x11proto-render-dev
x11proto-scrnsaver-dev x11proto-xext-dev xsltproc xtrans-dev zlib1g-dev
0 upgraded, 222 newly installed, 0 to remove and 0 not upgraded.
Need to get 0B/92.5MB of archives.
After this operation, 273MB of additional disk space will be used.
...
real 12m35.082s
user 0m6.760s
sys 0m14.969s
# time dpkg -P <222 kde4libs build-depends>
...
real 11m21.294s
user 0m2.488s
sys 0m9.305s
# export LD_PRELOAD=libnosync.so
# time apt-get -y build-dep kde4libs
...
real 0m31.748s
user 0m6.496s
sys 0m6.576s
# time dpkg -P <222 kde4libs build-depends>
...
real 0m11.788s
user 0m2.160s
sys 0m2.732s
nosync version is ~24 / 60 times faster.
--------------------------------------------------------------------
On the ext4 file system:
# unset LD_PRELOAD
# time apt-get -y build-dep kde4libs
...
real 6m38.181s
user 0m7.328s
sys 0m6.816s
# time dpkg -P <222 kde4libs build-depends>
...
real 4m49.146s
user 0m2.504s
sys 0m2.020s
# export LD_PRELOAD=libnosync.so
# time apt-get -y build-dep kde4libs
...
real 0m34.390s
user 0m6.888s
sys 0m4.816s
# time dpkg -P <222 kde4libs build-depends>
...
real 0m10.010s
user 0m2.072s
sys 0m1.564s
nosync version is ~11 / 34 times faster.
--------------------------------------------------------------------
-- System Information:
Debian Release: squeeze/sid
APT prefers unstable
APT policy: (500, 'unstable'), (500, 'testing'), (101, 'experimental')
Architecture: amd64 (x86_64)
Kernel: Linux 2.6.35-rc4 (SMP w/4 CPU cores)
Locale: LANG=lt_LT.UTF-8, LC_CTYPE=lt_LT.UTF-8 (charmap=UTF-8)
Shell: /bin/sh linked to /bin/dash
Versions of packages dpkg depends on:
ii coreutils 8.5-1 GNU core utilities
ii libbz2-1.0 1.0.5-4 high-quality block-sorting file co
ii libc6 2.11.2-2 Embedded GNU C Library: Shared lib
ii libselinux1 2.0.94-1 SELinux runtime shared libraries
ii xz-utils 4.999.9beta+20100527-1 XZ-format compression utilities
ii zlib1g 1:1.2.3.4.dfsg-3 compression library - runtime
dpkg recommends no packages.
Versions of packages dpkg suggests:
ii apt 0.7.25.3 Advanced front-end for dpkg
-- no debconf information
--- End Message ---
--- Begin Message ---
Source: dpkg
Source-Version: 1.15.8.6
We believe that the bug you reported is fixed in the latest version of
dpkg, which is due to be installed in the Debian FTP archive:
dpkg-dev_1.15.8.6_all.deb
to main/d/dpkg/dpkg-dev_1.15.8.6_all.deb
dpkg_1.15.8.6.dsc
to main/d/dpkg/dpkg_1.15.8.6.dsc
dpkg_1.15.8.6.tar.bz2
to main/d/dpkg/dpkg_1.15.8.6.tar.bz2
dpkg_1.15.8.6_amd64.deb
to main/d/dpkg/dpkg_1.15.8.6_amd64.deb
dselect_1.15.8.6_amd64.deb
to main/d/dpkg/dselect_1.15.8.6_amd64.deb
libdpkg-dev_1.15.8.6_amd64.deb
to main/d/dpkg/libdpkg-dev_1.15.8.6_amd64.deb
libdpkg-perl_1.15.8.6_all.deb
to main/d/dpkg/libdpkg-perl_1.15.8.6_all.deb
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to [email protected],
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Guillem Jover <[email protected]> (supplier of updated dpkg package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing [email protected])
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1
Format: 1.8
Date: Thu, 25 Nov 2010 07:10:48 +0100
Source: dpkg
Binary: libdpkg-dev dpkg dpkg-dev libdpkg-perl dselect
Architecture: source amd64 all
Version: 1.15.8.6
Distribution: unstable
Urgency: low
Maintainer: Dpkg Developers <[email protected]>
Changed-By: Guillem Jover <[email protected]>
Description:
dpkg - Debian package management system
dpkg-dev - Debian package development tools
dselect - Debian package management front-end
libdpkg-dev - Debian package management static library
libdpkg-perl - Dpkg perl modules
Closes: 584254 588339 595455 595927 596168 596519 597023 597651 598473 599923
600075 600240 601852 602518 604769
Changes:
dpkg (1.15.8.6) unstable; urgency=low
.
[ Raphaël Hertzog ]
* Ensure debian/source/local-options is always excluded from the source
package even if the user provides customized -i or -I options.
Closes: #597023
* Fix Dpkg::Version's handling of version with a debian revision but an
empty version (e.g. "-0.1"). Thanks to James Vega <[email protected]>
for the patch. Closes: #597651
* With "3.0 (quilt)" source package, create .pc/.quilt_series with the
correct series file if the source package provides vendor specific patch
sets.
.
[ Guillem Jover ]
* Disable by default usage of synchronous sync(2), as it causes undesired
I/O on unrelated file systems. Closes: #588339, #595927, #600075
* Add new --force-unsafe-io to disable safe I/O operations on unpack.
Closes: #584254
.
[ Updated man page translations ]
* French (Christian Perrier). Including a typo fix and a typographical
change reported by Vincent Danjean. Closes: #601852
* Spanish (Omar Campagne). Closes: #596519
.
[ Updated programs translations ]
* Basque (Iñaki Larrañaga Murgoitio). Closes: #599923
* Catalan (Jordi Mallach).
* Danish (Ask Hjorth Larsen). Closes: #600240
* German (Sven Joachim). Improved by Holger Wansing.
* Italian (Pietro Battiston). Fix translation of "however". Closes: #602518
* Portuguese (Miguel Figueiredo). Closes: #596168
* Romanian (Andrei Popescu). Closes: #604769
* Russian (Yuri Kozlov). Closes: #595455
* Vietnamese (Clytie Siddall). Closes: #598473
.
[ Updated scripts translations ]
* Catalan (Jordi Mallach).
* German (Sven Joachim).
.
[ Updated dselect translations ]
* Catalan (Jordi Mallach).
* German (Sven Joachim).
Checksums-Sha1:
0ac67a10e335d0ec5375ac523998546403525ff9 1208 dpkg_1.15.8.6.dsc
ebc9a6087f8f8c56c973f26f9bdb17ef1c570f0c 5222815 dpkg_1.15.8.6.tar.bz2
7a6e46450c5d8d89746a5ba16a6ed7b9306b2a07 426852 libdpkg-dev_1.15.8.6_amd64.deb
ba234426fb283f292efc9a0b202a09b5c579b504 2338026 dpkg_1.15.8.6_amd64.deb
196702535a3a609f134375d5d3c39bb26663f4ca 894426 dselect_1.15.8.6_amd64.deb
e6affdd25dcdc0b9c73e5059bc95f34531ed52e4 801736 dpkg-dev_1.15.8.6_all.deb
47cb193e3096afb8035632a1b6bae0da9771c226 682848 libdpkg-perl_1.15.8.6_all.deb
Checksums-Sha256:
a4355f87fa1466edcefff224182ae7824d1469d1ddc28126d00cd361c611e0a9 1208
dpkg_1.15.8.6.dsc
b319621a4d0f9fa7b356b4def978bad0b18b944405f2be9eace5e2713b5f1f49 5222815
dpkg_1.15.8.6.tar.bz2
eedd636f39cb03a28758558cd6f4b700a2168c1adb9e4c8a59ab4ca07549bad5 426852
libdpkg-dev_1.15.8.6_amd64.deb
6d3265e9aa6ef2d7ec341687df80e55da5996daa5d581e1864e9466bc7c36321 2338026
dpkg_1.15.8.6_amd64.deb
b4a073849a3944c4f7b13b67cc6db8a78e8edd38ca50d86134f7aa1977c40f4b 894426
dselect_1.15.8.6_amd64.deb
3368e9efe1206c720b92c69b89f3f43e8a0a4d13949aa24deceb02b2452ef756 801736
dpkg-dev_1.15.8.6_all.deb
6a05978ee576c2848aa0f748cd0208547c0f39856a782ba227db331b7cf24bd3 682848
libdpkg-perl_1.15.8.6_all.deb
Files:
6d69ce9fd47b97aef50f5ba1209c4b24 1208 admin required dpkg_1.15.8.6.dsc
4102648a08a4416bfc3e4f4275a438e4 5222815 admin required dpkg_1.15.8.6.tar.bz2
1c45d231edef121f36a575a6322e928b 426852 libdevel optional
libdpkg-dev_1.15.8.6_amd64.deb
4525021598d6810dcb3fa114e5433ba1 2338026 admin required dpkg_1.15.8.6_amd64.deb
b5662cfacfb969c3db3668cab230729b 894426 admin optional
dselect_1.15.8.6_amd64.deb
85db60da1da45941fd1789d156bdbd01 801736 utils optional
dpkg-dev_1.15.8.6_all.deb
00816223970f7700379ef13d2110878b 682848 perl optional
libdpkg-perl_1.15.8.6_all.deb
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
iEYEARECAAYFAkzuA9IACgkQuW9ciZ2SjJtWeACgw/Em7Z5+xaFqj6lJk3DpESzc
y/EAnA0oig96OBJD4pOI1fM62qXmzP00
=ugvO
-----END PGP SIGNATURE-----
--- End Message ---
