Re: Pre-approval request for dpkg sync() changes for squeeze

2010-11-15 Thread Sven Joachim 
On 2010-11-15 08:21 +0100, Raphael Hertzog wrote:

> On Wed, 10 Nov 2010, Philipp Kern wrote:
>> On Tue, Nov 09, 2010 at 07:50:23PM +0100, Philipp Kern wrote:
>> > On Fri, Oct 22, 2010 at 11:35:54AM +0200, Guillem Jover wrote:
>> > >   1) Switch back from sync() to fsync() before rename() (while keeping
>> > >  the sync() code around for the benefit of other distributions
>> > >  that might not want to switch just yet). So to avoid unrelated
>> > >  I/O when there's background work being done for example. This
>> > >  hack also only works on Linux where sync() is synchronous, so
>> > >  it would unify that code path for all dpkg supported platforms.
>> > > 
>> > >  Bug: #588339
>> > > 
>> > >   
>> > > 
>> > I'm still not sure we have all needed information.  Would you mind doing
>> > a matrix what this change would cause?  I.e. if we get performance
>> > regressions on ext4, btrfs or even ext3 with the squeeze kernel (we're
>> > really only interested in that one, not some random other revision)
>> > or if we even get data safety regressions.
>> 
>> I also forgot to throw nfs into the picture.
>
> I'm sorry, I won't have the time to do new benchmarks on this. 
>
> The only benchmarks we have have been made by Sven Joachim:
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=578635#20
> (asyncsync is the switch to sync() instead of fsync() so the opposite of
> the above patch)
>
> He mentionned that without the sync() trick it takes 3 to 5 times longer
> to unpack a package.

Even longer actually, see the figures below.

> Sven, would you have time to provide some of the stats asked by the
> release team?

I can only test ext4, here are some samples of dpkg unpacking a large
package (dpkg --unpack --no-triggers emacs23-common_23.2+1-5.1_all.deb),
leaving out user and sys times since those do not vary much (~ 0.5
seconds in every case):

dpkg versionCache   mount options  unpack time
̅̅̅
1.15.8.5colddefaults7.803s 
1.15.8.5warmdefaults5.283s 
1.15.8.5coldnodelalloc  7.608s 
1.15.8.5warmnodelalloc  3.783s 
1.15.7  colddefaults   40.429s 
1.15.7  warmdefaults   37.848s 
1.15.7  coldnodelalloc  7.945s 
1.15.7  warmnodelalloc  3.524s

All this is with a standard squeeze kernel on an otherwise idle system.
It should be noted that with lots of other disk activity such as writing
to USB disks, the figures in dpkg 1.15.8.5 can become much worse and
dpkg might even stall because of the many sync() calls:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=595927.

As far as ext4 is concerned, switching back to fsync() seems to be
acceptable only if the filesystem is mounted with the nodelalloc
option.  Maybe the installer should set this up.

Regards,
Sven


-- 
To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/87tyjjufnc@turtle.gmx.de

[bts-link] source package dpkg

2010-11-15 Thread bts-link-upstream 
#
# bts-link upstream status pull for source package dpkg
# see http://lists.debian.org/debian-devel-announce/2006/05/msg1.html
#

user bts-link-upstr...@lists.alioth.debian.org

# remote status report for #600075 (http://bugs.debian.org/600075)
#  * http://bugzilla.kernel.org/show_bug.cgi?id=18632
#  * remote status changed: (?) -> NEW
usertags 600075 + status-NEW

# remote status report for #600075 (http://bugs.debian.org/600075)
#  * http://bugzilla.kernel.org/show_bug.cgi?id=18632
#  * remote status changed: (?) -> NEW
usertags 600075 + status-NEW

thanks


--
To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: 
http://lists.debian.org/20101115163331.12325.446.btsl...@busoni.debian.org

Re: Pre-approval request for dpkg sync() changes for squeeze

2010-11-15 Thread Philipp Kern 
Dear kernel team,

On Mon, Nov 15, 2010 at 09:58:47AM +0100, Sven Joachim wrote:
> > I'm sorry, I won't have the time to do new benchmarks on this. 
> >
> > The only benchmarks we have have been made by Sven Joachim:
> > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=578635#20
> > (asyncsync is the switch to sync() instead of fsync() so the opposite of
> > the above patch)
> >
> > He mentionned that without the sync() trick it takes 3 to 5 times longer
> > to unpack a package.
> 
> Even longer actually, see the figures below.
> 
> > Sven, would you have time to provide some of the stats asked by the
> > release team?
> 
> I can only test ext4, here are some samples of dpkg unpacking a large
> package (dpkg --unpack --no-triggers emacs23-common_23.2+1-5.1_all.deb),
> leaving out user and sys times since those do not vary much (~ 0.5
> seconds in every case):
> 
> dpkg version  Cache   mount options  unpack time
> ̅̅̅
> 1.15.8.5colddefaults7.803s 
> 1.15.8.5warmdefaults5.283s 
> 1.15.8.5coldnodelalloc  7.608s 
> 1.15.8.5warmnodelalloc  3.783s 
> 1.15.7  colddefaults   40.429s 
> 1.15.7  warmdefaults   37.848s 
> 1.15.7  coldnodelalloc  7.945s 
> 1.15.7  warmnodelalloc  3.524s
> 
> All this is with a standard squeeze kernel on an otherwise idle system.
> It should be noted that with lots of other disk activity such as writing
> to USB disks, the figures in dpkg 1.15.8.5 can become much worse and
> dpkg might even stall because of the many sync() calls:
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=595927.
> 
> As far as ext4 is concerned, switching back to fsync() seems to be
> acceptable only if the filesystem is mounted with the nodelalloc
> option.  Maybe the installer should set this up.

and I don't suppose we could make that the default?  Is there anything
else the dpkg developers can try to be portable and still not be
sacrificing performance?

Kind regards
Philipp Kern 


signature.asc
Description: Digital signature

Re: Transferring conffiles between packages (Re: Bug#564254: conflicting /etc/bash_completion)

2010-11-15 Thread Jonathan Nieder 
tags 564254 - unreproducible
reassign 564254 bash-completion 1:1.1-3
found 564254 bash-completion/1:1.2-2
quit
[resetting cc list]

Guillem Jover wrote:

> My guess would be that bash got upgraded to the package w/o the
> obsolete conffile before the fixed dpkg on those systems.
> 
> A proof of that I guess, might be checking if the file has the obsolete
> flag, if it does not then it was installed by a buggy dpkg.
> 
>   $ dpkg-query -W -f '${Conffiles}\n' bash | grep bash_completion
> 
> And an unversioned Replaces in bash-completion would be the correct way
> to handle that. Fixing that in bash would imply removing the file on
> upgrade, removing it on remove or purge would not really happen (bash
> is Essential). And as such it would need to check if bash-completion is
> installed, to not remove a file it does not own, or check if it has the
> conffile listed in the Conffile field w/o the obsolete flag? etc, which
> seems overcomplex and just wrong, compared with just a Replaces field.

Thanks, Guillem.  Reassigning to bash-completion.

David et al, this bug is a request for unversioned Replaces: by
bash-completion on bash.  But feel free to do what you want with it;
it's yours now. :)

Regards,
Jonathan


-- 
To UNSUBSCRIBE, email to debian-dpkg-requ...@lists.debian.org
with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org
Archive: http://lists.debian.org/20101115231107.ga22...@burratino