Re: getdents - ext4 vs btrfs performance

2012-03-15 Thread Jacek Luczak
2012/3/11 Ted Ts'o : > > Well, my goal in proposing this optimization is that helps for the > "medium size" directories in the cold cache case.  The ext4 user who > first kicked off this thread was using his file system for an SVN > server, as I recall.  I could easily believe that he has thousand

Re: getdents - ext4 vs btrfs performance

2012-03-15 Thread Jacek Luczak
2012/3/10 Ted Ts'o : > Hey Jacek, > > I'm curious parameters of the set of directories on your production > server.  On an ext4 file system, assuming you've copied the > directories over, what are the result of this command pipeline when > you are cd'ed into the top of the directory hierarchy of in

Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-12 Thread Jacek Luczak
2012/3/10 Jacek Luczak : > 2012/3/9 David Sterba : >> On Fri, Mar 09, 2012 at 12:08:12PM +0100, Jacek Luczak wrote: >>> For this one I've created also a report [1]. >>> > >>> > so there is probably other problem in reservations and it just blew up &g

Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-11 Thread Jacek Luczak
2012/3/10 Jacek Luczak : > 2) A *regression* in 3.3.0-rc6-00197-g9f8050c > - completely unusable as reports ENOSPC > - to reproduce, mount volume and issue: > # CNT=1 ; while [ $CNT -lt 1 ] ; do  rm -f /btrfs/dd ; ! touch > /btrfs/dd && echo "$CNT" &&

Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-09 Thread Jacek Luczak
2012/3/9 David Sterba : > On Fri, Mar 09, 2012 at 12:08:12PM +0100, Jacek Luczak wrote: >> For this one I've created also a report [1]. >> > >> > so there is probably other problem in reservations and it just blew up >> > during >> > the unlink c

Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-09 Thread Jacek Luczak
2012/3/9 Jacek Luczak : > 2012/3/9 David Sterba : >> On Fri, Mar 09, 2012 at 03:33:24PM +0100, Jacek Luczak wrote: >>> > Those two issues go inline. After a longer while of WARN_ON the BUG_ON >>> > hit again. >>> >>> One more observation. Host is

Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-09 Thread Jacek Luczak
2012/3/9 David Sterba : > On Fri, Mar 09, 2012 at 03:33:24PM +0100, Jacek Luczak wrote: >> > Those two issues go inline. After a longer while of WARN_ON the BUG_ON >> > hit again. >> >> One more observation. Host is running builds from CI system. After >>

Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-09 Thread Jacek Luczak
2012/3/9 Jacek Luczak : > 2012/3/9 David Sterba : >> On Fri, Mar 09, 2012 at 12:08:12PM +0100, Jacek Luczak wrote: >>> For this one I've created also a report [1]. >>> > >>> > so there is probably other problem in reservations and it just blew up &g

Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-09 Thread Jacek Luczak
2012/3/9 David Sterba : > On Fri, Mar 09, 2012 at 12:08:12PM +0100, Jacek Luczak wrote: >> For this one I've created also a report [1]. >> > >> > so there is probably other problem in reservations and it just blew up >> > during >> > the unlink c

Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-09 Thread Jacek Luczak
2012/3/9 David Sterba : > On Fri, Mar 09, 2012 at 09:31:25AM +0800, Liu Bo wrote: >> > There were quite many things happening in the system at that time. >> > Can't really tell what could trigger this. >> > >> > Complete logs: http://91.234.146.107/~difrost/logs/tampere_log.gz >> > >> So are these

Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-09 Thread Jacek Luczak
2012/3/9 Chris Samuel : > On 09/03/12 12:31, Liu Bo wrote: > >> So are these warnings based on the latest upstream of btrfs? > > Looks like it was 3.2.7, his oops said: > > Pid: 1488, comm: mips-wrs-linux- Tainted: G        W    3.2.7 #2 HP Yep, that's 3.2.7. Now I can't upgrade to latest upstream

Re: kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-08 Thread Jacek Luczak
2012/3/8 David Sterba : > On Thu, Mar 08, 2012 at 01:10:45PM +0100, Jacek Luczak wrote: >> kernel BUG at fs/btrfs/delayed-inode.c:1466! > > 1461         ret = btrfs_delayed_item_reserve_metadata(trans, root, item); > 1462         /* > 1463          * we have reserved enough

kernel BUG at fs/btrfs/delayed-inode.c:1466!

2012-03-08 Thread Jacek Luczak
Hi, this shown up today. I had to do a hard reboot as graceful hanged on sync(). [ cut here ] kernel BUG at fs/btrfs/delayed-inode.c:1466! invalid opcode: [#1] SMP CPU 10 Modules linked in: btrfs zlib_deflate lzo_compress ipmi_devintf autofs4 be2iscsi iscsi_boot_sysfs

Re: WARNING: at fs/btrfs/extent-tree.c:5985

2012-03-08 Thread Jacek Luczak
2012/3/6 Jacek Luczak : > Hi All, > > I've noticed today below WARN_ON from btrfs. Google shows hits in the > same place ([1] and [2]) but the path is different. It could happen > when svn checout or few rsyncs were running - now I'm not able to put > in correct timings

WARNING: at fs/btrfs/extent-tree.c:5985

2012-03-06 Thread Jacek Luczak
Hi All, I've noticed today below WARN_ON from btrfs. Google shows hits in the same place ([1] and [2]) but the path is different. It could happen when svn checout or few rsyncs were running - now I'm not able to put in correct timings. There's btrfs_item_offset() in backtrace and I was not able t

Re: getdents - ext4 vs btrfs performance

2012-03-05 Thread Jacek Luczak
2012/3/4 Jacek Luczak : > 2012/3/3 Jacek Luczak : >> 2012/3/2 Chris Mason : >>> On Fri, Mar 02, 2012 at 03:16:12PM +0100, Jacek Luczak wrote: >>>> 2012/3/2 Chris Mason : >>>> > On Fri, Mar 02, 2012 at 11:05:56AM +0100, Jacek Luczak wrote: >>>&g

Re: getdents - ext4 vs btrfs performance

2012-03-04 Thread Jacek Luczak
2012/3/3 Jacek Luczak : > 2012/3/2 Chris Mason : >> On Fri, Mar 02, 2012 at 03:16:12PM +0100, Jacek Luczak wrote: >>> 2012/3/2 Chris Mason : >>> > On Fri, Mar 02, 2012 at 11:05:56AM +0100, Jacek Luczak wrote: >>> >> >>> >> I've

Re: getdents - ext4 vs btrfs performance

2012-03-03 Thread Jacek Luczak
2012/3/2 Chris Mason : > On Fri, Mar 02, 2012 at 03:16:12PM +0100, Jacek Luczak wrote: >> 2012/3/2 Chris Mason : >> > On Fri, Mar 02, 2012 at 11:05:56AM +0100, Jacek Luczak wrote: >> >> >> >> I've took both on tests. The subject is acp and spd_readdir

Re: getdents - ext4 vs btrfs performance

2012-03-02 Thread Jacek Luczak
2012/3/2 Chris Mason : > On Fri, Mar 02, 2012 at 11:05:56AM +0100, Jacek Luczak wrote: >> >> I've took both on tests. The subject is acp and spd_readdir used with >> tar, all on ext4: >> 1) acp: http://91.234.146.107/~difrost/seekwatcher/acp_ext4.png >> 2)

Re: getdents - ext4 vs btrfs performance

2012-03-02 Thread Jacek Luczak
2012/3/1 Chris Mason : > On Wed, Feb 29, 2012 at 11:44:31PM -0500, Theodore Tso wrote: >> You might try sorting the entries returned by readdir by inode number before >> you stat them.    This is a long-standing weakness in ext3/ext4, and it has >> to do with how we added hashed tree indexes to d

Re: getdents - ext4 vs btrfs performance

2012-03-02 Thread Jacek Luczak
2012/3/1 Ted Ts'o : > On Thu, Mar 01, 2012 at 03:43:41PM +0100, Jacek Luczak wrote: >> >> Yep, ext4 is close to my wife's closet. >> > > Were all of the file systems freshly laid down, or was this an aged > ext4 file system? Always fresh, recreated for eac

Re: getdents - ext4 vs btrfs performance

2012-03-01 Thread Jacek Luczak
2012/3/1 Chris Mason : > On Thu, Mar 01, 2012 at 03:43:41PM +0100, Jacek Luczak wrote: >> 2012/3/1 Chris Mason : >> > XFS will probably beat btrfs in this test.  Their directory indexes >> > reflect on disk layout very well. >> >> True, but not that fast on

Re: getdents - ext4 vs btrfs performance

2012-03-01 Thread Jacek Luczak
2012/3/1 Chris Mason : > On Thu, Mar 01, 2012 at 03:03:53PM +0100, Jacek Luczak wrote: >> 2012/3/1 Hillf Danton : >> > On Thu, Mar 1, 2012 at 9:35 PM, Jacek Luczak >> > wrote: >> >> >> >> While I was about to grab acp I've noticed seekwa

Re: getdents - ext4 vs btrfs performance

2012-03-01 Thread Jacek Luczak
2012/3/1 Hillf Danton : > On Thu, Mar 1, 2012 at 9:35 PM, Jacek Luczak wrote: >> >> While I was about to grab acp I've noticed seekwatcher with made my day :) >> >> seekwatcher run of tar cf to eliminate writes (all done on 3.2.7): >> 1) btrfs: http

Re: getdents - ext4 vs btrfs performance

2012-03-01 Thread Jacek Luczak
2012/2/29 Jacek Luczak : > 2012/2/29 Chris Mason : >> On Wed, Feb 29, 2012 at 03:07:45PM +0100, Jacek Luczak wrote: >> >> [ btrfs faster than ext for find and cp -a ] >> >>> 2012/2/29 Jacek Luczak : >>> >>> I will try to answer the question f

Re: getdents - ext4 vs btrfs performance

2012-02-29 Thread Jacek Luczak
2012/2/29 Chris Mason : > On Wed, Feb 29, 2012 at 03:07:45PM +0100, Jacek Luczak wrote: > > [ btrfs faster than ext for find and cp -a ] > >> 2012/2/29 Jacek Luczak : >> >> I will try to answer the question from the broken email I've sent. >> >>

Re: getdents - ext4 vs btrfs performance

2012-02-29 Thread Jacek Luczak
2012/2/29 Jacek Luczak : > 2012/2/29 Jacek Luczak : >> Hi Chris, >> >> the last one was borked :) Please check this one. >> >> -jacek >> >> 2012/2/29 Jacek Luczak : >>> Hi All, >>> >>> /*Sorry for sending incomplete email,

Re: getdents - ext4 vs btrfs performance

2012-02-29 Thread Jacek Luczak
2012/2/29 Jacek Luczak : > Hi Chris, > > the last one was borked :) Please check this one. > > -jacek > > 2012/2/29 Jacek Luczak : >> Hi All, >> >> /*Sorry for sending incomplete email, hit wrong button :) I guess I >> can't use Gmail */ >&g

Re: getdents - ext4 vs btrfs performance

2012-02-29 Thread Jacek Luczak
Hi Chris, the last one was borked :) Please check this one. -jacek 2012/2/29 Jacek Luczak : > Hi All, > > /*Sorry for sending incomplete email, hit wrong button :) I guess I > can't use Gmail */ > > Long story short: We've found that operations on a directory structu

getdents - ext4 vs btrfs performance

2012-02-29 Thread Jacek Luczak
Hi All, /*Sorry for sending incomplete email, hit wrong button :) I guess I can't use Gmail */ Long story short: We've found that operations on a directory structure holding many dirs takes ages on ext4. The Question: Why there's that huge difference in ext4 and btrfs? See below test results for

chris.ma...@oracle.com, Al Viro , Ted Ts'o

2012-02-29 Thread Jacek Luczak
Hi All, /*Sorry for sending incomplete email, hit wrong button :) */ Long story short: We've found that operations on a directory structure holding many dirs takes ages on ext4. The Question: Why there's that huge difference in ext4 and btrfs? See below test results for real values. Background:

getdents - ext4 vs btrfs performance

2012-02-29 Thread Jacek Luczak
Hi All, Long story short: We've found that operations on a directory structure holding many dirs takes ages on ext4. The Question: Why there's that huge difference in ext4 and btrfs? See below test results for real values. Background: I had to backup a Jenkins directory holding workspace for few