Re: Ceph on btrfs 3.4rc

2012-05-24 Thread Christian Brunner
Same thing here. I've tried really hard, but even after 12 hours I wasn't able to get a single warning from btrfs. I think you cracked it! Thanks, Christian 2012/5/24 Martin Mailand mar...@tuxadero.com: Hi, the ceph cluster is running under heavy load for the last 13 hours without a

Re: Ceph on btrfs 3.4rc

2012-05-23 Thread Christian Brunner
2012/5/22 Josef Bacik jo...@redhat.com: Yeah you would also need to change orphan_meta_reserved.  I fixed this by just taking the BTRFS_I(inode)-lock when messing with these since we don't want to take up all that space in the inode just for a marker.  I ran this patch for 3 hours with no

Re: Ceph on btrfs 3.4rc

2012-05-22 Thread Christian Brunner
2012/5/21 Miao Xie mi...@cn.fujitsu.com: Hi Josef, On fri, 18 May 2012 15:01:05 -0400, Josef Bacik wrote: diff --git a/fs/btrfs/btrfs_inode.h b/fs/btrfs/btrfs_inode.h index 9b9b15f..492c74f 100644 --- a/fs/btrfs/btrfs_inode.h +++ b/fs/btrfs/btrfs_inode.h @@ -57,9 +57,6 @@ struct

Re: Ceph on btrfs 3.4rc

2012-05-17 Thread Christian Brunner
2012/5/17 Josef Bacik jo...@redhat.com: On Thu, May 17, 2012 at 05:12:55PM +0200, Martin Mailand wrote: Hi Josef, no there was nothing above. Here the is another dmesg output. Hrm ok give this a try and hopefully this is it, still couldn't reproduce. Thanks, Josef Well, I hate to say it,

Re: Ceph on btrfs 3.4rc

2012-05-11 Thread Christian Brunner
2012/5/10 Josef Bacik jo...@redhat.com: On Fri, Apr 27, 2012 at 01:02:08PM +0200, Christian Brunner wrote: Am 24. April 2012 18:26 schrieb Sage Weil s...@newdream.net: On Tue, 24 Apr 2012, Josef Bacik wrote: On Fri, Apr 20, 2012 at 05:09:34PM +0200, Christian Brunner wrote: After running

Re: Ceph on btrfs 3.4rc

2012-05-04 Thread Christian Brunner
2012/5/3 Josef Bacik jo...@redhat.com: On Thu, May 03, 2012 at 09:38:27AM -0700, Josh Durgin wrote: On Thu, 3 May 2012 11:20:53 -0400, Josef Bacik jo...@redhat.com wrote: On Thu, May 03, 2012 at 08:17:43AM -0700, Josh Durgin wrote: Yeah all that was in the right place, I rebooted and I

Re: Ceph on btrfs 3.4rc

2012-04-30 Thread Christian Brunner
2012/4/29 tsuna tsuna...@gmail.com: On Fri, Apr 20, 2012 at 8:09 AM, Christian Brunner christ...@brunner-muc.de wrote: After running ceph on XFS for some time, I decided to try btrfs again. Performance with the current for-linux-min branch and big metadata is much better. I've heard

Re: Ceph on btrfs 3.4rc

2012-04-27 Thread Christian Brunner
Am 24. April 2012 18:26 schrieb Sage Weil s...@newdream.net: On Tue, 24 Apr 2012, Josef Bacik wrote: On Fri, Apr 20, 2012 at 05:09:34PM +0200, Christian Brunner wrote: After running ceph on XFS for some time, I decided to try btrfs again. Performance with the current for-linux-min branch

Re: Ceph on btrfs 3.4rc

2012-04-23 Thread Christian Brunner
Am 20. April 2012 17:09 schrieb Christian Brunner christ...@brunner-muc.de: After running ceph on XFS for some time, I decided to try btrfs again. Performance with the current for-linux-min branch and big metadata is much better. The only problem (?) I'm still seeing is a warning that seems

Ceph on btrfs 3.4rc

2012-04-20 Thread Christian Brunner
After running ceph on XFS for some time, I decided to try btrfs again. Performance with the current for-linux-min branch and big metadata is much better. The only problem (?) I'm still seeing is a warning that seems to occur from time to time: [87703.784552] [ cut here ]

Re: Strange prformance degradation when COW writes happen at fixed offsets

2012-02-27 Thread Christian Brunner
2012/2/24 Nik Markovic nmarkovi.nav...@gmail.com: To add... I also tried nodatasum (only) and nodatacow otions. I found somewhere that nodatacow doesn't really mean tthat COW is disabled. Test data is still the same - CPU spikes and times are the same. On Fri, Feb 24, 2012 at 2:38 PM, Nik

Re: Btrfs slowdown with ceph (how to reproduce)

2012-01-23 Thread Christian Brunner
2012/1/23 Chris Mason chris.ma...@oracle.com: On Mon, Jan 23, 2012 at 01:19:29PM -0500, Josef Bacik wrote: On Fri, Jan 20, 2012 at 01:13:37PM +0100, Christian Brunner wrote: As you might know, I have been seeing btrfs slowdowns in our ceph cluster for quite some time. Even with the latest

Btrfs slowdown with ceph (how to reproduce)

2012-01-20 Thread Christian Brunner
As you might know, I have been seeing btrfs slowdowns in our ceph cluster for quite some time. Even with the latest btrfs code for 3.3 I'm still seeing these problems. To make things reproducible, I've now written a small test, that imitates ceph's behavior: On a freshly created btrfs filesystem

Re: [3.2-rc7] slowdown, warning + oops creating lots of files

2012-01-12 Thread Christian Brunner
2012/1/7 Christian Brunner c...@muc.de: 2012/1/5 Chris Mason chris.ma...@oracle.com: On Fri, Jan 06, 2012 at 07:12:16AM +1100, Dave Chinner wrote: On Thu, Jan 05, 2012 at 02:45:00PM -0500, Chris Mason wrote: On Thu, Jan 05, 2012 at 01:46:57PM -0500, Chris Mason wrote: Unfortunately

Re: [3.2-rc7] slowdown, warning + oops creating lots of files

2012-01-07 Thread Christian Brunner
2012/1/5 Chris Mason chris.ma...@oracle.com: On Fri, Jan 06, 2012 at 07:12:16AM +1100, Dave Chinner wrote: On Thu, Jan 05, 2012 at 02:45:00PM -0500, Chris Mason wrote: On Thu, Jan 05, 2012 at 01:46:57PM -0500, Chris Mason wrote: On Thu, Jan 05, 2012 at 10:01:22AM +1100, Dave Chinner wrote:

Re: WARNING: at fs/btrfs/extent-tree.c:5980

2011-12-13 Thread Christian Brunner
Sorry - I forgot to mention, that I'm still seeing this with: [PATCH] Btrfs: update global block_rsv when creating a new block group Christian 2011/12/13 Christian Brunner c...@muc.de: Hi, with the latest btrfs for-linus I'm seeing seeing occasional btrfs_alloc_free_block warnings

Re: [PATCH 02/20] Btrfs: initialize new bitmaps' list

2011-12-12 Thread Christian Brunner
2011/12/12 Alexandre Oliva ol...@lsd.ic.unicamp.br: On Dec  7, 2011, Christian Brunner c...@muc.de wrote: With this patch applied I get much higher write-io values than without it. Some of the other patches help to reduce the effect, but it's still significant. iostat on an unpatched node

Re: avoid redundant block group free-space checks

2011-12-12 Thread Christian Brunner
2011/12/12 Alexandre Oliva ol...@lsd.ic.unicamp.br: It was pointed out to me that the test for enough free space in a block group was wrong in that it would skip a block group that had most of its free space reserved by a cluster. I offer two mutually exclusive, (so far) very lightly tested

Re: [PATCH 02/20] Btrfs: initialize new bitmaps' list

2011-12-09 Thread Christian Brunner
2011/12/7 Christian Brunner c...@muc.de: 2011/12/1 Christian Brunner c...@muc.de: 2011/12/1 Alexandre Oliva ol...@lsd.ic.unicamp.br: On Nov 29, 2011, Christian Brunner c...@muc.de wrote: When I'm doing havy reading in our ceph cluster. The load and wait-io on the patched servers is higher

Re: [PATCH 02/20] Btrfs: initialize new bitmaps' list

2011-12-07 Thread Christian Brunner
2011/12/1 Christian Brunner c...@muc.de: 2011/12/1 Alexandre Oliva ol...@lsd.ic.unicamp.br: On Nov 29, 2011, Christian Brunner c...@muc.de wrote: When I'm doing havy reading in our ceph cluster. The load and wait-io on the patched servers is higher than on the unpatched ones. That's

Re: [PATCH 02/20] Btrfs: initialize new bitmaps' list

2011-11-29 Thread Christian Brunner
2011/11/28 Alexandre Oliva ol...@lsd.ic.unicamp.br: We're failing to create clusters with bitmaps because setup_cluster_no_bitmap checks that the list is empty before inserting the bitmap entry in the list for setup_cluster_bitmap, but the list field is only initialized when it is restored

Re: WARNING: at fs/btrfs/inode.c:2198 btrfs_orphan_commit_root+0xa8/0xc0

2011-11-26 Thread Christian Brunner
2011/11/26 Stefan Kleijkers ste...@unilogicnetworks.net: Hello Josef, I've new results, is this the trace you are looking for? Trace of OSD0: http://pastebin.com/gddLBXE4 Dmesg of OSD0: http://pastebin.com/Uebzgkjv OSD1 crashed a while later with the same messages. Stefan Hi Josef, I

Re: BUG at fs/btrfs/inode.c:1587

2011-11-16 Thread Christian Brunner
2011/11/16 Chris Mason chris.ma...@oracle.com: On Tue, Nov 15, 2011 at 09:19:53AM +0100, Christian Brunner wrote: Hi, this time I've hit a new bug. This happened while ceph was rebuilding his filestore (heavy io). The btrfs version is from 3.2-rc1, applied to a 3.0 kernel. This one means

BUG at fs/btrfs/inode.c:1587

2011-11-15 Thread Christian Brunner
Hi, this time I've hit a new bug. This happened while ceph was rebuilding his filestore (heavy io). The btrfs version is from 3.2-rc1, applied to a 3.0 kernel. Regards, Christian [28981.550478] [ cut here ] [28981.555625] kernel BUG at fs/btrfs/inode.c:1587!

Re: WARNING: at fs/btrfs/inode.c:2198 btrfs_orphan_commit_root+0xa8/0xc0

2011-11-09 Thread Christian Brunner
2011/11/9 Stefan Kleijkers ste...@unilogicnetworks.net: Hello, I'm seeing a lot of warnings in dmesg with a BTRFS filesystem. I'm using the 3.1 kernel, I found a patch for these warnings ( http://marc.info/?l=linux-btrfsm=131547325515336w=2)

Re: ceph on btrfs [was Re: ceph on non-btrfs file systems]

2011-10-31 Thread Christian Brunner
2011/10/31 Christian Brunner c...@muc.de: 2011/10/31 Christian Brunner c...@muc.de: The patch didn't hurt, but I've to tell you that I'm still seeing the same old problems. Load is going up again:  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND  5502 root      20   0

Re: ceph on btrfs [was Re: ceph on non-btrfs file systems]

2011-10-31 Thread Christian Brunner
2011/10/31 Christian Brunner c...@muc.de: 2011/10/31 Christian Brunner c...@muc.de: 2011/10/31 Christian Brunner c...@muc.de: The patch didn't hurt, but I've to tell you that I'm still seeing the same old problems. Load is going up again:  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM

Re: ceph on btrfs [was Re: ceph on non-btrfs file systems]

2011-10-27 Thread Christian Brunner
2011/10/27 Josef Bacik jo...@redhat.com: On Tue, Oct 25, 2011 at 01:56:48PM +0200, Christian Brunner wrote: 2011/10/24 Josef Bacik jo...@redhat.com: On Mon, Oct 24, 2011 at 10:06:49AM -0700, Sage Weil wrote: [adding linux-btrfs to cc] Josef, Chris, any ideas on the below issues

Re: ceph on btrfs [was Re: ceph on non-btrfs file systems]

2011-10-26 Thread Christian Brunner
2011/10/26 Sage Weil s...@newdream.net: On Wed, 26 Oct 2011, Christian Brunner wrote: Christian, have you tweaked those settings in your ceph.conf?  It would be something like 'journal dio = false'.  If not, can you verify that directio shows true when the journal is initialized from

Re: ceph on btrfs [was Re: ceph on non-btrfs file systems]

2011-10-26 Thread Christian Brunner
2011/10/26 Christian Brunner c...@muc.de: 2011/10/25 Josef Bacik jo...@redhat.com: On Tue, Oct 25, 2011 at 04:15:45PM -0400, Chris Mason wrote: On Tue, Oct 25, 2011 at 11:05:12AM -0400, Josef Bacik wrote: On Tue, Oct 25, 2011 at 04:25:02PM +0200, Christian Brunner wrote: Attached

Re: ceph on btrfs [was Re: ceph on non-btrfs file systems]

2011-10-25 Thread Christian Brunner
2011/10/25 Josef Bacik jo...@redhat.com: On Tue, Oct 25, 2011 at 01:56:48PM +0200, Christian Brunner wrote: 2011/10/24 Josef Bacik jo...@redhat.com: On Mon, Oct 24, 2011 at 10:06:49AM -0700, Sage Weil wrote: [adding linux-btrfs to cc] Josef, Chris, any ideas on the below issues

Re: ceph on btrfs [was Re: ceph on non-btrfs file systems]

2011-10-25 Thread Christian Brunner
2011/10/25 Josef Bacik jo...@redhat.com: On Tue, Oct 25, 2011 at 04:25:02PM +0200, Christian Brunner wrote: 2011/10/25 Josef Bacik jo...@redhat.com: On Tue, Oct 25, 2011 at 01:56:48PM +0200, Christian Brunner wrote: [...] In our Ceph-OSD server we have 4 disks with 4 btrfs filesystems

Re: ceph on btrfs [was Re: ceph on non-btrfs file systems]

2011-10-25 Thread Christian Brunner
2011/10/25 Sage Weil s...@newdream.net: On Tue, 25 Oct 2011, Josef Bacik wrote: At this point it seems like the biggest problem with latency in ceph-osd is not related to btrfs, the latency seems to all be from the fact that ceph-osd is fsyncing a block dev for whatever reason. There is one

Re: ceph on btrfs [was Re: ceph on non-btrfs file systems]

2011-10-25 Thread Christian Brunner
2011/10/25 Josef Bacik jo...@redhat.com: On Tue, Oct 25, 2011 at 04:15:45PM -0400, Chris Mason wrote: On Tue, Oct 25, 2011 at 11:05:12AM -0400, Josef Bacik wrote: On Tue, Oct 25, 2011 at 04:25:02PM +0200, Christian Brunner wrote: Attached is a perf-report. I have included the whole

Re: ceph on btrfs [was Re: ceph on non-btrfs file systems]

2011-10-24 Thread Christian Brunner
2011/10/24 Chris Mason chris.ma...@oracle.com: On Mon, Oct 24, 2011 at 03:51:47PM -0400, Josef Bacik wrote: On Mon, Oct 24, 2011 at 10:06:49AM -0700, Sage Weil wrote: [adding linux-btrfs to cc] Josef, Chris, any ideas on the below issues? On Mon, 24 Oct 2011, Christian Brunner wrote

Re: WARNING: at fs/btrfs/inode.c:2114

2011-10-20 Thread Christian Brunner
2011/10/20 Liu Bo liubo2...@cn.fujitsu.com: On 10/17/2011 11:23 PM, Christian Brunner wrote: 2011/10/11 Christian Brunner c...@muc.de: I have updated to a 3.0.6 kernel, with all the btrfs patches from josef's git repo this weekend. But I'm still seeing the following warning: Hi, Would

Re: WARNING: at fs/btrfs/inode.c:2114

2011-10-17 Thread Christian Brunner
2011/10/11 Christian Brunner c...@muc.de: 2011/10/11 Liu Bo liubo2...@cn.fujitsu.com: On 10/10/2011 12:41 AM, Christian Brunner wrote: I just realized that this is still the same warning I reported some month ago. I thought that this had been fixed

Re: [PATCH] Btrfs: allow us to overcommit our enospc reservations TEST THIS PLEASE!!!

2011-10-13 Thread Christian Brunner
2011/10/13 Josef Bacik jo...@redhat.com: [...] [  175.956273] kernel BUG at fs/btrfs/inode.c:2176! Ok I think I see what's happening, this patch replaces the previous one, let me know how it goes.  Thanks, Getting a slightly different BUG this time: Ok looks like I've fixed the

Re: WARNING: at fs/btrfs/inode.c:2114

2011-10-11 Thread Christian Brunner
2011/10/11 Liu Bo liubo2...@cn.fujitsu.com: On 10/10/2011 12:41 AM, Christian Brunner wrote: I just realized that this is still the same warning I reported some month ago. I thought that this had been fixed with 25d37af374263243214be9d912cbb46a8e469bc7 which is included in the kernel I'm

Re: WARNING: at fs/btrfs/inode.c:2114

2011-10-09 Thread Christian Brunner
I just realized that this is still the same warning I reported some month ago. I thought that this had been fixed with 25d37af374263243214be9d912cbb46a8e469bc7 which is included in the kernel I'm using. So I think there must be another Problem. Regards, Christian 2011/10/9 Christian Brunner c

Re: Btrfs slowdown

2011-08-09 Thread Christian Brunner
Christian 2011/8/8 Sage Weil s...@newdream.net: Hi Christian, Are you still seeing this slowness? sage On Wed, 27 Jul 2011, Christian Brunner wrote: 2011/7/25 Chris Mason chris.ma...@oracle.com: Excerpts from Christian Brunner's message of 2011-07-25 03:54:47 -0400: Hi, we are running

Re: Btrfs slowdown

2011-07-28 Thread Christian Brunner
2011/7/28 Marcus Sorensen shadow...@gmail.com: Christian, Have you checked up on the disks themselves and hardware? High utilization can mean that the i/o load has increased, but it can also mean that the i/o capacity has decreased.  Your traces seem to indicate that a good portion of the

Btrfs slowdown

2011-07-25 Thread Christian Brunner
Hi, we are running a ceph cluster with btrfs as it's base filesystem (kernel 3.0). At the beginning everything worked very well, but after a few days (2-3) things are getting very slow. When I look at the object store servers I see heavy disk-i/o on the btrfs filesystems (disk utilization is

Re: [PATCH] Btrfs: don't be as agressive with delalloc metadata reservations V2

2011-07-21 Thread Christian Brunner
2011/7/18 Josef Bacik jo...@redhat.com: On 07/18/2011 02:11 PM, Josef Bacik wrote: Currently we reserve enough space to COW an entirely full btree for every extent we have reserved for an inode.  This _sucks_, because you only need to COW once, and then everybody else is ok.  Unfortunately

WARNING: at fs/btrfs/inode.c:2204

2011-07-21 Thread Christian Brunner
I'm running a Ceph Object Store with 3.0-rc7 and patches from Josef. Occasionally I get the attached warning. Everything seems to be working after this warning, but I am concerned... Thanks, Christian [13319.808020] [ cut here ] [13319.813284] WARNING: at

Re: Delayed inode operations not doing the right thing with enospc

2011-07-14 Thread Christian Brunner
2011/7/13 Josef Bacik jo...@redhat.com: On 07/12/2011 11:20 AM, Christian Brunner wrote: 2011/6/7 Josef Bacik jo...@redhat.com: On 06/06/2011 09:39 PM, Miao Xie wrote: On fri, 03 Jun 2011 14:46:10 -0400, Josef Bacik wrote: I got a lot of these when running stress.sh on my test box

Re: Delayed inode operations not doing the right thing with enospc

2011-07-12 Thread Christian Brunner
2011/6/7 Josef Bacik jo...@redhat.com: On 06/06/2011 09:39 PM, Miao Xie wrote: On fri, 03 Jun 2011 14:46:10 -0400, Josef Bacik wrote: I got a lot of these when running stress.sh on my test box This is because use_block_rsv() is having to do a reserve_metadata_bytes(), which shouldn't

kernel BUG at fs/btrfs/extent-tree.c:5637!

2011-05-19 Thread Christian Brunner
Hi, we are running a ceph cluster with a btrfs store. Last night we ran across this btrfs BUG. Any hints on how to solve this are welcome. Regards Christian May 19 06:10:07 os00 kernel: [247212.342712] [ cut here ] May 19 06:10:07 os00 kernel: [247212.347953] kernel BUG

Re: [PATCH] Prevent oopsing in posix_acl_valid()

2011-05-03 Thread Christian Brunner
2011/5/3 Josef Bacik jo...@redhat.com: On 05/03/2011 12:44 PM, Daniel J Blueman wrote: If posix_acl_from_xattr() returns an error code, a negative address is dereferenced causing an oops; fix by checking for error code first. Signed-off-by: Daniel J Bluemandaniel.blue...@gmail.com ---