Re: [PATCH 1/3] fs: remove FS_COW_FL

2011-05-03 Thread Li Zefan
Any comments? If this patch is acceptable, it should be queued for .39?
since fs.h is exported to userspace.

Li Zefan wrote:
 FS_COW_FL and FS_NOCOW_FL were newly introduced to control per file
 COW in btrfs, but FS_NOCOW_FL is sufficient.
 
 The fact is we don't have corresponding BTRFS_INODE_COW flag.
 
 COW is default, and FS_NOCOW_FL can be used to switch off COW for
 a single file.
 
 If we mount btrfs with nodatacow, a newly created file will be set with
 the FS_NOCOW_FL flag. So to turn on COW for it, we can just clear the
 FS_NOCOW_FL flag.
 
 Signed-off-by: Li Zefan l...@cn.fujitsu.com
 ---
  fs/btrfs/ioctl.c   |   15 ++-
  include/linux/fs.h |1 -
  2 files changed, 6 insertions(+), 10 deletions(-)
 
 diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c
 index f580a3a..3240dd9 100644
 --- a/fs/btrfs/ioctl.c
 +++ b/fs/btrfs/ioctl.c
 @@ -144,16 +144,13 @@ static int check_flags(unsigned int flags)
   if (flags  ~(FS_IMMUTABLE_FL | FS_APPEND_FL | \
 FS_NOATIME_FL | FS_NODUMP_FL | \
 FS_SYNC_FL | FS_DIRSYNC_FL | \
 -   FS_NOCOMP_FL | FS_COMPR_FL | \
 -   FS_NOCOW_FL | FS_COW_FL))
 +   FS_NOCOMP_FL | FS_COMPR_FL |
 +   FS_NOCOW_FL))
   return -EOPNOTSUPP;
  
   if ((flags  FS_NOCOMP_FL)  (flags  FS_COMPR_FL))
   return -EINVAL;
  
 - if ((flags  FS_NOCOW_FL)  (flags  FS_COW_FL))
 - return -EINVAL;
 -
   return 0;
  }
  
 @@ -218,6 +215,10 @@ static int btrfs_ioctl_setflags(struct file *file, void 
 __user *arg)
   ip-flags |= BTRFS_INODE_DIRSYNC;
   else
   ip-flags = ~BTRFS_INODE_DIRSYNC;
 + if (flags  FS_NOCOW_FL)
 + ip-flags |= BTRFS_INODE_NODATACOW;
 + else
 + ip-flags = ~BTRFS_INODE_NODATACOW;
  
   /*
* The COMPRESS flag can only be changed by users, while the NOCOMPRESS
 @@ -231,10 +232,6 @@ static int btrfs_ioctl_setflags(struct file *file, void 
 __user *arg)
   ip-flags |= BTRFS_INODE_COMPRESS;
   ip-flags = ~BTRFS_INODE_NOCOMPRESS;
   }
 - if (flags  FS_NOCOW_FL)
 - ip-flags |= BTRFS_INODE_NODATACOW;
 - else if (flags  FS_COW_FL)
 - ip-flags = ~BTRFS_INODE_NODATACOW;
  
   trans = btrfs_join_transaction(root, 1);
   BUG_ON(IS_ERR(trans));
 diff --git a/include/linux/fs.h b/include/linux/fs.h
 index de9dd81..56a4141 100644
 --- a/include/linux/fs.h
 +++ b/include/linux/fs.h
 @@ -365,7 +365,6 @@ struct inodes_stat_t {
  #define FS_EXTENT_FL 0x0008 /* Extents */
  #define FS_DIRECTIO_FL   0x0010 /* Use direct i/o */
  #define FS_NOCOW_FL  0x0080 /* Do not cow file */
 -#define FS_COW_FL0x0200 /* Cow file */
  #define FS_RESERVED_FL   0x8000 /* reserved for ext2 
 lib */
  
  #define FS_FL_USER_VISIBLE   0x0003DFFF /* User visible flags */
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: abysmal performance

2011-05-03 Thread Bernhard Schmidt
Peter Stuge pe...@stuge.se wrote:

Hey,

defragging btrfs does not seem to work for me. I have run the filefrag
command over the whole fs and (manually) tried to defrag a few heavily
fragmented files, but I don't get it to work (it still has the same
number of extends and they are horrently uncorrelated)

root@schleppi:~# filefrag
/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1: 72 extents found
root@schleppi:~# btrfs filesystem defrag
/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
root@schleppi:~# filefrag
/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1: 72 extents found

I'm using Ubuntu Natty (2.6.38.4) and tried both btrfs-tools from Natty
(201006xx) and from Debian experimental (git from 20101101). Both show
the same symptoms. I don't think fragmentation is bad on this box (due
to having an SSD), but my system at home is getting dog slow and I'd
like to try that when I come home end of the week.

Best Regards,
Bernhard

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: abysmal performance

2011-05-03 Thread cwillu
On Tue, May 3, 2011 at 4:33 AM, Bernhard Schmidt be...@birkenwald.de wrote:
 Peter Stuge pe...@stuge.se wrote:

 Hey,

 defragging btrfs does not seem to work for me. I have run the filefrag
 command over the whole fs and (manually) tried to defrag a few heavily
 fragmented files, but I don't get it to work (it still has the same
 number of extends and they are horrently uncorrelated)

 root@schleppi:~# filefrag
 /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
 /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1: 72 extents found
 root@schleppi:~# btrfs filesystem defrag
 /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
 root@schleppi:~# filefrag
 /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
 /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1: 72 extents found

 I'm using Ubuntu Natty (2.6.38.4) and tried both btrfs-tools from Natty
 (201006xx) and from Debian experimental (git from 20101101). Both show
 the same symptoms. I don't think fragmentation is bad on this box (due
 to having an SSD), but my system at home is getting dog slow and I'd
 like to try that when I come home end of the week.

You're not using compression on that filesystem are you?  If so, be
aware that the number of extents isn't going to change after
defragmentation, although you should find that the _locations_ of
those extents is contiguous.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: abysmal performance

2011-05-03 Thread Chris Mason
Excerpts from John Wyzer's message of 2011-04-30 18:33:20 -0400:
 Excerpts from Mitch Harder's message of Sun May 01 00:16:53 +0200 2011:
   Hmm.
   Tried it and it gives me about 50 lines of
  
   FIBMAP: Invalid argument
  
   and then:
  
   large_file: 1 extent found
  
   Is that the way it is supposed to work?
   Just asking because this was part of a vmware disk image. Both the virtual
   machine and the rest of the host system are almost unusable once the VM 
   ist
   started (even more unusable than without vmware :-D )
  
  No.  It sounds like the filefrag command is getting confused in the
  virtual environment.
 
 Misunderstanding :-) I tried filefrag _on_ a vmware disk image, not  inside a
 virtual machine.
 The whole btrfs story here is on a real machine.

The most important files to defrag are going to be your internal firefox
files (I think in .mozilla), your sup database (in .sup) and your vmware
images.  I would definitely suggest that you try to narrow down if
vmware is making the machine seem much slower after the defrag is done.

It might make sense to run the nocow ioctl on your vmware images, they
are probably triggering lots of seeks.

You'll notice the machine is much slower after a reboot, this is because
we have to do a lot of IO to recache the extent allocation tree.  If you
pull from the btrfs-unstable tree, you can use mount -o space_cache,
which cuts down on the reading after a reboot dramatically.

If none of this works, we'll look at the files that you're fsyncing.
That seems to be the bulk of your latencies.

-chris
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: abysmal performance

2011-05-03 Thread Chris Mason
Excerpts from John Wyzer's message of 2011-04-30 18:33:20 -0400:
 Excerpts from Mitch Harder's message of Sun May 01 00:16:53 +0200 2011:
   Hmm.
   Tried it and it gives me about 50 lines of
  
   FIBMAP: Invalid argument
  
   and then:
  
   large_file: 1 extent found
  
   Is that the way it is supposed to work?
   Just asking because this was part of a vmware disk image. Both the virtual
   machine and the rest of the host system are almost unusable once the VM 
   ist
   started (even more unusable than without vmware :-D )
  
  No.  It sounds like the filefrag command is getting confused in the
  virtual environment.
 
 Misunderstanding :-) I tried filefrag _on_ a vmware disk image, not  inside a
 virtual machine.
 The whole btrfs story here is on a real machine.

Older filefrag uses fibmap, which we don't support (we use fiemap instead).
If you update your e2fsprogs you should get a newer filefrag.

-chris
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: abysmal performance

2011-05-03 Thread Chris Mason
Excerpts from Bernhard Schmidt's message of 2011-05-03 06:33:25 -0400:
 Peter Stuge pe...@stuge.se wrote:
 
 Hey,
 
 defragging btrfs does not seem to work for me. I have run the filefrag
 command over the whole fs and (manually) tried to defrag a few heavily
 fragmented files, but I don't get it to work (it still has the same
 number of extends and they are horrently uncorrelated)
 
 root@schleppi:~# filefrag
 /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
 /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1: 72 extents found
 root@schleppi:~# btrfs filesystem defrag
 /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
 root@schleppi:~# filefrag
 /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
 /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1: 72 extents found
 
 I'm using Ubuntu Natty (2.6.38.4) and tried both btrfs-tools from Natty
 (201006xx) and from Debian experimental (git from 20101101). Both show
 the same symptoms. I don't think fragmentation is bad on this box (due
 to having an SSD), but my system at home is getting dog slow and I'd
 like to try that when I come home end of the week.

Do you have compression on?

The file the defrag ioctl works is that it schedules things for defrag
but doesn't force out the IO immediately unless you use -f.

So, to test the result of the defrag, you need to either wait a bit or
run sync.

-chris
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 1/3] fs: remove FS_COW_FL

2011-05-03 Thread Chris Mason
Excerpts from Li Zefan's message of 2011-05-03 05:11:44 -0400:
 Any comments? If this patch is acceptable, it should be queued for .39?
 since fs.h is exported to userspace.

 Li Zefan wrote:
  FS_COW_FL and FS_NOCOW_FL were newly introduced to control per file
  COW in btrfs, but FS_NOCOW_FL is sufficient.
  

Good point, I'll send this in.

-chris
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: abysmal performance

2011-05-03 Thread Bernhard Schmidt
Am 03.05.2011 13:08, schrieb Chris Mason:

 defragging btrfs does not seem to work for me. I have run the filefrag
 command over the whole fs and (manually) tried to defrag a few heavily
 fragmented files, but I don't get it to work (it still has the same
 number of extends and they are horrently uncorrelated)

 root@schleppi:~# filefrag
 /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
 /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1: 72 extents found
 root@schleppi:~# btrfs filesystem defrag
 /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
 root@schleppi:~# filefrag
 /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
 /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1: 72 extents found

 I'm using Ubuntu Natty (2.6.38.4) and tried both btrfs-tools from Natty
 (201006xx) and from Debian experimental (git from 20101101). Both show
 the same symptoms. I don't think fragmentation is bad on this box (due
 to having an SSD), but my system at home is getting dog slow and I'd
 like to try that when I come home end of the week.
 
 Do you have compression on?

Yes. lzo to be exact.

 The file the defrag ioctl works is that it schedules things for defrag
 but doesn't force out the IO immediately unless you use -f.
 
 So, to test the result of the defrag, you need to either wait a bit or
 run sync.

Did so, no change. See my reply to cwillu for the data.

I usually mount my / without any compression option and did mount -o
remount,compress=lzo / before. I cannot reboot at the moment and I did
not find any option to disable compression again. There does not seem to
be a nocompress or compress=[none|off] option. Is this correct?

Bernhard
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: abysmal performance

2011-05-03 Thread Chris Mason
Excerpts from Bernhard Schmidt's message of 2011-05-03 07:30:36 -0400:
 Am 03.05.2011 13:08, schrieb Chris Mason:
 
  defragging btrfs does not seem to work for me. I have run the filefrag
  command over the whole fs and (manually) tried to defrag a few heavily
  fragmented files, but I don't get it to work (it still has the same
  number of extends and they are horrently uncorrelated)
 
  root@schleppi:~# filefrag
  /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
  /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1: 72 extents found
  root@schleppi:~# btrfs filesystem defrag
  /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
  root@schleppi:~# filefrag
  /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
  /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1: 72 extents found
 
  I'm using Ubuntu Natty (2.6.38.4) and tried both btrfs-tools from Natty
  (201006xx) and from Debian experimental (git from 20101101). Both show
  the same symptoms. I don't think fragmentation is bad on this box (due
  to having an SSD), but my system at home is getting dog slow and I'd
  like to try that when I come home end of the week.
  
  Do you have compression on?
 
 Yes. lzo to be exact.
 
  The file the defrag ioctl works is that it schedules things for defrag
  but doesn't force out the IO immediately unless you use -f.
  
  So, to test the result of the defrag, you need to either wait a bit or
  run sync.
 
 Did so, no change. See my reply to cwillu for the data.
 
 I usually mount my / without any compression option and did mount -o
 remount,compress=lzo / before. I cannot reboot at the moment and I did
 not find any option to disable compression again. There does not seem to
 be a nocompress or compress=[none|off] option. Is this correct?

Using compression is not a problem, but in order to reduce the maximum
amount of ram we need to uncompress an extent, we enforce a max size on
the extent.  So you'll tend to have more extents, but they should be
close together on disk.

Could you please do a filefrag -v on the file?  Lets see how bad it
really is.

-chris
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: abysmal performance

2011-05-03 Thread Bernhard Schmidt
Am 03.05.2011 13:00, schrieb cwillu:

Hi,

 defragging btrfs does not seem to work for me. I have run the filefrag
 command over the whole fs and (manually) tried to defrag a few heavily
 fragmented files, but I don't get it to work (it still has the same
 number of extends and they are horrently uncorrelated)

 root@schleppi:~# filefrag
 /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
 /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1: 72 extents found
 root@schleppi:~# btrfs filesystem defrag
 /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
 root@schleppi:~# filefrag
 /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
 /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1: 72 extents found

 I'm using Ubuntu Natty (2.6.38.4) and tried both btrfs-tools from Natty
 (201006xx) and from Debian experimental (git from 20101101). Both show
 the same symptoms. I don't think fragmentation is bad on this box (due
 to having an SSD), but my system at home is getting dog slow and I'd
 like to try that when I come home end of the week.
 
 You're not using compression on that filesystem are you?  If so, be
 aware that the number of extents isn't going to change after
 defragmentation, although you should find that the _locations_ of
 those extents is contiguous.

Actually I do run compression, but the location is not continguous
afterwards (even after btrfs filesystem sync / or using -f):

root@schleppi:~# btrfs filesystem defrag -f
/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
root@schleppi:~# btrfs filesystem sync /FSSync '/'
root@schleppi:~# filefrag -v
/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
Filesystem type is: 9123683e
File size of /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1 is
9361528 (2286 blocks, blocksize 4096)
 ext logical physical expected length flags
   0   0  4542111  32
   1  32  4542134  4542142 32
   2  64  4573263  4542165 32
   3  96  4573285  4573294 32
   4 128  4579639  4573316 32
   5 160  4579664  4579670 32
   6 192  4581178  4579695 32
   7 224  4579811  4581209 32
   8 256  4579836  4579842 32
   9 288  4579861  4579867 32
  10 320  4579884  4579892 32
  11 352  4580698  4579915 32
  12 384  4580720  4580729 32
  13 416  4580746  4580751 32
  14 448  4580768  4580777 32
  15 480  4580793  4580799 32
  16 512  4580819  4580824 32
  17 544  4581238  4580850 32
  18 576  4600396  4581269 32
  19 608  4600422  4600427 32
  20 640  4600447  4600453 32
  21 672  4600472  4600478 32
  22 704  4600498  4600503 32
  23 736  4600523  4600529 32
  24 768  4601483  4600554 32
  25 800  4601509  4601514 32
  26 832  4601534  4601540 32
  27 864  4601558  4601565 32
  28 896  4601583  4601589 32
  29 928  4601608  4601614 32
  30 960  4618420  4601639 32
  31 992  4618443  4618451 32
  321024  4541221  4618474 32
  331056  4618463  4541252 32
  341088  4618485  4618494 32
  351120  4618505  4618516 32
  361152  4579536  4618536 32
  371184  4579688  4579567 32
  381216  4579740  4579719 32
  391248  4618526  4579771 32
  401280  4618544  4618557 32
  411312  4618563  4618575 32
  421344  4618583  4618594 32
  431376  4618605  4618614 32
  441408  4618626  4618636 32
  451440  4618652  4618657 32
  461472  4618677  4618683 32
  471504  4618703  4618708 32
  481536  4618728  4618734 32
  491568  4618754  4618759 32
  501600  4618774  4618785 32
  511632  4618782  4618805 32
  521664  4561195  4618813 32
  531696  4600548  4561226 32
  541728  4618793  4600579 32
  551760  4618807  4618824 32
  561792  4539912  4618838 32
  571824  4542619  4539943 32
  581856  4556887  4542650 32
  591888  4601632  4556918 32
  601920  4558150  4601663 32
  611952  4561224  4558181 32
  621984  4618816  4561255 32
  632016  4618835  4618847 32
  642048  4618861  4618866 32
  652080  4618881  4618892 32
  662112  4618901  4618912 32
  672144  4618917  4618932 32
  682176  4539241  4618948 32
  692208  4539915  4539272 32
  702240  4539985  4539946 32
  712272  4540096  4540016 14 eof
/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1: 72 extents found

Best Regards,
Bernhard

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: abysmal performance

2011-05-03 Thread Bernhard Schmidt
Hi,

 Using compression is not a problem, but in order to reduce the maximum
 amount of ram we need to uncompress an extent, we enforce a max size on
 the extent.  So you'll tend to have more extents, but they should be
 close together on disk.
 
 Could you please do a filefrag -v on the file?  Lets see how bad it
 really is.


root@schleppi:~# filefrag -v
/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
Filesystem type is: 9123683e
File size of /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1 is
9361528 (2286 blocks, blocksize 4096)
 ext logical physical expected length flags
   0   0  4542111  32
   1  32  4542134  4542142 32
   2  64  4573263  4542165 32
   3  96  4573285  4573294 32
   4 128  4579639  4573316 32
   5 160  4579664  4579670 32
   6 192  4581178  4579695 32
   7 224  4579811  4581209 32
   8 256  4579836  4579842 32
   9 288  4579861  4579867 32
  10 320  4579884  4579892 32
  11 352  4580698  4579915 32
  12 384  4580720  4580729 32
  13 416  4580746  4580751 32
  14 448  4580768  4580777 32
  15 480  4580793  4580799 32
  16 512  4580819  4580824 32
  17 544  4581238  4580850 32
  18 576  4600396  4581269 32
  19 608  4600422  4600427 32
  20 640  4600447  4600453 32
  21 672  4600472  4600478 32
  22 704  4600498  4600503 32
  23 736  4600523  4600529 32
  24 768  4601483  4600554 32
  25 800  4601509  4601514 32
  26 832  4601534  4601540 32
  27 864  4601558  4601565 32
  28 896  4601583  4601589 32
  29 928  4601608  4601614 32
  30 960  4618420  4601639 32
  31 992  4618443  4618451 32
  321024  4541221  4618474 32
  331056  4618463  4541252 32
  341088  4618485  4618494 32
  351120  4618505  4618516 32
  361152  4579536  4618536 32
  371184  4579688  4579567 32
  381216  4579740  4579719 32
  391248  4618526  4579771 32
  401280  4618544  4618557 32
  411312  4618563  4618575 32
  421344  4618583  4618594 32
  431376  4618605  4618614 32
  441408  4618626  4618636 32
  451440  4618652  4618657 32
  461472  4618677  4618683 32
  471504  4618703  4618708 32
  481536  4618728  4618734 32
  491568  4618754  4618759 32
  501600  4618774  4618785 32
  511632  4618782  4618805 32
  521664  4561195  4618813 32
  531696  4600548  4561226 32
  541728  4618793  4600579 32
  551760  4618807  4618824 32
  561792  4539912  4618838 32
  571824  4542619  4539943 32
  581856  4556887  4542650 32
  591888  4601632  4556918 32
  601920  4558150  4601663 32
  611952  4561224  4558181 32
  621984  4618816  4561255 32
  632016  4618835  4618847 32
  642048  4618861  4618866 32
  652080  4618881  4618892 32
  662112  4618901  4618912 32
  672144  4618917  4618932 32
  682176  4539241  4618948 32
  692208  4539915  4539272 32
  702240  4539985  4539946 32
  712272  4540096  4540016 14 eof
/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1: 72 extents found

Bernhard
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: abysmal performance

2011-05-03 Thread Chris Mason
Excerpts from Bernhard Schmidt's message of 2011-05-03 07:43:04 -0400:
 Hi,
 
  Using compression is not a problem, but in order to reduce the maximum
  amount of ram we need to uncompress an extent, we enforce a max size on
  the extent.  So you'll tend to have more extents, but they should be
  close together on disk.
  
  Could you please do a filefrag -v on the file?  Lets see how bad it
  really is.
 
 
 root@schleppi:~# filefrag -v
 /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1
 Filesystem type is: 9123683e
 File size of /usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.4/cc1 is
 9361528 (2286 blocks, blocksize 4096)
  ext logical physical expected length flags
0   0  4542111  32
1  32  4542134  4542142 32
2  64  4573263  4542165 32

Ok, looks like we could be doing a little better job when compression is
on to build out a bigger extent.  This shouldn't be causing trouble on
an ssd at all but on your rotating disk it'll be slightly slower.

Still most of these extents are somewhat close together, this is roughly
what mount -o ssd (which is enabled automatically when we detect a
non-rotating drive) would try for.

The problematic files are going to have thousands of extents, this file
should be fine.

-chris
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: abysmal performance

2011-05-03 Thread Bernhard Schmidt
Hi,

 Ok, looks like we could be doing a little better job when compression is
 on to build out a bigger extent.  This shouldn't be causing trouble on
 an ssd at all but on your rotating disk it'll be slightly slower.
 
 Still most of these extents are somewhat close together, this is roughly
 what mount -o ssd (which is enabled automatically when we detect a
 non-rotating drive) would try for.
 
 The problematic files are going to have thousands of extents, this file
 should be fine.

Thanks, I'll check on my system with rotating disks at home when I get back.

Bernhard
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: resize ate my root node

2011-05-03 Thread Peter Stuge
Peter Stuge wrote:
 Sorry for not following up on this until now. :( I've been busy and
 have been using a backup. But I'm still very interested in restoring
 the btrfs and finding this bug! Let me know if I should refresh any
 details.

Ping?

I've created a small btrfs to see if I can learn a little about the
structure (wiki next to the terminal) but not really expect that I'll
be efficient. Any hints on how/where I can find my root node again?

Or if unidentifiable, could it be reconstructed?


//Peter
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: abysmal performance

2011-05-03 Thread Mitch Harder
On Tue, May 3, 2011 at 8:03 AM, Bernhard Schmidt be...@birkenwald.de wrote:
 Hi,

 Ok, looks like we could be doing a little better job when compression is
 on to build out a bigger extent.  This shouldn't be causing trouble on
 an ssd at all but on your rotating disk it'll be slightly slower.

 Still most of these extents are somewhat close together, this is roughly
 what mount -o ssd (which is enabled automatically when we detect a
 non-rotating drive) would try for.

 The problematic files are going to have thousands of extents, this file
 should be fine.

 Thanks, I'll check on my system with rotating disks at home when I get back.


I'd also be curious to see if mounting with -o compress-force=lzo
affects anything.

As I recall, the compress-force option was added because performance
could be affected negatively when trying to optimize compression with
-o compress.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Is it possible for the ext4/btrfs file system to pass some context related info to low level block driver?

2011-05-03 Thread Martin K. Petersen
 Yunpeng == Gao, Yunpeng yunpeng@intel.com writes:

Yunpeng So, my question is, is there any plan or discussion on
Yunpeng supporting this feature (passing data type info to low level
Yunpeng block device driver) on file system developments? Especially
Yunpeng for ext4/btrfs, since now they are very hot in Linux? Thanks.

Yes, I have been working on some changes that allow us to tag bios and
pass the information out to storage. These patches have been on the back
burner for a while due to other commitments. But I'll dig them out and
post them later. We just discussed them a couple of weeks ago at the
Linux Storage Workshop.

In the meantime: Can you point me to the relevant eMMC stuff so I can
see how many tiers or classes we have to work with there?

-- 
Martin K. Petersen  Oracle Linux Engineering
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: abysmal performance

2011-05-03 Thread Daniel J Blueman
On 3 May 2011 19:30, Bernhard Schmidt be...@birkenwald.de wrote:
[]
 The file the defrag ioctl works is that it schedules things for defrag
 but doesn't force out the IO immediately unless you use -f.

 So, to test the result of the defrag, you need to either wait a bit or
 run sync.

 Did so, no change. See my reply to cwillu for the data.

Can you try with the compression option enabled? Eg:

# filefrag foo.dat
foo.dat: 11 extents found

# find . -xdev -type f -print0 | xargs -0 btrfs filesystem defragment -c

# filefrag foo.dat
foo.dat: 1 extent found

Seems to work fine on 2.6.39-rc5; I mounted with '-o
compress,clear_cache' though.

Daniel
-- 
Daniel J Blueman
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] Btrfs: fix how we do space reservation for truncate

2011-05-03 Thread Josef Bacik
The ceph guys keep running into problems where we have space reserved in our
orphan block rsv when freeing it up.  This is because they tend to do snapshots
alot, so their truncates tend to use a bunch of space, so when we go to do
things like update the inode we have to steal reservation space in order to make
the reservation happen.  This happens because truncate can use as much space as
it freaking feels like, but we still have to hold space for removing the orphan
item and updating the inode, which will definitely always happen.  So in order
to fix this we need to split all of the reservation stuf up.  So with this patch
we have

1) The orphan block reserve which only holds the space for deleting our orphan
item when everything is over.

2) The truncate block reserve which gets allocated and used specifically for the
space that the truncate will use on a per truncate basis.

3) The transaction will always have 1 item's worth of data reserved so we can
update the inode normally.

Hopefully this will make the ceph problem go away.  Thanks,

Signed-off-by: Josef Bacik jo...@redhat.com
---
 fs/btrfs/ctree.h   |3 +
 fs/btrfs/extent-tree.c |   46 +++
 fs/btrfs/inode.c   |  111 
 3 files changed, 123 insertions(+), 37 deletions(-)

diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index d2991c8..1a41d77 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -2223,6 +2223,9 @@ int btrfs_block_rsv_migrate(struct btrfs_block_rsv 
*src_rsv,
 void btrfs_block_rsv_release(struct btrfs_root *root,
 struct btrfs_block_rsv *block_rsv,
 u64 num_bytes);
+int btrfs_truncate_reserve_metadata(struct btrfs_trans_handle *trans,
+   struct btrfs_root *root,
+   struct btrfs_block_rsv *rsv);
 int btrfs_set_block_group_ro(struct btrfs_root *root,
 struct btrfs_block_group_cache *cache);
 int btrfs_set_block_group_rw(struct btrfs_root *root,
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index a33d9a3..64d2514 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -3934,6 +3934,37 @@ static u64 calc_trans_metadata_size(struct btrfs_root 
*root, int num_items)
3 * num_items;
 }
 
+int btrfs_truncate_reserve_metadata(struct btrfs_trans_handle *trans,
+   struct btrfs_root *root,
+   struct btrfs_block_rsv *rsv)
+{
+   struct btrfs_block_rsv *trans_rsv = root-fs_info-trans_block_rsv;
+   u64 num_bytes;
+   int ret;
+
+   /*
+* Truncate should be freeing data, but give us 2 items just in case it
+* needs to use some space.  We may want to be smarter about this in the
+* future.
+*/
+   num_bytes = calc_trans_metadata_size(root, 2);
+
+   /* We already have enough bytes, just return */
+   if (rsv-reserved = num_bytes)
+   return 0;
+
+   num_bytes -= rsv-reserved;
+
+   /*
+* You should have reserved enough space before hand to do this, so this
+* should not fail.
+*/
+   ret = block_rsv_migrate_bytes(trans_rsv, rsv, num_bytes);
+   BUG_ON(ret);
+
+   return 0;
+}
+
 int btrfs_trans_reserve_metadata(struct btrfs_trans_handle *trans,
 struct btrfs_root *root,
 int num_items)
@@ -3974,23 +4005,18 @@ int btrfs_orphan_reserve_metadata(struct 
btrfs_trans_handle *trans,
struct btrfs_block_rsv *dst_rsv = root-orphan_block_rsv;
 
/*
-* one for deleting orphan item, one for updating inode and
-* two for calling btrfs_truncate_inode_items.
-*
-* btrfs_truncate_inode_items is a delete operation, it frees
-* more space than it uses in most cases. So two units of
-* metadata space should be enough for calling it many times.
-* If all of the metadata space is used, we can commit
-* transaction and use space it freed.
+* We need to hold space in order to delete our orphan item once we've
+* added it, so this takes the reservation so we can release it later
+* when we are truly done with the orphan item.
 */
-   u64 num_bytes = calc_trans_metadata_size(root, 4);
+   u64 num_bytes = calc_trans_metadata_size(root, 1);
return block_rsv_migrate_bytes(src_rsv, dst_rsv, num_bytes);
 }
 
 void btrfs_orphan_release_metadata(struct inode *inode)
 {
struct btrfs_root *root = BTRFS_I(inode)-root;
-   u64 num_bytes = calc_trans_metadata_size(root, 4);
+   u64 num_bytes = calc_trans_metadata_size(root, 1);
btrfs_block_rsv_release(root, root-orphan_block_rsv, num_bytes);
 }
 
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 3217919..1aa8cc0 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -6591,6 

Re: btrfs warnings from 2.6.39-rc5

2011-05-03 Thread Josef Bacik

On 04/27/2011 02:43 PM, Jim Schutt wrote:

Hi,

I'm not sure if they matter, but I got these warnings on
one of the machines I'm using as a Ceph OSD server:

[ 1806.549469] [ cut here ]
[ 1806.554593] WARNING: at fs/btrfs/extent-tree.c:5790
use_block_rsv+0xa7/0x101 [btrfs]()
[ 1806.562903] Hardware name: PowerEdge 1950
[ 1806.567126] Modules linked in: loop btrfs zlib_deflate lzo_compress
ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4
xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables
x_tables bridge stp i2c_dev i2c_core ext3 jbd ib_iser libi]
[ 1806.689084] Pid: 12025, comm: cosd Not tainted
2.6.39-rc5-6-g7b0bd4b #2
[ 1806.697425] Call Trace:
[ 1806.700332] [a06fe816] ? use_block_rsv+0xa7/0x101 [btrfs]
[ 1806.707032] [810494d8] ? warn_slowpath_common+0x85/0x9e
[ 1806.713502] [8104950b] ? warn_slowpath_null+0x1a/0x1c
[ 1806.720755] [a06fe816] ? use_block_rsv+0xa7/0x101 [btrfs]
[ 1806.731488] [a070590b] ? btrfs_alloc_free_block+0x30/0x198
[btrfs]
[ 1806.743858] [a072ae8a] ?
map_private_extent_buffer+0xb2/0xd9 [btrfs]
[ 1806.752600] [a072a4b8] ? __kmap_atomic+0x12/0x47 [btrfs]
[ 1806.760057] [a072b036] ? read_extent_buffer+0xc2/0xd4 [btrfs]
[ 1806.770897] [a06f84ac] ? __btrfs_cow_block+0x10e/0x2f5 [btrfs]
[ 1806.779552] [a0709dbe] ? btrfs_header_generation+0x1f/0x25
[btrfs]
[ 1806.788994] [a06f878f] ? btrfs_cow_block+0xfc/0x121 [btrfs]
[ 1806.798434] [a06fa746] ? btrfs_search_slot+0x144/0x3ae [btrfs]
[ 1806.806444] [a0709877] ? btrfs_lookup_inode+0x31/0x86 [btrfs]
[ 1806.813611] [a071618b] ? btrfs_update_inode+0x52/0xc1 [btrfs]
[ 1806.820904] [a071668c] ? btrfs_truncate+0x239/0x297 [btrfs]
[ 1806.829290] [a071bb23] ? btrfs_setsize+0x8c/0x9b [btrfs]
[ 1806.835962] [a071bb93] ? btrfs_setattr+0x61/0x9d [btrfs]
[ 1806.842892] [81127ea4] ? notify_change+0x174/0x1bc
[ 1806.850878] [81110c4f] ? do_truncate+0x6e/0x8a
[ 1806.857374] [8111b126] ? generic_permission+0x1c/0x8e
[ 1806.868498] [81110e42] ? do_sys_truncate+0xf8/0x10a
[ 1806.877746] [81110e62] ? sys_truncate+0xe/0x10
[ 1806.902751] [813b6deb] ? system_call_fastpath+0x16/0x1b
[ 1806.909736] ---[ end trace cd0ae33f1a4433d9 ]---
[ 1812.142333] [ cut here ]
[ 1812.146996] WARNING: at fs/btrfs/inode.c:2180
btrfs_orphan_commit_root+0x8c/0xab [btrfs]()
[ 1812.155275] Hardware name: PowerEdge 1950
[ 1812.159280] Modules linked in: loop btrfs zlib_deflate lzo_compress
ipt_MASQUERADE iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4
xt_state nf_conntrack ipt_REJECT xt_tcpudp iptable_filter ip_tables
x_tables bridge stp i2c_dev i2c_core ext3 jbd ib_iser libi]
[ 1812.240872] Pid: 7923, comm: kworker/3:2 Tainted: G W
2.6.39-rc5-6-g7b0bd4b #2
[ 1812.249216] Call Trace:
[ 1812.251712] [a0715048] ? btrfs_orphan_commit_root+0x8c/0xab
[btrfs]
[ 1812.258976] [810494d8] ? warn_slowpath_common+0x85/0x9e
[ 1812.265199] [8104950b] ? warn_slowpath_null+0x1a/0x1c
[ 1812.271420] [a0715048] ? btrfs_orphan_commit_root+0x8c/0xab
[btrfs]
[ 1812.278704] [a070f5e0] ? commit_fs_roots+0x95/0xfd [btrfs]
[ 1812.285220] [a07039b3] ? btrfs_run_delayed_refs+0x112/0x15e
[btrfs]
[ 1812.292459] [810367bf] ? need_resched+0x23/0x2d
[ 1812.297961] [810367d7] ? should_resched+0xe/0x2f
[ 1812.303604] [a07105e3] ?
btrfs_commit_transaction+0x349/0x5a3 [btrfs]
[ 1812.310999] [8106622c] ? list_del_init+0x21/0x21
[ 1812.316615] [a071085c] ? do_async_commit+0x1f/0x2c [btrfs]
[ 1812.323055] [8106076b] ? process_one_work+0x124/0x1e0
[ 1812.329083] [a071083d] ?
btrfs_commit_transaction+0x5a3/0x5a3 [btrfs]
[ 1812.336526] [81061dd8] ? workqueue_congested+0x1e/0x1e
[ 1812.342636] [81061e67] ? worker_thread+0x8f/0x124
[ 1812.348353] [81065d26] ? kthread+0x72/0x7a
[ 1812.353439] [813b7ed4] ? kernel_thread_helper+0x4/0x10
[ 1812.359555] [813b07ad] ? retint_restore_args+0xe/0xe
[ 1812.365484] [81065cb4] ? kthread_bind+0x64/0x64
[ 1812.370986] [813b7ed0] ? gs_change+0xb/0xb
[ 1812.376047] ---[ end trace cd0ae33f1a4433da ]---

Please let me know if I can do anything to help sort these out.



I just posted a patch for this problem, it's titled

Btrfs: fix how we do space reservation for truncate

Please apply it and test it and see if it makes this problem go away. 
Thanks,


Josef
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: kernel BUG at fs/btrfs/inode.c:2271!

2011-05-03 Thread Josef Bacik

On 04/27/2011 02:52 PM, Marco Neubauer wrote:

Hi,

this is happening mostly every night. I can't reproduce it right now.
vanilla kernel 2.6.38.4



Can you update to a newer kernel, this should be fixed there.  Thanks,

Josef
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: question about percentage utilization...

2011-05-03 Thread Josef Bacik

On 04/27/2011 02:52 PM, K. Richard Pixley wrote:

At what percentage utilization are folks typically seeing no space left
on device?

And what would constitute pathological values worth reporting and/or
offering up as debugging instances?



Well if you are having problems let us know what you are seeing and then 
we can tell you :).  Be sure to include


df
btrfs fi df /your/mount/point
btrfs-show

Thanks,

Josef
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: abysmal performance

2011-05-03 Thread Bernhard Schmidt
Am 03.05.2011 16:54, schrieb Daniel J Blueman:

Hi,

 The file the defrag ioctl works is that it schedules things for defrag
 but doesn't force out the IO immediately unless you use -f.

 So, to test the result of the defrag, you need to either wait a bit or
 run sync.

 Did so, no change. See my reply to cwillu for the data.
 
 Can you try with the compression option enabled? Eg:
 
 # filefrag foo.dat
 foo.dat: 11 extents found
 
 # find . -xdev -type f -print0 | xargs -0 btrfs filesystem defragment -c
 
 # filefrag foo.dat
 foo.dat: 1 extent found
 
 Seems to work fine on 2.6.39-rc5; I mounted with '-o
 compress,clear_cache' though.

Maybe I was expecting too much. I tried it on a file with 72 extends and
was expecting for the count to go down to 1 (or very very few). This
does not seem to happen with this particular file. I just tested another
file (with 193 extends) and it was reduced to 5. defrag with -f, but
without -c. Still mounted with compress=lzo.

However, the 72 frags file is not getting any better, no matter which
flags I tried. No big problem at the moment, I've found an older (Ubuntu
Maverick) based system with a rotating disk that had like 5 extends
for a single file. I expect defragging that will increase performance
quite nicely :-)

Bernhard
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: abysmal performance

2011-05-03 Thread Mitch Harder
On Tue, May 3, 2011 at 9:41 AM, Daniel J Blueman
daniel.blue...@gmail.com wrote:

 It does seem the case generally; on 2.6.39-rc5, writing to a fresh
 filesystem using rsync with BTRFS compression enabled, 128KB extents
 seem very common [1] (filefrag inconsistency noted).

 Defragmenting with compression gives a nice linear extent [2]. It
 looks like it'll be a good win to prevent extents being split at
 writeout for the read case on rotational media.


Yes, 128KB extents are hardcoded in Btrfs right now.

There are two reasons cited in the comments for this:

(1)  Ease the RAM required when spreading compression across several CPUs.
(2)  Make sure the amount of IO required to do a random read is
reasonably small.

For about 4 months, I've been playing locally with 2 patches to
increase the extent size to 512KB.

I haven't noticed any issues running with these patches.  However, I
only have a Core2duo with 2 CPUs, so I'm probably not running into
issues that someone with more CPUs might encounter.

I'll submit these patches to the list as an RFC so more people can at
least see where this is done.  But with my limited hardware, I can't
assert this change is the best for everyone.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] Prevent oopsing in posix_acl_valid()

2011-05-03 Thread Daniel J Blueman
If posix_acl_from_xattr() returns an error code, a negative address is
dereferenced causing an oops; fix by checking for error code first.

Signed-off-by: Daniel J Blueman daniel.blue...@gmail.com
---
 fs/btrfs/acl.c |5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/acl.c b/fs/btrfs/acl.c
index 5d505aa..cad6fbb 100644
--- a/fs/btrfs/acl.c
+++ b/fs/btrfs/acl.c
@@ -178,12 +178,13 @@ static int btrfs_xattr_acl_set(struct dentry *dentry, 
const char *name,
 
if (value) {
acl = posix_acl_from_xattr(value, size);
+   if (IS_ERR(acl)
+   return PTR_ERR(acl);
+
if (acl) {
ret = posix_acl_valid(acl);
if (ret)
goto out;
-   } else if (IS_ERR(acl)) {
-   return PTR_ERR(acl);
}
}
 
-- 
1.7.4.1

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: abysmal performance

2011-05-03 Thread Chris Mason
Excerpts from Mitch Harder's message of 2011-05-03 11:42:56 -0400:
 On Tue, May 3, 2011 at 9:41 AM, Daniel J Blueman
 daniel.blue...@gmail.com wrote:
 
  It does seem the case generally; on 2.6.39-rc5, writing to a fresh
  filesystem using rsync with BTRFS compression enabled, 128KB extents
  seem very common [1] (filefrag inconsistency noted).
 
  Defragmenting with compression gives a nice linear extent [2]. It
  looks like it'll be a good win to prevent extents being split at
  writeout for the read case on rotational media.
 
 
 Yes, 128KB extents are hardcoded in Btrfs right now.
 
 There are two reasons cited in the comments for this:
 
 (1)  Ease the RAM required when spreading compression across several CPUs.
 (2)  Make sure the amount of IO required to do a random read is
 reasonably small.
 
 For about 4 months, I've been playing locally with 2 patches to
 increase the extent size to 512KB.
 
 I haven't noticed any issues running with these patches.  However, I
 only have a Core2duo with 2 CPUs, so I'm probably not running into
 issues that someone with more CPUs might encounter.
 
 I'll submit these patches to the list as an RFC so more people can at
 least see where this is done.  But with my limited hardware, I can't
 assert this change is the best for everyone.

The problem is just that any random read into the file will require
reading the full 512KB in order to decompress the extent.  And you need
to make sure you have enough room in ram to represent the decompressed
bytes in order to find the pages you care about.

The alternative is to keep the smaller compressed extent size and make
the allocator work harder to find contiguous 128KB extents to store all
of the file bytes.  This will work out much better ;)

-chris
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Cannot Deinstall a Debian Package

2011-05-03 Thread CACook

Having a failure that may be because grub2 doesn't BTRFS.  /boot is ext3 and / 
is BTRFS.

# dpkg -r linux-image-2.6.32-5-amd64
(Reading database ... 136673 files and directories currently installed.)
Removing linux-image-2.6.32-5-amd64 ...
Examining /etc/kernel/postrm.d .
run-parts: executing /etc/kernel/postrm.d/initramfs-tools 2.6.32-5-amd64 
/boot/vmlinuz-2.6.32-5-amd64
run-parts: executing /etc/kernel/postrm.d/zz-update-grub 2.6.32-5-amd64 
/boot/vmlinuz-2.6.32-5-amd64
/usr/sbin/grub-probe: error: cannot find a device for / (is /dev mounted?).
run-parts: /etc/kernel/postrm.d/zz-update-grub exited with return code 1
Failed to process /etc/kernel/postrm.d at 
/var/lib/dpkg/info/linux-image-2.6.32-5-amd64.postrm line 234.
dpkg: error processing linux-image-2.6.32-5-amd64 (--remove):
 subprocess installed post-removal script returned error exit status 1
Errors were encountered while processing:
 linux-image-2.6.32-5-amd64

I can't install or remove packages, of do my dist-upgrades.  What gives?

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[RFC PATCH 0/2] Change Size of Compressed Extent

2011-05-03 Thread Mitch Harder

In recent thread on the list (see: abysmal performance), there
were some questions regarding why Btrfs seems to break up compressed files
into 32 block (128KB) chunks.

This is done for two reasons:
(1)  Ease the RAM required when spreading compression across several CPUs.
(2)  Make sure the amount of IO required to do a random read is
reasonably small.

The two attached patches show how to increase limit to 512KB (128 blocks).

I'm submitting these patches more for the purpose of documenting this issue
on the M/L.  I haven't fully explored the effect on performance.

As Chris Mason pointed out in the referenced thread, you would have to
decompress 512KB instead of just 128KB if you have a random read of
1KB in the middle of one of the chunks.

It should also be noted that just because filefrag reports the file as
fragmented, the extent fragments are often adjacent on the storage medium
(especially if you've just run defragment on that file).

It's possible that system performance could improve if the chunk size
was even smaller perhaps, instead of larger.

However, I have seen a decrease in the size of Metadata on my compressed
file systems after applying this patch, and defragmenting files with the
larger extent size.

Since I'd already done the experiment, and since someone was asking
about it, I thought I'd share my findings.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 2/2] [RFC] Btrfs: Increase limit of relocated extent size

2011-05-03 Thread Mitch Harder
The size of relocated compressed extents was limited to 128K.
This limit is put in place to ease the RAM required when spreading
compression across several CPUs, and to make sure the amount
of IO required to do a random read is reasonably small.

Increase this limit to 512K.
---
 fs/btrfs/relocation.c |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/fs/btrfs/relocation.c b/fs/btrfs/relocation.c
index 58250e0..dd17a35 100644
--- a/fs/btrfs/relocation.c
+++ b/fs/btrfs/relocation.c
@@ -143,7 +143,7 @@ struct tree_block {
unsigned int key_ready:1;
 };
 
-#define MAX_EXTENTS 128
+#define MAX_EXTENTS 512
 
 struct file_extent_cluster {
u64 start;
-- 
1.7.3.4

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 1/2] [RFC] Btrfs: Increase limit on size of compressed extent

2011-05-03 Thread Mitch Harder
The size of compressed extents was limited to 128K, which
leads to fragmentation of the extents (although the extents
themselves may still be located contiguously).  This limit is
put in place to ease the RAM required when spreading compression
across several CPUs, and to make sure the amount of IO required
to do a random read is reasonably small.

Increase this limit to 512K.
---
 fs/btrfs/inode.c |8 
 1 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 870869a..7ef2b34 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -333,8 +333,8 @@ static noinline int compress_file_range(struct inode *inode,
unsigned long nr_pages_ret = 0;
unsigned long total_compressed = 0;
unsigned long total_in = 0;
-   unsigned long max_compressed = 128 * 1024;
-   unsigned long max_uncompressed = 128 * 1024;
+   unsigned long max_compressed = 512 * 1024;
+   unsigned long max_uncompressed = 512 * 1024;
int i;
int will_compress;
int compress_type = root-fs_info-compress_type;
@@ -343,7 +343,7 @@ static noinline int compress_file_range(struct inode *inode,
 again:
will_compress = 0;
nr_pages = (end  PAGE_CACHE_SHIFT) - (start  PAGE_CACHE_SHIFT) + 1;
-   nr_pages = min(nr_pages, (128 * 1024UL) / PAGE_CACHE_SIZE);
+   nr_pages = min(nr_pages, (512 * 1024UL) / PAGE_CACHE_SIZE);
 
/*
 * we don't want to send crud past the end of i_size through
@@ -368,7 +368,7 @@ again:
 *
 * We also want to make sure the amount of IO required to do
 * a random read is reasonably small, so we limit the size of
-* a compressed extent to 128k.
+* a compressed extent to 512k (was 128k).
 */
total_compressed = min(total_compressed, max_uncompressed);
num_bytes = (end - start + blocksize)  ~(blocksize - 1);
-- 
1.7.3.4

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] Prevent oopsing in posix_acl_valid()

2011-05-03 Thread Christian Brunner
2011/5/3 Josef Bacik jo...@redhat.com:
 On 05/03/2011 12:44 PM, Daniel J Blueman wrote:

 If posix_acl_from_xattr() returns an error code, a negative address is
 dereferenced causing an oops; fix by checking for error code first.

 Signed-off-by: Daniel J Bluemandaniel.blue...@gmail.com
 ---
  fs/btrfs/acl.c |    5 +++--
  1 files changed, 3 insertions(+), 2 deletions(-)

 diff --git a/fs/btrfs/acl.c b/fs/btrfs/acl.c
 index 5d505aa..cad6fbb 100644
 --- a/fs/btrfs/acl.c
 +++ b/fs/btrfs/acl.c
 @@ -178,12 +178,13 @@ static int btrfs_xattr_acl_set(struct dentry
 *dentry, const char *name,

        if (value) {
                acl = posix_acl_from_xattr(value, size);
 +               if (IS_ERR(acl)

A small typo: The right parenthesis is missing.

Christian
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Cannot Deinstall a Debian Package

2011-05-03 Thread Fajar A. Nugraha
On Wed, May 4, 2011 at 2:27 AM,  cac...@quantum-sci.com wrote:

 Having a failure that may be because grub2 doesn't BTRFS.  /boot is ext3 and 
 / is BTRFS.

Does Debian (or whatever distro you use) support BTRFS /?
If yes, you should ask them.
If no, then you should've already known that there's a risk when using
unsupported filesystem.


 # dpkg -r linux-image-2.6.32-5-amd64
 (Reading database ... 136673 files and directories currently installed.)
 Removing linux-image-2.6.32-5-amd64 ...
 Examining /etc/kernel/postrm.d .
 run-parts: executing /etc/kernel/postrm.d/initramfs-tools 2.6.32-5-amd64 
 /boot/vmlinuz-2.6.32-5-amd64
 run-parts: executing /etc/kernel/postrm.d/zz-update-grub 2.6.32-5-amd64 
 /boot/vmlinuz-2.6.32-5-amd64
 /usr/sbin/grub-probe: error: cannot find a device for / (is /dev mounted?).
 run-parts: /etc/kernel/postrm.d/zz-update-grub exited with return code 1
 Failed to process /etc/kernel/postrm.d at 
 /var/lib/dpkg/info/linux-image-2.6.32-5-amd64.postrm line 234.
 dpkg: error processing linux-image-2.6.32-5-amd64 (--remove):
  subprocess installed post-removal script returned error exit status 1
 Errors were encountered while processing:
  linux-image-2.6.32-5-amd64

Looks like grub problem. I know that Ubuntu Natty's grub-pc (grub2)
work just fine, so you might be able to fix it by upgrading to newer
grub/grub-pc (perhaps from Debian-unstable).

-- 
Fajar
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


btrfs csum failed

2011-05-03 Thread Martin Schitter
since my last debian kernel-update to 2.6.38-2-amd64 i got troubles with 
csum failures. it's a volume full of huge kvm-images on md-RAID1 and 
LVM, so i used the mount options: 'noatime,nodatasum' to maximize the 
performance.


it happened two weeks ago for the fist time. and now again a kvm-image 
isn't readable again. i have to use an older snapshot to substitute the 
virtual machine.


this are the entries in dmesg/kernel-log on any access:
...
 [2412668.409442] btrfs csum failed ino 258 off 2331529216 csum 
3632892464 private 2115348581

...

it's a production machine, so i can not make to much experiments on it.
do you see an obvious way to solve this problem?

thanks!
martin
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Cannot Deinstall a Debian Package

2011-05-03 Thread CACook
On Tuesday 3 May, 2011 14:26:52 Fajar A. Nugraha wrote:
 Does Debian (or whatever distro you use) support BTRFS /?
 If yes, you should ask them.

What do you mean 'does Debian support BTRFS'?  The kernel supports it.  And why 
would they know more about BTRFS than you?

My whole system is installed over BTRFS.  If this is non-functional in any OS 
there should be a warning indicating it is non-functional.


 Looks like grub problem. I know that Ubuntu Natty's grub-pc (grub2)
 work just fine, so you might be able to fix it by upgrading to newer
 grub/grub-pc (perhaps from Debian-unstable).

I would be happy to upgrade grub, but the package management system is jammed 
because of this.

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Cannot Deinstall a Debian Package

2011-05-03 Thread Chuck Burns
On Tuesday, May 03, 2011 05:20:49 PM cac...@quantum-sci.com wrote:
-snip- 
 I would be happy to upgrade grub, but the package management system is
 jammed because of this.
-snip-

You should Re-read the error you posted.  The package management system goes 
bonkers, because

 /usr/sbin/grub-probe: error: cannot find a device for / (is /dev mounted?).
 run-parts: /etc/kernel/postrm.d/zz-update-grub exited with return code 1

-- 
Chuck Burns
The Southern Libertarian - owner
http://thesouthernlibertarian.com/
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Cannot Deinstall a Debian Package

2011-05-03 Thread CACook

I know what the error says;  we've established that / is in fact mounted.  The 
system boots and runs, but grub doesn't understand it.  My only answer is that 
grub-probe does not understand BTRFS.

The question is what to do about this.  I have three major systems committed to 
this filesystem.  Can you not see how this is an emergency?

How is re-reading the error supposed to help anything?  Or was your intention 
to help?


On Tuesday 3 May, 2011 16:02:43 Chuck Burns wrote:
 On Tuesday, May 03, 2011 05:20:49 PM cac...@quantum-sci.com wrote:
 -snip- 
  I would be happy to upgrade grub, but the package management system is
  jammed because of this.
 -snip-
 
 You should Re-read the error you posted.  The package management system goes 
 bonkers, because
 
  /usr/sbin/grub-probe: error: cannot find a device for / (is /dev mounted?).
  run-parts: /etc/kernel/postrm.d/zz-update-grub exited with return code 1
 
 
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Cannot Deinstall a Debian Package

2011-05-03 Thread Chuck Burns
On Tuesday, May 03, 2011 07:12:42 PM cac...@quantum-sci.com wrote:
 I know what the error says;  we've established that / is in fact mounted. 
 The system boots and runs, but grub doesn't understand it.  My only answer
 is that grub-probe does not understand BTRFS.
 
 The question is what to do about this.  I have three major systems
 committed to this filesystem.  Can you not see how this is an emergency?
 
 How is re-reading the error supposed to help anything?  Or was your
 intention to help?

You should upgrade grub, manually (ie, do not use the package manager) and you 
need to contact your distribution to find out how to do that.

-- 
Chuck Burns
The Southern Libertarian - owner
http://thesouthernlibertarian.com/
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: btrfs csum failed

2011-05-03 Thread Martin Schitter

Am 2011-05-04 02:28, schrieb Josef Bacik:

Wait why are you running with btrfs in production?


do you know a better alternative for continuous snapshots? :)

it works surprisingly well since more than a year.
well the performance could be better for vm-image-hosting but it works.

we used cache='writeback' for a long time but now all virtual instances 
have set cache='none'



What OS is in this vm image?


2.6.30-bpo.1-amd64 with virtio-driver

could you give me some advice how to debug/report this specific problem 
more precise?


thanks
martin
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Cannot Deinstall a Debian Package

2011-05-03 Thread Chris Samuel
On 04/05/11 08:20, cac...@quantum-sci.com wrote:

 My whole system is installed over BTRFS.  If this is
 non-functional in any OS there should be a warning
 indicating it is non-functional.

The text for btrfs in the kernel configuration says:

 Btrfs is highly experimental, and THE DISK FORMAT IS NOT YET
 FINALIZED.  You should say N here unless you are interested in
 testing Btrfs with non-critical data.

The distros may well enable it for such people but you also
have a duty of care to ensure that you understand the filesystems
that you are choosing to use.  Nevertheless this is not a bug
in btrfs at all.

You need to report the bug in grub-probe (part of the grub-common
package I believe) to Debian. There was a similar bug reported here:

 http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=540786

But that should have been fixed in 1.98+20100602-1 and the
current version in Squeeze is 1.98+20100804-14.

Best of luck,
Chris
-- 
 Chris Samuel  :  http://www.csamuel.org/  :  Melbourne, VIC
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


btrfs seed with luks encrypted devices

2011-05-03 Thread Geoff Ritter
Not sure where to report bugs or even find a coherent list of them.  
Sorry if this is already well known.


When attempting to use an unlocked encrypted device as either a seed 
device or the writeable device, a kernel bug will be displayed at 
fs/btrfs/volumes.c:2402 after attempting to add the writeable device to 
the mounted read-only seed.


STR:
1.  cryptsetup luksFormat /dev/sdx1
2.  cryptsetup luksOpen /dev/sdx1 luksSeed
3.  mkfs.btrfs /dev/mapper/luksSeed
4.  mount and add files if you want, then unmount
5.  btrfstune -S 1 /dev/mapper/luksSeed
6.  mount /dev/mapper/luksSeed /mnt/luksSeed
7.  btrfs device add /dev/sdx2 /mnt/luksSeed
8.  Observe kernel BUG.

I would hope to expect to see an error message if this is never intended 
to be possible.  But normal btrfs file systems appear to function 
normally under both encrypted and lvm partitions.


This attached kernel message was from two LVM logical volumes on a luks 
encrypted partition.  However, I also tested this with two regular 
partitions between endrypted-seed/unencrypted-rw,  
endrypted-rw/unencrypted-seed, and both encrypted.


[ cut here ]
 kernel BUG at fs/btrfs/volumes.c:2402!
 invalid opcode:  [#1] SMP
 last sysfs file: 
/sys/devices/pci:00/:00:05.0/host0/target0:0:0/0:0:0:0/block/sda/dev

 CPU 0
 Modules linked in: usbhid parport_pc hid firewire_ohci i2c_nforce2 
firewire_core i2c_core forcedeth parport
 Pid: 1845, comm: btrfs Not tainted 2.6.37.6 #3 System manufacturer 
System Product Name/M2N-SLI DELUXE
 RIP: 0010:[8149cdba]  [8149cdba] 
__finish_chunk_alloc+0x21a/0x220

 RSP: 0018:880175533668  EFLAGS: 00010286
 RAX: ffe4 RBX: 880176004500 RCX: 0040
 RDX:  RSI: ea000523aff0 RDI: 88017788df00
 RBP: 8801755336e8 R08: bda5 R09: 
 R10:  R11: ffe4 R12: 880177e8e000
 R13: 880176956b00 R14: 8801773cb000 R15: 0070
 FS:  7fe05f784760() GS:8800bfc0() 
knlGS:

 CS:  0010 DS:  ES:  CR0: 8005003b
 CR2: 00683a10 CR3: 00017549c000 CR4: 06f0
 DR0:  DR1:  DR2: 
 DR3:  DR6: 0ff0 DR7: 0400
 Process btrfs (pid: 1845, threadinfo 880175532000, task 
88017547ed00)

 Stack:
  1000 8801 0800 00030a40
  8801773cc000 0800 8801770ef810 8801770ef810
  1910  8800bfd119ff 8801773cb000
 Call Trace:
  [8149f18e] btrfs_alloc_chunk+0x8e/0xa0
  [814647ee] do_chunk_alloc+0x14e/0x2a0
  [814685f2] btrfs_reserve_extent+0xd2/0x180
  [81468bb1] btrfs_alloc_free_block+0xc1/0x330
  [8145696d] __btrfs_cow_block+0x14d/0x610
  [81456f3f] btrfs_cow_block+0x10f/0x200
  [8145bfaa] btrfs_search_slot+0x50a/0x880
  [81455f5a] ? btrfs_free_path+0x2a/0x40
  [8145d38e] btrfs_insert_empty_items+0x7e/0xe0
  [8146efa7] btrfs_insert_empty_inode+0x37/0x40
  [814b554f] create_reloc_inode.clone.41+0x9f/0x230
  [81113257] ? kmem_cache_alloc+0xb7/0x110
  [814bb91b] btrfs_relocate_block_group+0x14b/0x2e0
  [8149bf03] btrfs_relocate_chunk.clone.41+0x83/0x5b0
  [8149a180] ? map_extent_buffer+0xb0/0xc0
  [814890f5] ? btrfs_chunk_type+0xe5/0xf0
  [814a0b6b] btrfs_init_new_device+0xaeb/0xd00
  [814a6476] ? btrfs_ioctl+0x496/0x9d0
  [814a6498] btrfs_ioctl+0x4b8/0x9d0
  [8102a0a4] ? do_page_fault+0x1a4/0x3d0
  [8113002d] do_vfs_ioctl+0x9d/0x580
  [811339fe] ? dput+0x7e/0x160
  [811211a2] ? fput+0x192/0x250
  [81130591] sys_ioctl+0x81/0xa0
  [81002a2b] system_call_fastpath+0x16/0x1b
 Code: ef e8 eb 5e c7 ff 48 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f 
c9 c3 48 83 c4 58 b8 f4 ff ff ff 5b 41 5c 41 5d 41 5e 41 5f c9 c3 0f 
0b 0f 0b 0f 0b 55 48 89 e5 41 57 41 56 41 55 41 54 53 48 83

 RIP  [8149cdba] __finish_chunk_alloc+0x21a/0x220
  RSP 880175533668
 ---[ end trace 8fa94cbaf8bdef31 ]---

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: btrfs seed with luks encrypted devices

2011-05-03 Thread cwillu
On Tue, May 3, 2011 at 7:32 PM, Geoff Ritter geoff.rit...@gmail.com wrote:
 Not sure where to report bugs or even find a coherent list of them.  Sorry
 if this is already well known.

 When attempting to use an unlocked encrypted device as either a seed device
 or the writeable device, a kernel bug will be displayed at
 fs/btrfs/volumes.c:2402 after attempting to add the writeable device to the
 mounted read-only seed.

 STR:
 1.  cryptsetup luksFormat /dev/sdx1
 2.  cryptsetup luksOpen /dev/sdx1 luksSeed
 3.  mkfs.btrfs /dev/mapper/luksSeed
 4.  mount and add files if you want, then unmount
 5.  btrfstune -S 1 /dev/mapper/luksSeed
 6.  mount /dev/mapper/luksSeed /mnt/luksSeed
 7.  btrfs device add /dev/sdx2 /mnt/luksSeed
 8.  Observe kernel BUG.

 I would hope to expect to see an error message if this is never intended to
 be possible.  But normal btrfs file systems appear to function normally
 under both encrypted and lvm partitions.

 This attached kernel message was from two LVM logical volumes on a luks
 encrypted partition.  However, I also tested this with two regular
 partitions between endrypted-seed/unencrypted-rw,
  endrypted-rw/unencrypted-seed, and both encrypted.

 [ cut here ]
  kernel BUG at fs/btrfs/volumes.c:2402!
  invalid opcode:  [#1] SMP
  last sysfs file:
 /sys/devices/pci:00/:00:05.0/host0/target0:0:0/0:0:0:0/block/sda/dev
  CPU 0
  Modules linked in: usbhid parport_pc hid firewire_ohci i2c_nforce2
 firewire_core i2c_core forcedeth parport
  Pid: 1845, comm: btrfs Not tainted 2.6.37.6 #3 System manufacturer System
 Product Name/M2N-SLI DELUXE
  RIP: 0010:[8149cdba]  [8149cdba]
 __finish_chunk_alloc+0x21a/0x220
  RSP: 0018:880175533668  EFLAGS: 00010286
  RAX: ffe4 RBX: 880176004500 RCX: 0040
  RDX:  RSI: ea000523aff0 RDI: 88017788df00
  RBP: 8801755336e8 R08: bda5 R09: 
  R10:  R11: ffe4 R12: 880177e8e000
  R13: 880176956b00 R14: 8801773cb000 R15: 0070
  FS:  7fe05f784760() GS:8800bfc0()
 knlGS:
  CS:  0010 DS:  ES:  CR0: 8005003b
  CR2: 00683a10 CR3: 00017549c000 CR4: 06f0
  DR0:  DR1:  DR2: 
  DR3:  DR6: 0ff0 DR7: 0400
  Process btrfs (pid: 1845, threadinfo 880175532000, task
 88017547ed00)
  Stack:
  1000 8801 0800 00030a40
  8801773cc000 0800 8801770ef810 8801770ef810
  1910  8800bfd119ff 8801773cb000
  Call Trace:
  [8149f18e] btrfs_alloc_chunk+0x8e/0xa0
  [814647ee] do_chunk_alloc+0x14e/0x2a0
  [814685f2] btrfs_reserve_extent+0xd2/0x180
  [81468bb1] btrfs_alloc_free_block+0xc1/0x330
  [8145696d] __btrfs_cow_block+0x14d/0x610
  [81456f3f] btrfs_cow_block+0x10f/0x200
  [8145bfaa] btrfs_search_slot+0x50a/0x880
  [81455f5a] ? btrfs_free_path+0x2a/0x40
  [8145d38e] btrfs_insert_empty_items+0x7e/0xe0
  [8146efa7] btrfs_insert_empty_inode+0x37/0x40
  [814b554f] create_reloc_inode.clone.41+0x9f/0x230
  [81113257] ? kmem_cache_alloc+0xb7/0x110
  [814bb91b] btrfs_relocate_block_group+0x14b/0x2e0
  [8149bf03] btrfs_relocate_chunk.clone.41+0x83/0x5b0
  [8149a180] ? map_extent_buffer+0xb0/0xc0
  [814890f5] ? btrfs_chunk_type+0xe5/0xf0
  [814a0b6b] btrfs_init_new_device+0xaeb/0xd00
  [814a6476] ? btrfs_ioctl+0x496/0x9d0
  [814a6498] btrfs_ioctl+0x4b8/0x9d0
  [8102a0a4] ? do_page_fault+0x1a4/0x3d0
  [8113002d] do_vfs_ioctl+0x9d/0x580
  [811339fe] ? dput+0x7e/0x160
  [811211a2] ? fput+0x192/0x250
  [81130591] sys_ioctl+0x81/0xa0
  [81002a2b] system_call_fastpath+0x16/0x1b
  Code: ef e8 eb 5e c7 ff 48 83 c4 58 31 c0 5b 41 5c 41 5d 41 5e 41 5f c9 c3
 48 83 c4 58 b8 f4 ff ff ff 5b 41 5c 41 5d 41 5e 41 5f c9 c3 0f 0b 0f 0b 0f
 0b 55 48 89 e5 41 57 41 56 41 55 41 54 53 48 83
  RIP  [8149cdba] __finish_chunk_alloc+0x21a/0x220
  RSP 880175533668
  ---[ end trace 8fa94cbaf8bdef31 ]---

Can you try again using the latest 2.6.39rc?  This is a enospc-related
error (RAX: ffe4), and a bunch of those have been fixed
since 2.6.37.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: btrfs csum failed

2011-05-03 Thread Fajar A. Nugraha
On Wed, May 4, 2011 at 7:44 AM, Martin Schitter m...@mur.at wrote:
 Am 2011-05-04 02:28, schrieb Josef Bacik:

 Wait why are you running with btrfs in production?

 do you know a better alternative for continuous snapshots? :)

zfs :D


 it works surprisingly well since more than a year.
 well the performance could be better for vm-image-hosting but it works.

 we used cache='writeback' for a long time but now all virtual instances have
 set cache='none'

 What OS is in this vm image?

 2.6.30-bpo.1-amd64 with virtio-driver

 could you give me some advice how to debug/report this specific problem more
 precise?

If it's not reproducible then I'd suspect it'd be hard to do.

Usually checksum errors is early sign of hardware failure (most common
are disk or power supply).

-- 
Fajar
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Cannot Deinstall a Debian Package

2011-05-03 Thread Fajar A. Nugraha
On Wed, May 4, 2011 at 5:20 AM,  cac...@quantum-sci.com wrote:
 On Tuesday 3 May, 2011 14:26:52 Fajar A. Nugraha wrote:
 Does Debian (or whatever distro you use) support BTRFS /?
 If yes, you should ask them.

 What do you mean 'does Debian support BTRFS'?  The kernel supports it.

Just because you can use something doesn't mean it's supported by the distro.

For example, RHEL6 marks btrfs as technology preview, in other words
you can use it to try out new features but don't complain if
anything's broken. There should be a similar warning in Debian.

In your case, the broken part is the integration with other distro
components (e.g grub)

 And why would they know more about BTRFS than you?

A good distro would normally test all components included, and mark
them as supported or not. If it's not supported, then you should
expect lower level of funcionality or integration compared to
supported components. Some signs of unsupported components:
- it's marked as technology preview (like in RHEL6 case)
- kernel supports it, but the distro installer does not let you use it
by default (needs some manual setup of installer flags)
- it's on a different repository (like Ubuntu's universe/multiverse)
- it's not listed as supported component

Sometimes when a distro includes a technology preview, they'd also
include known issues, caveats, or workaround needed to make it work.
In Ubuntu maverick (I suspect it's also the same in Debian) you need
to manually update to newer version of grub-pc.


 My whole system is installed over BTRFS.  If this is non-functional in any OS 
 there should be a warning indicating it is non-functional.


There is, though the location and form may be distributed all over the place.

There's a warning in the kernel (see Chris's post)

Debian install manual
(http://www.debian.org/releases/stable/amd64/ch06s03.html.en#di-partition)
also doesn't list btrfs as supported partition type, in other words
it's unsupported.


 Looks like grub problem. I know that Ubuntu Natty's grub-pc (grub2)
 work just fine, so you might be able to fix it by upgrading to newer
 grub/grub-pc (perhaps from Debian-unstable).

 I would be happy to upgrade grub, but the package management system is jammed 
 because of this.

You can download grub-related packages (should be grub and grub-pc,
possibly from Debian unstable) and install it manually using dpkg.

You might also need to temporarily rename /usr/sbin/update-grub
manually elsewhere and replace it with symlink to /bin/true, or move
/etc/kernel/postinst.d/zz-update-grub out of the way (just to enable
update process run correctly).

-- 
Fajar
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH, fixed] Prevent oopsing in posix_acl_valid()

2011-05-03 Thread Daniel J Blueman
If posix_acl_from_xattr() returns an error code, a negative address is
dereferenced causing an oops; fix by checking for an error code first.

Typo fixed; too much late-night coding.

Signed-off-by: Daniel J Blueman daniel.blue...@gmail.com
---
 fs/btrfs/acl.c |5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/acl.c b/fs/btrfs/acl.c
index 5d505aa..44ea5b9 100644
--- a/fs/btrfs/acl.c
+++ b/fs/btrfs/acl.c
@@ -178,12 +178,13 @@ static int btrfs_xattr_acl_set(struct dentry *dentry, 
const char *name,
 
if (value) {
acl = posix_acl_from_xattr(value, size);
+   if (IS_ERR(acl))
+   return PTR_ERR(acl);
+
if (acl) {
ret = posix_acl_valid(acl);
if (ret)
goto out;
-   } else if (IS_ERR(acl)) {
-   return PTR_ERR(acl);
}
}
 
-- 
1.7.4.1

--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html