Re: btrfs and 1 billion small files

2012-05-08 Thread Chris Samuel
On 07/05/12 20:06, Boyd Waters wrote: Use a directory hierarchy. Even if the filesystem handles a flat structure effectively, userspace programs will choke on tens of thousands of files in a single directory. For example 'ls' will try to lexically sort its output (very slowly) unless given

Can btrfs silently repair read-error in raid1

2012-05-08 Thread Clemens Eisserer
Hi, I have a quite unreliable SSD here which develops some bad blocks from time to time which result in read-errors. Once the block is written to again, its remapped internally and everything is fine again for that block. Would it be possible to create 2 btrfs partitions on that drive and use it

Re: Can btrfs silently repair read-error in raid1

2012-05-08 Thread Jan Schmidt
On Tue, May 08, 2012 at 09:13 (+0200), Clemens Eisserer wrote: Would it be possible to create 2 btrfs partitions on that drive and use it in RAID1 - with btrfs silently repairing read-errors when they occur? Would it require special settings, to not fallback to read-only mode when a

Re: Can btrfs silently repair read-error in raid1

2012-05-08 Thread Fajar A. Nugraha
On Tue, May 8, 2012 at 2:13 PM, Clemens Eisserer linuxhi...@gmail.com wrote: Hi, I have a quite unreliable SSD here which develops some bad blocks from time to time which result in read-errors. Once the block is written to again, its remapped internally and everything is fine again for that

Re: kernel 3.3.4 damages filesystem (?)

2012-05-08 Thread Helmut Hullen
Hallo, Martin, Du meintest am 08.05.12: No - since some years I use a kind of outsourced backup. A copy of all data is on a bundle of disks somewhere in the neighbourhood. As mentionend: the data isn't business critical, it's just nice to have. It's not worth something like raid1 or so

Re: kernel 3.3.4 damages filesystem (?)

2012-05-08 Thread Fajar A. Nugraha
On Tue, May 8, 2012 at 2:39 PM, Helmut Hullen hul...@t-online.de wrote: And you can use three BTRFS filesystems the same way as three Ext4 filesystems if you prefer such a setup if the time spent for restoring the backup does not make up the cost for one additional disk for you. But where's

[PATCH] Btrfs: use ALIGN macro instead of open-coded expression

2012-05-08 Thread Yuanhan Liu
According to section 'Find open-coded helpers or macros' at https://btrfs.wiki.kernel.org/index.php/Cleanup_ideas, here in the patch we use ALIGN macro to do the alignment. Signed-off-by: Yuanhan Liu yuanhan@linux.intel.com --- fs/btrfs/extent-tree.c |4 ++-- fs/btrfs/extent_io.c |

3.4-rc6: delayed alloc deadlock...

2012-05-08 Thread Daniel J Blueman
Delayed allocation ref mutexes are taken [1] inside btrfs_commit_transaction. A later call fails and jumps to the cleanup_transaction label (transaction.c:1501) with these mutexes still held causing deadlock [2] when they are reacquired. Either we can introduce an earlier label

Re: kernel 3.3.4 damages filesystem (?)

2012-05-08 Thread Helmut Hullen
Hallo, Fajar, Du meintest am 08.05.12: And you can use three BTRFS filesystems the same way as three Ext4 filesystems if you prefer such a setup if the time spent for restoring the backup does not make up the cost for one additional disk for you. But where's the gain? If a disk fails I

Re: [PATCH 2/4] Btrfs: fix deadlock on sb-s_umount when doing umount

2012-05-08 Thread Miao Xie
On Mon, 30 Apr 2012 18:41:39 +0200, David Sterba wrote: On Fri, Apr 27, 2012 at 06:55:13PM +0800, Miao Xie wrote: But I found you add a trylock for -s_umount in cleaner_kthread(), this method can fix the deadlock problem, I think. It may be introduced by the other patch, could you send

Re: kernel 3.3.4 damages filesystem (?)

2012-05-08 Thread Clemens Eisserer
Hi Helmut, But where's the gain? If a disk fails I have a lot of tools for repairing an ext2/3/4 system. Nope, when a disk in your ext4 raid0 array fails, you are just as doomed. But when I use ext2/3/4 I neither need RAID0 nor do I need LVM. You can use btrfs, without using its raid

Re: Can btrfs silently repair read-error in raid1

2012-05-08 Thread cwillu
On Tue, May 8, 2012 at 1:36 AM, Fajar A. Nugraha l...@fajar.net wrote: On Tue, May 8, 2012 at 2:13 PM, Clemens Eisserer linuxhi...@gmail.com wrote: Hi, I have a quite unreliable SSD here which develops some bad blocks from time to time which result in read-errors. Once the block is written

Re: btrfs and 1 billion small files

2012-05-08 Thread Chris Mason
On Mon, May 07, 2012 at 11:28:13AM +0200, Alessio Focardi wrote: Hi, I need some help in designing a storage structure for 1 billion of small files (512 Bytes), and I was wondering how btrfs will fit in this scenario. Keep in mind that I never worked with btrfs - I just read some

Re: kernel 3.3.4 damages filesystem (?)

2012-05-08 Thread Helmut Hullen
Hallo, Clemens, Du meintest am 08.05.12: But where's the gain? If a disk fails I have a lot of tools for repairing an ext2/3/4 system. Nope, when a disk in your ext4 raid0 array fails, you are just as doomed. Why should I use RAID0 with a bundle of ext2/3/4? Mounting on/in the directory

Re: kernel 3.3.4 damages filesystem (?)

2012-05-08 Thread Felix Blanke
On 5/8/12 3:13 PM, Helmut Hullen wrote: Hallo, Clemens, Du meintest am 08.05.12: But where's the gain? If a disk fails I have a lot of tools for repairing an ext2/3/4 system. Nope, when a disk in your ext4 raid0 array fails, you are just as doomed. Why should I use RAID0 with a bundle of

Re: kernel 3.3.4 damages filesystem (?)

2012-05-08 Thread Hugo Mills
On Tue, May 08, 2012 at 03:44:12PM +0200, Felix Blanke wrote: On 5/8/12 3:13 PM, Helmut Hullen wrote: Hallo, Clemens, Du meintest am 08.05.12: But where's the gain? If a disk fails I have a lot of tools for repairing an ext2/3/4 system. Nope, when a disk in your ext4 raid0 array fails,

Re: [PATCH 2/4] Btrfs: fix deadlock on sb-s_umount when doing umount

2012-05-08 Thread David Sterba
On Tue, May 08, 2012 at 06:38:01PM +0800, Miao Xie wrote: I think this method can not fix the problem safely because if the other background threads(not the cleaner) call shrink_delalloc(), the problem can still occur. And it does not indeed fix the problem completely, I found xfstests/269

Re: btrfs and 1 billion small files

2012-05-08 Thread Martin
On 07/05/12 12:05, viv...@gmail.com wrote: Il 07/05/2012 11:28, Alessio Focardi ha scritto: Hi, I need some help in designing a storage structure for 1 billion of small files (512 Bytes), and I was wondering how btrfs will fit in this scenario. Keep in mind that I never worked with btrfs - I

Re: btrfs and 1 billion small files

2012-05-08 Thread Martin
On 08/05/12 13:31, Chris Mason wrote: [...] A few people have already mentioned how btrfs will pack these small files into metadata blocks. If you're running btrfs on a single disk, [...] But the cost is increased CPU usage. Btrfs hits memmove and memcpy pretty hard when you're using

Re: kernel 3.3.4 damages filesystem (?)

2012-05-08 Thread Helmut Hullen
Hallo, Felix, Du meintest am 08.05.12: Why should I use RAID0 with a bundle of ext2/3/4? Mounting on/in the directory tree does the job. Nobody told you that you should do it. What EVERYBODY here is telling you: The problem you have right now would be the same damn problem, no matter what

Re: kernel 3.3.4 damages filesystem (?)

2012-05-08 Thread Felix Blanke
On 5/8/12 6:53 PM, Helmut Hullen wrote: Hallo, Felix, Du meintest am 08.05.12: Why should I use RAID0 with a bundle of ext2/3/4? Mounting on/in the directory tree does the job. Nobody told you that you should do it. What EVERYBODY here is telling you: The problem you have right now would

Re: kernel 3.3.4 damages filesystem (?)

2012-05-08 Thread Helmut Hullen
Hallo, Felix, Du meintest am 08.05.12: As I've written many times: I want a system for my video collection which allows adding a bigger disk deleting/removing a smaller disk with simple commands. btrfs seems to be able to do that (and I have tested this job many

Re: kernel 3.3.4 damages filesystem (?)

2012-05-08 Thread Felix Blanke
On 5/8/12 8:29 PM, Helmut Hullen wrote: Hallo, Felix, Du meintest am 08.05.12: As I've written many times: I want a system for my video collection which allows adding a bigger disk deleting/removing a smaller disk with simple commands. btrfs seems to be able to do that

Re: kernel 3.3.4 damages filesystem (?)

2012-05-08 Thread David Sterba
On Tue, May 08, 2012 at 08:41:47PM +0200, Felix Blanke wrote: As I've written some hours ago: I run btrfs fi balance ... after adding and after deleting a disk. Maybe it's not necessary. Especially it seems not to be necessary after adding a disk. What are the steps you're

Re: kernel 3.3.4 damages filesystem (?)

2012-05-08 Thread Helmut Hullen
Hallo, Felix, Du meintest am 08.05.12: adding a bigger disk deleting/removing a smaller disk with simple commands. [...] Is it really possible to remove a disk from btrfs (created with -d single) without losing the data on that disk? When the system is configured

Re: kernel 3.3.4 damages filesystem (?)

2012-05-08 Thread Hugo Mills
On Tue, May 08, 2012 at 09:34:00PM +0200, Helmut Hullen wrote: Hallo, Felix, Du meintest am 08.05.12: adding a bigger disk deleting/removing a smaller disk with simple commands. [...] Is it really possible to remove a disk from btrfs (created with -d

Re: kernel 3.3.4 damages filesystem (?)

2012-05-08 Thread Helmut Hullen
Hallo, Hugo, Du meintest am 08.05.12: Otherwise if you remove a disk from a raid0 (doesn't matter if you have 2 or 5 or x disks in the fs, btrfs should stripe above all disks) your fs should be broken. Not with btrfs ... there it works even with mkfs.btrfs -m raid1 -d raid0 ...

Re: btrfs and 1 billion small files

2012-05-08 Thread Chris Mason
On Tue, May 08, 2012 at 05:51:05PM +0100, Martin wrote: On 08/05/12 13:31, Chris Mason wrote: [...] A few people have already mentioned how btrfs will pack these small files into metadata blocks. If you're running btrfs on a single disk, [...] But the cost is increased CPU usage.

Re: kernel 3.3.4 damages filesystem (?)

2012-05-08 Thread Roman Mamedov
On 08 May 2012 22:19:00 +0200 hul...@t-online.de (Helmut Hullen) wrote: What I still hope (may be it's impossible): when 1 disk/partition fails, then the contents of the other disks is somehow restorable. And not irreproducable. You should look for file/directory-level tree merging, e.g.

Re: kernel 3.3.4 damages filesystem (?)

2012-05-08 Thread Hubert Kario
On Tuesday 08 of May 2012 12:00:00 Helmut Hullen wrote: Hallo, Fajar, Du meintest am 08.05.12: And you can use three BTRFS filesystems the same way as three Ext4 filesystems if you prefer such a setup if the time spent for restoring the backup does not make up the cost for one additional

Re: Can btrfs silently repair read-error in raid1

2012-05-08 Thread Hubert Kario
On Tuesday 08 of May 2012 04:45:51 cwillu wrote: On Tue, May 8, 2012 at 1:36 AM, Fajar A. Nugraha l...@fajar.net wrote: On Tue, May 8, 2012 at 2:13 PM, Clemens Eisserer linuxhi...@gmail.com wrote: Hi, I have a quite unreliable SSD here which develops some bad blocks from time to time

Re: [PATCH 3/3] btrfs: extended inode refs

2012-05-08 Thread Mark Fasheh
Hi Jan, comments inline as usual! On Thu, Apr 12, 2012 at 07:59:36PM +0200, Jan Schmidt wrote: @@ -858,62 +859,75 @@ static int inode_ref_info(u64 inum, u64 ioff, struct btrfs_root *fs_root, } /* - * this iterates to turn a btrfs_inode_ref into a full filesystem path. elements

Re: Can btrfs silently repair read-error in raid1

2012-05-08 Thread Chris Samuel
On 08/05/12 17:13, Clemens Eisserer wrote: Would it be possible to create 2 btrfs partitions on that drive and use it in RAID1 - with btrfs silently repairing read-errors when they occur? You can, I tried that in 2009:

Re: 3.4-rc6: delayed alloc deadlock...

2012-05-08 Thread Liu Bo
On 05/08/2012 04:56 PM, Daniel J Blueman wrote: Delayed allocation ref mutexes are taken [1] inside btrfs_commit_transaction. A later call fails and jumps to the cleanup_transaction label (transaction.c:1501) with these mutexes still held causing deadlock [2] when they are reacquired.

Re: [PATCH 2/4] Btrfs: fix deadlock on sb-s_umount when doing umount

2012-05-08 Thread Miao Xie
On Tue, 8 May 2012 17:33:26 +0200, David Sterba wrote: On Tue, May 08, 2012 at 06:38:01PM +0800, Miao Xie wrote: I think this method can not fix the problem safely because if the other background threads(not the cleaner) call shrink_delalloc(), the problem can still occur. And it does not

Re: 3.4-rc6: delayed alloc deadlock...

2012-05-08 Thread Miao Xie
On tue, 8 May 2012 16:56:27 +0800, Daniel J Blueman wrote: Delayed allocation ref mutexes are taken [1] inside btrfs_commit_transaction. A later call fails and jumps to the cleanup_transaction label (transaction.c:1501) with these mutexes still held causing deadlock [2] when they are

Is there an announce mailing list?

2012-05-08 Thread spiralofhope
I find the mailing list too busy. Is there an announce mailing list or some other simple way to get notified when a stable is released? -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: Is there an announce mailing list?

2012-05-08 Thread cwillu
On Tue, May 8, 2012 at 10:45 PM, spiralofhope spiralofh...@lavabit.com wrote: I find the mailing list too busy.  Is there an announce mailing list or some other simple way to get notified when a stable is released? That'll only ever be in a new kernel release; there isn't a separate release