Re: btrfs scrub failing

2015-12-31 Thread Duncan
John Center posted on Thu, 31 Dec 2015 11:20:28 -0500 as excerpted: > I run a weekly scrub, using Marc Merlin's btrfs-scrub script. > Usually, it completes without a problem, but this week it failed. I ran > the scrub manually & it stops shortly: > > john@mariposa:~$ sudo /sbin/btrfs scrub start

Re: btrfs fail behavior when a device vanishes

2015-12-31 Thread ronnie sahlberg
On Thu, Dec 31, 2015 at 5:27 PM, Chris Murphy wrote: > On Thu, Dec 31, 2015 at 6:09 PM, ronnie sahlberg > wrote: >> Here is a kludge I hacked up. >> Someone that cares could clean this up and start building a proper >> test suite or something. >> >> This test script creates a 3 disk raid1 filesys

Re: btrfs fail behavior when a device vanishes

2015-12-31 Thread ronnie sahlberg
On Thu, Dec 31, 2015 at 5:27 PM, Chris Murphy wrote: > On Thu, Dec 31, 2015 at 6:09 PM, ronnie sahlberg > wrote: >> Here is a kludge I hacked up. >> Someone that cares could clean this up and start building a proper >> test suite or something. >> >> This test script creates a 3 disk raid1 filesys

Re: btrfs fail behavior when a device vanishes

2015-12-31 Thread Chris Murphy
On Thu, Dec 31, 2015 at 6:09 PM, ronnie sahlberg wrote: > Here is a kludge I hacked up. > Someone that cares could clean this up and start building a proper > test suite or something. > > This test script creates a 3 disk raid1 filesystem and very slowly > writes a large file onto the filesystem w

Re: Unrecoverable fs corruption?

2015-12-31 Thread Chris Murphy
On Thu, Dec 31, 2015 at 4:36 PM, Alexander Duscheleit wrote: > Hello, > > I had a power fail today at my home server and after the reboot the btrfs > RAID1 won't come back up. > > When trying to mount one of the 2 disks of the array I get the following > error: > [ 4126.316396] BTRFS info (device

Re: btrfs fail behavior when a device vanishes

2015-12-31 Thread ronnie sahlberg
Here is a kludge I hacked up. Someone that cares could clean this up and start building a proper test suite or something. This test script creates a 3 disk raid1 filesystem and very slowly writes a large file onto the filesystem while, one by one each disk is disconnected then reconnected in a loo

Re: kernel BUG at fs/btrfs/send.c:1482

2015-12-31 Thread Christoph Anton Mitterer
On Thu, 2015-12-31 at 18:29 +, Filipe Manana wrote: > As for fixing the (very) rare cases where we end up creating empty > symlinks, it's not trivial to fix. Would it be reasonable to have btrfs-check list such broken symlinks? Cheers, Chris. smime.p7s Description: S/MIME cryptographic signa

Unrecoverable fs corruption?

2015-12-31 Thread Alexander Duscheleit
Hello, I had a power fail today at my home server and after the reboot the btrfs RAID1 won't come back up. When trying to mount one of the 2 disks of the array I get the following error: [ 4126.316396] BTRFS info (device sdb2): disk space caching is enabled [ 4126.316402] BTRFS: has skinny e

Re: btrfs fail behavior when a device vanishes

2015-12-31 Thread ronnie sahlberg
On Thu, Dec 31, 2015 at 12:11 PM, Chris Murphy wrote: > This is a torture test, no data is at risk. > > Two devices, btrfs raid1 with some stuff on them. > Copy from that array, elsewhere. > During copy, yank the active device. > > dmesg shows many of these: > > [ 7179.373245] BTRFS error (device

Re: RAID10 question

2015-12-31 Thread Chris Murphy
On Thu, Dec 31, 2015 at 6:31 AM, Duncan <1i5t5.dun...@cox.net> wrote: > Additionally, if you're going to put btrfs on mdraid, then you may wish > to consider reversing the above, doing raid01, which while ordinarily > discouraged in favor of raid10, has some things going for it when the top > leve

btrfs send clone use case

2015-12-31 Thread Chris Murphy
I haven't previously heard of this use case for -c option. It seems to work (no errors or fs weirdness afterward). The gist: send a snapshot from drive 1 to drive 2; rw snapshot of the drive 2 copy, and then make changes to it, then make an ro snapshot; now send it back to drive 1 *as an increment

Re: btrfs fail behavior when a device vanishes

2015-12-31 Thread Chris Murphy
On Thu, Dec 31, 2015 at 1:24 PM, Hugo Mills wrote: > On Thu, Dec 31, 2015 at 01:11:25PM -0700, Chris Murphy wrote: >> This is a torture test, no data is at risk. >> >> Two devices, btrfs raid1 with some stuff on them. >> Copy from that array, elsewhere. >> During copy, yank the active device. >> >

Re: btrfs fail behavior when a device vanishes

2015-12-31 Thread Hugo Mills
On Thu, Dec 31, 2015 at 01:11:25PM -0700, Chris Murphy wrote: > This is a torture test, no data is at risk. > > Two devices, btrfs raid1 with some stuff on them. > Copy from that array, elsewhere. > During copy, yank the active device. > > dmesg shows many of these: > > [ 7179.373245] BTRFS erro

Re: btrfs fail behavior when a device vanishes

2015-12-31 Thread Chris Murphy
On Thu, Dec 31, 2015 at 1:11 PM, Chris Murphy wrote: > Also, is there a command to make a block device go away? Maybe? echo 1 > /sys/block/device-name/device/delete -- Chris Murphy -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...

btrfs fail behavior when a device vanishes

2015-12-31 Thread Chris Murphy
This is a torture test, no data is at risk. Two devices, btrfs raid1 with some stuff on them. Copy from that array, elsewhere. During copy, yank the active device. dmesg shows many of these: [ 7179.373245] BTRFS error (device sdc1): bdev /dev/sdc1 errs: wr 652123, rd 697237, flush 0, corrupt 0,

Re: kernel BUG at fs/btrfs/send.c:1482

2015-12-31 Thread Filipe Manana
On Thu, Dec 31, 2015 at 4:27 PM, Stephen R. van den Berg wrote: > Stephen R. van den Berg wrote: >>I'm running 4.4.0-rc7. >>This exact problem was present on 4.0.5 and 4.3.3 too though. > >>I do a "btrfs send /var/lib/lxc/template64/rootfs", that generates >>the following error consistently at the

[PATCH] Btrfs: don't leave dangling dentry if symlink creation failed

2015-12-31 Thread fdmanana
From: Filipe Manana When we are creating a symlink we might fail with an error after we created its inode and added the corresponding directory indexes to its parent inode. In this case we end up never removing the directory indexes because the inode eviction handler, called for our symlink inode

[PATCH] Btrfs: fix number of transaction units required to create symlink

2015-12-31 Thread fdmanana
From: Filipe Manana We weren't accounting for the insertion of an inline extent item for the symlink inode nor that we need to update the parent inode item (through the call to btrfs_add_nondir()). So fix this by including two more transaction units. Signed-off-by: Filipe Manana --- fs/btrfs/i

[PATCH] Btrfs: send, don't BUG_ON() when an empty symlink is found

2015-12-31 Thread fdmanana
From: Filipe Manana When a symlink is successfully created it always has an inline extent containing the source path. However if an error happens when creating the symlink, we can leave in the subvolume's tree a symlink inode without any such inline extent item - this happens if after btrfs_symli

Re: kernel BUG at fs/btrfs/send.c:1482

2015-12-31 Thread Stephen R. van den Berg
Stephen R. van den Berg wrote: >I'm running 4.4.0-rc7. >This exact problem was present on 4.0.5 and 4.3.3 too though. >I do a "btrfs send /var/lib/lxc/template64/rootfs", that generates >the following error consistently at the same file, over and over again: >Dec 29 14:49:04 argo kernel: kernel B

btrfs scrub failing

2015-12-31 Thread John Center
Hi, I run a weekly scrub, using Marc Merlin's btrfs-scrub script. Usually, it completes without a problem, but this week it failed. I ran the scrub manually & it stops shortly: john@mariposa:~$ sudo /sbin/btrfs scrub start -BdR /dev/md124p2 ERROR: scrubbing /dev/md124p2 failed for device id 1: r

kernel BUG at fs/btrfs/send.c:1482

2015-12-31 Thread Stephen R. van den Berg
I'm running 4.4.0-rc7. This exact problem was present on 4.0.5 and 4.3.3 too though. I do a "btrfs send /var/lib/lxc/template64/rootfs", that generates the following error consistently at the same file, over and over again: Dec 29 14:49:04 argo kernel: kernel BUG at fs/btrfs/send.c:1482! Dec 29 1

[PATCH 10/10] btrfs: reada: Fix a debug code typo

2015-12-31 Thread Zhao Lei
Remove one copy of loop to fix the typo of iterate zones. Signed-off-by: Zhao Lei --- fs/btrfs/reada.c | 11 +++ 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/reada.c b/fs/btrfs/reada.c index 902f899..53ee7b1 100644 --- a/fs/btrfs/reada.c +++ b/fs/btrfs/reada.c @

[PATCH 06/10] btrfs: reada: move reada_extent_put to place after __readahead_hook()

2015-12-31 Thread Zhao Lei
We can't release reada_extent earlier than __readahead_hook(), because __readahead_hook() still need to use it, it is necessary to hode a refcnt to avoid it be freed. Actually it is not a problem after my patch named: Avoid many times of empty loop It make reada_extent in above line include at l

[PATCH 08/10] btrfs: reada: Use fs_info instead of root in __readahead_hook's argument

2015-12-31 Thread Zhao Lei
What __readahead_hook() need exactly is fs_info, no need to convert fs_info to root in caller and convert back in __readahead_hook() Signed-off-by: Zhao Lei --- fs/btrfs/ctree.h | 4 ++-- fs/btrfs/disk-io.c | 22 +++--- fs/btrfs/reada.c | 23 +++ 3 files

[PATCH 04/10] btrfs: reada: bypass adding extent when all zone failed

2015-12-31 Thread Zhao Lei
When failed adding all dev_zones for a reada_extent, the extent will have no chance to be selected to run, and keep in memory for ever. We should bypass this extent to avoid above case. Signed-off-by: Zhao Lei --- fs/btrfs/reada.c | 5 + 1 file changed, 5 insertions(+) diff --git a/fs/btrf

[PATCH 01/10] btrfs: reada: Avoid many times of empty loop

2015-12-31 Thread Zhao Lei
We can see following loop(1 times) in trace_log: [ 75.416137] ZL_DEBUG: reada_start_machine_dev:730: pid=771 comm=kworker/u2:3 re->ref_cnt 88003741e0c0 1 -> 2 [ 75.417413] ZL_DEBUG: reada_extent_put:524: pid=771 comm=kworker/u2:3 re = 88003741e0c0, refcnt = 2 -> 1 [ 75.418611

[PATCH 09/10] btrfs: reada: Jump into cleanup in direct way for __readahead_hook()

2015-12-31 Thread Zhao Lei
Current code set nritems to 0 to make for_loop useless to bypass it, and set generation's value which is not necessary. Jump into cleanup directly is better choise. Signed-off-by: Zhao Lei --- fs/btrfs/reada.c | 40 +--- 1 file changed, 21 insertions(+), 19 de

[PATCH 05/10] btrfs: reada: Remove level argument in severial functions

2015-12-31 Thread Zhao Lei
level is not used in severial functions, remove them from arguments, and remove relative code for get its value. Signed-off-by: Zhao Lei --- fs/btrfs/reada.c | 15 ++- 1 file changed, 6 insertions(+), 9 deletions(-) diff --git a/fs/btrfs/reada.c b/fs/btrfs/reada.c index ef9457e..664

[PATCH 07/10] btrfs: reada: Pass reada_extent into __readahead_hook directly

2015-12-31 Thread Zhao Lei
reada_start_machine_dev() already have reada_extent pointer, pass it into __readahead_hook() directly instead of search radix_tree will make code run faster. Signed-off-by: Zhao Lei --- fs/btrfs/reada.c | 45 - 1 file changed, 24 insertions(+), 21 dele

[PATCH 02/10] btrfs: reada: Move is_need_to_readahead contition earlier

2015-12-31 Thread Zhao Lei
Move is_need_to_readahead contition earlier to avoid useless loop to get relative data for readahead. Signed-off-by: Zhao Lei --- fs/btrfs/reada.c | 20 +--- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/fs/btrfs/reada.c b/fs/btrfs/reada.c index fb21bf0..dcc5b69

[PATCH 00/10] btrfs: reada: Avoid many times of empty loop

2015-12-31 Thread Zhao Lei
This is some cleanup, bugfix and enhance for reada, tested by a script running scrub and relatice trace log. Zhao Lei (10): btrfs: reada: Avoid many times of empty loop btrfs: reada: Move is_need_to_readahead contition earlier btrfs: reada: add all reachable mirrors into reada device list

[PATCH 03/10] btrfs: reada: add all reachable mirrors into reada device list

2015-12-31 Thread Zhao Lei
If some device is not reachable, we should bypass and continus addingb next, instead of break on bad device. Signed-off-by: Zhao Lei --- fs/btrfs/reada.c | 20 +--- 1 file changed, 9 insertions(+), 11 deletions(-) diff --git a/fs/btrfs/reada.c b/fs/btrfs/reada.c index dcc5b69..7

Re: RAID10 question

2015-12-31 Thread Duncan
Hugo Mills posted on Thu, 31 Dec 2015 11:51:53 + as excerpted: > On Thu, Dec 31, 2015 at 09:52:16AM +, Xavier Romero wrote: >> Hello, >> >> I have 2 completely independent set of 12 disks each, let's name them >> A1, A2, A3... A12 for first set, and B1, B2, B3...B12 for second set. >> For

Re: quota rescan hangs

2015-12-31 Thread Duncan
Xavier Romero posted on Thu, 31 Dec 2015 10:09:22 + as excerpted: > Using BTRFS on CentOS 7, I get the filesystem hung by running btrfs > quota rescan /mnt/btrfs/ > > After that I could not access to the filesystem anymore until system > restart. > Additional Info: > > [root@nex-dstrg-ctrl

subvolume capacity

2015-12-31 Thread Xavier Romero
Hello, I create a subvolume, I assign a quota on it, I mount the subvolume, the capacity reported for that mount point is not the assigned quota but the total btrfs filesystem capacity. I would rather expect it to report the assigned space (unless there is no quota assigned obviously), that wou

Re: BTRFS File trace from superblock using memory dumps

2015-12-31 Thread Hugo Mills
On Thu, Dec 31, 2015 at 10:20:17AM +, Ahtisham wani wrote: > Respected sir, > I have been researching on how btrfs works and manages files for a long > time. What I want to achieve here is to trace a path of a file starting > from superblock, then to root and so on. The problem is I dont know

Re: RAID10 question

2015-12-31 Thread Hugo Mills
On Thu, Dec 31, 2015 at 09:52:16AM +, Xavier Romero wrote: > Hello, > > I have 2 completely independent set of 12 disks each, let's name them A1, A2, > A3... A12 for first set, and B1, B2, B3...B12 for second set. For > availability purposes I want disks to be paired that way: > A1 <--> B1:

RE: quota rescan hangs

2015-12-31 Thread Xavier Romero
After restarting and removing all data, "qgroup show " tells me that a rescan is in progress, but "quota rescan -s" tells me the opposite. [root@nex-dstrg-ctrl-1 btrfs]# btrfs quota rescan -w /mnt/btrfs/ quota rescan started [root@nex-dstrg-ctrl-1 btrfs]# btrfs quota rescan -s /mnt/btrfs/ no res

RAID10 question

2015-12-31 Thread Xavier Romero
Hello, I have 2 completely independent set of 12 disks each, let's name them A1, A2, A3... A12 for first set, and B1, B2, B3...B12 for second set. For availability purposes I want disks to be paired that way: A1 <--> B1: RAID1 A2 <--> B2: RAID1 ... A12 <--> B12: RAID1 And then I want a RAID0 ou

BTRFS File trace from superblock using memory dumps

2015-12-31 Thread Ahtisham wani
Respected sir, I have been researching on how btrfs works and manages files for a long time. What I want to achieve here is to trace a path of a file starting from superblock, then to root and so on. The problem is I dont know how to do it. I have been using dd and hexdump but I dont know what t

Re: BTRFS - Write Barriers

2015-12-31 Thread fugazzi®
Thanks Duncan for your answer. Yes, I read that part of the wiki and since I saw the "system crash" just wanted to be sure because always only the power loss part is mentioned regarding barriers by other fs such as for example XFS. The fact is that since the barrier stuff was removed from the e

quota rescan hangs

2015-12-31 Thread Xavier Romero
Hello, Using BTRFS on CentOS 7, I get the filesystem hung by running btrfs quota rescan /mnt/btrfs/ After that I could not access to the filesystem anymore until system restart. I did the scan because BTRFS suggested it: [root@nex-dstrg-ctrl-1 btrfs]# btrfs qgroup show -F /mnt/btrfs/ WARNING: Q

Re: BTRFS - Write Barriers

2015-12-31 Thread Duncan
fugazzi® posted on Thu, 31 Dec 2015 09:01:51 + as excerpted: > Just one question for the gurus here. > > I was wondering: if I disable write barriers in btrfs with the mount > option nobarrier I just disable the periodic flushes of the hardware > disk cache or I'm disabling also the order of

BTRFS - Write Barriers

2015-12-31 Thread fugazzi®
Hi everyone :-) Just one question for the gurus here. I was wondering: if I disable write barriers in btrfs with the mount option nobarrier I just disable the periodic flushes of the hardware disk cache or I'm disabling also the order of the writes directed to the hard disk? What I mean is: i