[PATCH] btrfs-progs: prop: remove an unnecessary condition on parse_args

2016-04-19 Thread Satoru Takeuchi
>From commit c742debab11f ('btrfs-progs: fix a regression that
"property" with -t option doesn't work'), the number of arguments
is checked strictly. So the following condition never be
satisfied.

Signed-off-by: Satoru Takeuchi 
---
 cmds-property.c | 5 -
 1 file changed, 5 deletions(-)

diff --git a/cmds-property.c b/cmds-property.c
index eed5f4a..48a8945 100644
--- a/cmds-property.c
+++ b/cmds-property.c
@@ -352,11 +352,6 @@ static void parse_args(int argc, char **argv,
if (value && optind < argc)
*value = argv[optind++];

-   if (optind != argc) {
-   error("unexpected agruments found");
-   usage(usage_str);
-   }
-
if (!*types && object && *object) {
ret = autodetect_object_types(*object, types);
if (ret < 0) {
-- 
2.5.5
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Question: raid1 behaviour on failure

2016-04-19 Thread Matthias Bodenbinder
Am 18.04.2016 um 09:22 schrieb Qu Wenruo:
> BTW, it would be better to post the dmesg for better debug.

So here we. I did the same test again. Here is a full log of what i did. It 
seems to be mean like a bug in btrfs. 
Sequenz of events:
1. mount the raid1 (2 disc with different size)
2. unplug the biggest drive (hotplug)
3. try to copy something to the degraded raid1
4. plugin the device again (hotplug)

This scenario does not work. The disc array is NOT redundant! I can not work 
with it while a drive is missing and I can not reattach the device so that 
everything works again.

The btrfs module crashes during the test.

I am using LMDE2 with backports:
btrfs-tools 4.4-1~bpo8+1
linux-image-4.4.0-0.bpo.1-amd64

Matthias


rakete - root - /root
1# mount /mnt/raid1/

Journal:

Apr 20 07:01:16 rakete kernel: BTRFS info (device sdi): enabling auto defrag
Apr 20 07:01:16 rakete kernel: BTRFS info (device sdi): disk space caching is 
enabled
Apr 20 07:01:16 rakete kernel: BTRFS: has skinny extents

rakete - root - /mnt/raid1
3# ll
insgesamt 0
drwxrwxr-x 1 root root   36 Nov 14  2014 AfterShot2(64-bit)
drwxrwxr-x 1 root root 5082 Apr 17 09:06 etc
drwxr-xr-x 1 root root  108 Mär 24 07:31 var

4# btrfs fi show
Label: none  uuid: 16d5891f-5d52-4b29-8591-588ddf11e73d
Total devices 3 FS bytes used 1.60GiB
devid1 size 698.64GiB used 3.03GiB path /dev/sdg
devid2 size 465.76GiB used 3.03GiB path /dev/sdh
devid3 size 232.88GiB used 0.00B path /dev/sdi


unplug device sdg:

Apr 20 07:03:05 rakete kernel: Buffer I/O error on dev sdf1, logical block 
243826688, lost sync page write
Apr 20 07:03:05 rakete kernel: JBD2: Error -5 detected when updating journal 
superblock for sdf1-8.
Apr 20 07:03:05 rakete kernel: Aborting journal on device sdf1-8.
Apr 20 07:03:05 rakete kernel: Buffer I/O error on dev sdf1, logical block 
243826688, lost sync page write
Apr 20 07:03:05 rakete kernel: JBD2: Error -5 detected when updating journal 
superblock for sdf1-8.
Apr 20 07:03:05 rakete umount[16405]: umount: /mnt/raid1: target is busy
Apr 20 07:03:05 rakete umount[16405]: (In some cases useful info about 
processes that
Apr 20 07:03:05 rakete umount[16405]: use the device is found by lsof(8) or 
fuser(1).)
Apr 20 07:03:05 rakete systemd[1]: mnt-raid1.mount mount process exited, 
code=exited status=32
Apr 20 07:03:05 rakete systemd[1]: Failed unmounting /mnt/raid1.
Apr 20 07:03:24 rakete kernel: usb 3-1: new SuperSpeed USB device number 3 
using xhci_hcd
Apr 20 07:03:24 rakete kernel: usb 3-1: New USB device found, idVendor=152d, 
idProduct=0567
Apr 20 07:03:24 rakete kernel: usb 3-1: New USB device strings: Mfr=10, 
Product=11, SerialNumber=5
Apr 20 07:03:24 rakete kernel: usb 3-1: Product: USB to ATA/ATAPI Bridge
Apr 20 07:03:24 rakete kernel: usb 3-1: Manufacturer: JMicron
Apr 20 07:03:24 rakete kernel: usb 3-1: SerialNumber: 152D00539000
Apr 20 07:03:24 rakete kernel: usb-storage 3-1:1.0: USB Mass Storage device 
detected
Apr 20 07:03:24 rakete kernel: usb-storage 3-1:1.0: Quirks match for vid 152d 
pid 0567: 500
Apr 20 07:03:24 rakete kernel: scsi host9: usb-storage 3-1:1.0
Apr 20 07:03:24 rakete mtp-probe[16424]: checking bus 3, device 3: 
"/sys/devices/pci:00/:00:1c.5/:04:00.0/usb3/3-1"
Apr 20 07:03:24 rakete mtp-probe[16424]: bus: 3, device: 3 was not an MTP device
Apr 20 07:03:25 rakete kernel: scsi 9:0:0:0: Direct-Access WDC WD20 
02FAEX-007BA00125 PQ: 0 ANSI: 6
Apr 20 07:03:25 rakete kernel: scsi 9:0:0:1: Direct-Access WDC WD50 
01AALS-00L3B20125 PQ: 0 ANSI: 6
Apr 20 07:03:25 rakete kernel: scsi 9:0:0:2: Direct-Access SAMSUNG  SP2504C 
 0125 PQ: 0 ANSI: 6
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: Attached scsi generic sg6 type 0
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: Attached scsi generic sg7 type 0
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] 3907029168 512-byte logical 
blocks: (2.00 TB/1.82 TiB)
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Write Protect is off
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Mode Sense: 67 00 10 08
Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: Attached scsi generic sg8 type 0
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] 976773168 512-byte logical 
blocks: (500 GB/466 GiB)
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] No Caching mode page found
Apr 20 07:03:25 rakete kernel: sd 9:0:0:0: [sdf] Assuming drive cache: write 
through
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Write Protect is off
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Mode Sense: 67 00 10 08
Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: [sdk] 488395055 512-byte logical 
blocks: (250 GB/233 GiB)
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] No Caching mode page found
Apr 20 07:03:25 rakete kernel: sd 9:0:0:1: [sdj] Assuming drive cache: write 
through
Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: [sdk] Write Protect is off
Apr 20 07:03:25 rakete kernel: sd 9:0:0:2: [sdk] Mode Sense: 67 00 10 08
Apr 20 07:03:2

Re: [PATCH v8 00/27][For 4.7] Btrfs: Add inband (write time) de-duplication framework

2016-04-19 Thread Qu Wenruo

Hi David,

Any new comment about the ondisk format and ioctl interface?

Thanks,
Qu

Qu Wenruo wrote on 2016/04/05 11:08 +0800:



David Sterba wrote on 2016/04/04 18:55 +0200:

On Fri, Mar 25, 2016 at 09:38:50AM +0800, Qu Wenruo wrote:

Please use the newly added BTRFS_PERSISTENT_ITEM_KEY instead of a new
key type. As this is the second user of that item, there's no
precendent
how to select the subtype. Right now 0 is for the dev stats item, but
I'd like to leave some space between them, so it should be 256 at best.
The space is 64bit so there's enough room but this also means defining
the on-disk format.


After checking BTRFS_PERSISENT_ITEM_KEY, it seems that its value is
larger than current DEDUPE_BYTENR/HASH_ITEM_KEY, and since the objectid
of DEDUPE_HASH_ITEM_KEY, it won't be the first item of the tree.

Although that's not a big problem, but for user using debug-tree, it
would be quite annoying to find it located among tons of other hashes.


You can alternatively store it in the tree_root, but I don't know how
frquently it's supposed to be changed.


Storing in tree root sounds pretty good.
As such status doesn't change until we enable/disable (including
configure), so tree root seems good.

But we still need to consider the later dedupe rate statistics key order.
In that case, I hope to restore them both into dedupe tree.




So personally, if using PERSISTENT_ITEM_KEY, at least I prefer to keep
objectid to 0, and modify DEDUPE_BYTENR/HASH_ITEM_KEY to higher value,
to ensure dedupe status to be the first item of dedupe tree.


0 is unfortnuatelly taken by BTRFS_DEV_STATS_OBJECTID, but I don't see
problem with the ordering. DEDUPE_BYTENR/HASH_ITEM_KEY store a large
number in the objectid, either part of a hash, that's unlikely to be
almost-all zeros and bytenr which will be larger than 1MB.


OK, as long as we can search the status item with exactly match key, it
shouldn't cause big problem.




4) Ioctl interface with persist dedup status


I'd like to see the ioctl specified in more detail. So far there's
enable, disable and status. I'd expect some way to control the
in-memory
limits, let it "forget" current hash cache, specify the dedupe chunk
size, maybe sync of the in-memory hash cache to disk.


So current and planned ioctl should be the following, with some
details
related to your in-memory limit control concerns.

1) Enable
  Enable dedupe if it's not enabled already. (disabled -> enabled)


Ok, so it should also take a parameter which bckend is about to be
enabled.


It already has.
It also has limit_nr and limit_mem parameter for in-memory backend.




  Or change current dedupe setting to another. (re-configure)


Doing that in 'enable' sounds confusing, any changes belong to a
separate command.


This depends the aspect of view.

For "Enable/config/disable" case, it will introduce a state machine for
end-user.


Yes, that's exacly my point.


Personally, I doesn't state machine for end user. Yes, I also hate
merging play and pause button together on music player.


I don't see this reference relevant, we're not designing a music player.


If using state machine, user must ensure the dedupe is enabled before
doing any configuration.


For user convenience we can copy the configuration options to the dedup
enable subcommand, but it will still do separate enable and configure
ioctl calls.


So, that's to say, user can assume there is a state machine, and to do
enable-configure method.
And other user can use the state-less enable-enable method.

If so, I'm OK to add a configure ioctl interface.
(As it's still enable-enable stateless one beneath the stateful ioctl)

But in that case, if user forget to enable dedupe and call configure
directly, btrfs won't give any warning and just enable dedupe.

Will that design be OK for you? Or we need to share most part of enable
and configure ioctl, but configure ioctl will do extra check?





For me, user only need to care the result of the operation. User can now
configure dedupe to their need without need to know previous setting.
  From this aspect of view, "Enable/Disable" is much easier than
"Enable/Config/Disable".


Getting the usability is hard and that's why we're having this
dicussion. What suites you does not suite others, we have different
habits, expectations and there are existing usage patterns. We better
stick to something that's not too surprising yet still flexible enough
to cover broad needs. I'm leaving this open, but I strongly disagree
with the current interface proposal.


I'm still open to new ioctl interface design, as long as we can re-use
most of current code.

Anyway, just as you pointed, the stateless one is just my personal taste.




  For dedupe_bs/backend/hash algorithm(only SHA256 yet) change, it
  will disable dedupe(dropping all hash) and then enable with new
  setting.

  For in-memory backend, if only limit is different from previous
  setting, limit can be changed on the fly without d

Re: [PATCH] btrfs: Test that qgroup counts are valid after snapshot creation

2016-04-19 Thread Satoru Takeuchi

On 2016/04/20 7:25, Mark Fasheh wrote:

This has been broken since Linux v4.1. We may have worked out a solution on
the btrfs list but in the meantime sending a test to expose the issue seems
like a good idea.

Signed-off-by: Mark Fasheh 
---
  tests/btrfs/122   | 88 +++
  tests/btrfs/group |  1 +


You forgot to add tests/btrfs/122.out.


  2 files changed, 89 insertions(+)
  create mode 100755 tests/btrfs/122

diff --git a/tests/btrfs/122 b/tests/btrfs/122
new file mode 100755
index 000..b7e9e4b
--- /dev/null
+++ b/tests/btrfs/122
@@ -0,0 +1,88 @@
+#! /bin/bash
+# FS QA Test No. btrfs/122
+#
+# Test that qgroup counts are valid after snapshot creation. This has
+# been broken in btrfs since Linux v4.1
+#
+#---
+# Copyright (C) 2016 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#---
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1   # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+   cd /
+   rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+rm -f $seqres.full
+
+# Force a small leaf size to make it easier to blow out our root
+# subvolume tree
+_scratch_mkfs "--nodesize 16384"


nodesize 16384 is the default value. Do you
intend other value, for example 4096?


+_scratch_mount
+_run_btrfs_util_prog quota enable $SCRATCH_MNT
+
+mkdir "$SCRATCH_MNT/snaps"
+
+# First make some simple snapshots - the bug was initially reproduced like this
+_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT 
"$SCRATCH_MNT/snaps/empty1"
+_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT 
"$SCRATCH_MNT/snaps/empty2"
+
+# This forces the fs tree out past level 0, adding at least one tree
+# block which must be properly accounted for when we make our next
+# snapshots.
+mkdir "$SCRATCH_MNT/data"
+for i in `seq 0 640`; do
+$XFS_IO_PROG -f -c "pwrite 0 1M" "$SCRATCH_MNT/data/file$i" > /dev/null 
2>&1
+done;


";" after "done" is not necessary.

Thanks,
Satoru


+
+# Snapshot twice.
+_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/snap1"
+_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/snap2"
+
+_scratch_unmount
+
+# generate a qgroup report and look for inconsistent groups
+$BTRFS_UTIL_PROG check --qgroup-report $SCRATCH_DEV 2>&1 | \
+   grep -q -E "Counts for qgroup.*are different"
+if [ $? -ne 0 ]; then
+   status=0
+fi
+
+exit
diff --git a/tests/btrfs/group b/tests/btrfs/group
index 9403daa..f7e8cff 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -122,3 +122,4 @@
  119 auto quick snapshot metadata qgroup
  120 auto quick snapshot metadata
  121 auto quick snapshot qgroup
+122 auto quick snapshot qgroup


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH] btrfs: Test that qgroup counts are valid after snapshot creation

2016-04-19 Thread Mark Fasheh
This has been broken since Linux v4.1. We may have worked out a solution on
the btrfs list but in the meantime sending a test to expose the issue seems
like a good idea.

Signed-off-by: Mark Fasheh 
---
 tests/btrfs/122   | 88 +++
 tests/btrfs/group |  1 +
 2 files changed, 89 insertions(+)
 create mode 100755 tests/btrfs/122

diff --git a/tests/btrfs/122 b/tests/btrfs/122
new file mode 100755
index 000..b7e9e4b
--- /dev/null
+++ b/tests/btrfs/122
@@ -0,0 +1,88 @@
+#! /bin/bash
+# FS QA Test No. btrfs/122
+#
+# Test that qgroup counts are valid after snapshot creation. This has
+# been broken in btrfs since Linux v4.1
+#
+#---
+# Copyright (C) 2016 SUSE Linux Products GmbH. All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#---
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1   # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+   cd /
+   rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+rm -f $seqres.full
+
+# Force a small leaf size to make it easier to blow out our root
+# subvolume tree
+_scratch_mkfs "--nodesize 16384"
+_scratch_mount
+_run_btrfs_util_prog quota enable $SCRATCH_MNT
+
+mkdir "$SCRATCH_MNT/snaps"
+
+# First make some simple snapshots - the bug was initially reproduced like this
+_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT 
"$SCRATCH_MNT/snaps/empty1"
+_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT 
"$SCRATCH_MNT/snaps/empty2"
+
+# This forces the fs tree out past level 0, adding at least one tree
+# block which must be properly accounted for when we make our next
+# snapshots.
+mkdir "$SCRATCH_MNT/data"
+for i in `seq 0 640`; do
+$XFS_IO_PROG -f -c "pwrite 0 1M" "$SCRATCH_MNT/data/file$i" > /dev/null 
2>&1
+done;
+
+# Snapshot twice.
+_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/snap1"
+_run_btrfs_util_prog subvolume snapshot $SCRATCH_MNT "$SCRATCH_MNT/snaps/snap2"
+
+_scratch_unmount
+
+# generate a qgroup report and look for inconsistent groups
+$BTRFS_UTIL_PROG check --qgroup-report $SCRATCH_DEV 2>&1 | \
+   grep -q -E "Counts for qgroup.*are different"
+if [ $? -ne 0 ]; then
+   status=0
+fi
+
+exit
diff --git a/tests/btrfs/group b/tests/btrfs/group
index 9403daa..f7e8cff 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -122,3 +122,4 @@
 119 auto quick snapshot metadata qgroup
 120 auto quick snapshot metadata
 121 auto quick snapshot qgroup
+122 auto quick snapshot qgroup
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v4] btrfs: qgroup: Fix qgroup accounting when creating snapshot

2016-04-19 Thread Mark Fasheh
On Fri, Apr 15, 2016 at 05:08:22PM +0800, Qu Wenruo wrote:
> Current btrfs qgroup design implies a requirement that after calling
> btrfs_qgroup_account_extents() there must be a commit root switch.
> 
> Normally this is OK, as btrfs_qgroup_accounting_extents() is only called
> inside btrfs_commit_transaction() just be commit_cowonly_roots().
> 
> However there is a exception at create_pending_snapshot(), which will
> call btrfs_qgroup_account_extents() but no any commit root switch.
> 
> In case of creating a snapshot whose parent root is itself (create a
> snapshot of fs tree), it will corrupt qgroup by the following trace:
> (skipped unrelated data)
> ==
> btrfs_qgroup_account_extent: bytenr = 29786112, num_bytes = 16384, 
> nr_old_roots = 0, nr_new_roots = 1
> qgroup_update_counters: qgid = 5, cur_old_count = 0, cur_new_count = 1, rfer 
> = 0, excl = 0
> qgroup_update_counters: qgid = 5, cur_old_count = 0, cur_new_count = 1, rfer 
> = 16384, excl = 16384
> btrfs_qgroup_account_extent: bytenr = 29786112, num_bytes = 16384, 
> nr_old_roots = 0, nr_new_roots = 0
> ==
> 
> The problem here is in first qgroup_account_extent(), the
> nr_new_roots of the extent is 1, which means its reference got
> increased, and qgroup increased its rfer and excl.
> 
> But at second qgroup_account_extent(), its reference got decreased, but
> between these two qgroup_account_extent(), there is no switch roots.
> This leads to the same nr_old_roots, and this extent just got ignored by
> qgroup, which means this extent is wrongly accounted.
> 
> Fix it by call commit_cowonly_roots() after qgroup_account_extent() in
> create_pending_snapshot(), with needed preparation.
> 
> Reported-by: Mark Fasheh 
> Signed-off-by: Qu Wenruo 

Ok, this version seems to be giving me the right numbers. I'll send a test
for it shortly. I'd still like to know if this patch introduces an
unintended side effects but otherwise, thanks Qu.
--Mark

--
Mark Fasheh
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Infinite loop/hang in in btrfs_find_space_cluster()

2016-04-19 Thread Eric Wheeler
On Wed, 13 Apr 2016, Eric Wheeler wrote:

> Hello all,
> 
> We just got this backtrace in 4.4.6 on an ARM AM335x (beaglebone 
> compatible).  The trace looks similar to this one:
>   http://permalink.gmane.org/gmane.comp.file-systems.btrfs/54734 

We solved the problem so I thought I would report back in case anyone else 
runs in to the same issue.  

This is running on a memory constrained 700mhz ARM (single core) so we had 
these mount options that worked fine until we went from a single volume to 
raid1, at which time it hung after the first reboot (or remount?):
  
ssd,ssd_spread,thread_pool=1,noatime,compress-force=lzo,nospace_cache,commit=37

so we switched to these on the raid1 system:
  noatime,compress-force=lzo,commit=37
and it stopped hanging.

Ultimately my guess is either of thread_pool=1 causing deadlock (no 
progress), or nospace_cache causing space churn.  

We used thread_pool=1 which worked for a long time on a single volume.  
We just added a 2nd volume and went to raid1, so perhaps the minimum 
thread_pool should be the number of volumes?  I cannot be certain, so FYI 
in case others run into this.

--
Eric Wheeler






but I 
> don't have nice backtraces on this hardware (maybe hang traces are a 
> compile option?).
> 
> In my case, the system appeared to hang but sysrq functions still worked 
> and I was able to send a sysrq-(c)rash over serial.  The filesystem was 
> just formatted in RAID1, and while I cannot access it because this hangs 
> at boot, there can't be very much data yet.
> 
> This looks like the top of the relevant section, full trace below:
> 
> [   80.738518] [] (setup_cluster_no_bitmap [btrfs]) from 
> [] (btrfs_find_space_cluster+0x10c/0x1dc [btrfs])
> 
> Any help you can offer would be greatly appreciated!
> 
> -Eric
> 
> [   80.005546] sysrq: SysRq : Trigger a crash
> [   80.018339] pgd = c0004000
> [   80.021160] [] *pgd=
> [   80.024897] Internal error: Oops: 817 [#1] ARM
> [   80.029531] Modules linked in: btrfs raid10 raid456 async_raid6_recov 
> async_memcpy async_pq async_xor async_tx xor
>  raid6_pq raid0 multipath linear zram lz4_compress zsmalloc dm_thin_pool 
> dm_persistent_data dm_bio_prison dm_snapshot
>  dm_bufio dm_zero dm_mod raid1 md_mod
> [   80.054411] CPU: 0 PID: 6 Comm: kworker/u2:0 Not tainted 
> 4.4.6-vr3-4-g2dfa78e #15
> [   80.062579] Hardware name: Generic AM33XX (Flattened Device Tree)
> [   80.069476] Workqueue: btrfs-delalloc btrfs_delalloc_helper [btrfs]
> [   80.076026] task: cc8344c0 ti: cc858000 task.ti: cc858000
> [   80.081669] PC is at sysrq_handle_crash+0x28/0x30
> [   80.086576] LR is at sysrq_handle_crash+0x24/0x30
> [   80.091484] pc : []lr : []psr: 6193
> [   80.091484] sp : cc8599e8  ip : cc8599e8  fp : cc8599fc
> [   80.103460] r10: cc8ecb80  r9 : c06008ce  r8 : 0001
> [   80.108909] r7 : 0007  r6 : 0063  r5 : c05d6308  r4 : 0001
> [   80.115717] r3 :   r2 : c05d62c8  r1 :   r0 : 0063
> [   80.122529] Flags: nZCv  IRQs off  FIQs on  Mode SVC_32  ISA ARM  Segment 
> none
> [   80.130063] Control: 10c5387d  Table: 8a42c019  DAC: 0051
> [   80.136057] Process kworker/u2:0 (pid: 6, stack limit = 0xcc858210)
> [   80.142595] Stack: (0xcc8599e8 to 0xcc85a000)
> [   80.147145] 99e0:   c02936cc c05e34f4 cc859a24 cc859a00 
> c0293d00 c02936d8
> [   80.155683] 9a00: c05b2bc4 cca49010  0101 0002 0005 
> cc859a34 cc859a28
> [   80.164220] 9a20: c0294208 c0293c64 cc859a74 cc859a38 c02ad3fc c02941e4 
>  
> [   80.172758] 9a40: 0001 0001 00015200 cca22380 cc8ecb90  
>  009b
> [   80.181296] 9a60: c06008ce cc8ecb80 cc859aac cc859a78 c006652c c02ad050 
> c00409f4 c00405e0
> [   80.189833] 9a80:  cc8ecb80 cc8ecb90   cc804000 
> cc15ec50 ccd02518
> [   80.198373] 9aa0: cc859ac4 cc859ab0 c008 c00664d4 00022000 cc8ecb80 
> cc859adc cc859ac8
> [   80.206912] 9ac0: c00690ec c0066644 009b c05de0a0 cc859aec cc859ae0 
> c0065e58 c006905c
> [   80.215451] 9ae0: cc859b14 cc859af0 c0065f7c c0065e44 cc859b30 c06342c0 
> 6013 
> [   80.223990] 9b00: cc859b64 00201000 cc859b2c cc859b18 c00093b8 c0065f34 
> bf187974 bf18799c
> [   80.232529] 9b20: cc859bcc cc859b30 c0412e14 c000938c cc15ec50 cc15fba8 
> 00201000 
> [   80.241066] 9b40: 5000  cc859bf8 ca144880 00201000 cc15ec50 
> ccd02518 cc859bcc
> [   80.249604] 9b60: cc859b80 cc859b80 bf187974 bf18799c 6013  
> 0051 c020aba4
> [   80.258142] 9b80:  0001  cc859bac cc859bf8 c0203d08 
> 2013 ccd02518
> [   80.266680] 9ba0: 0051 00201000  00201000  00201000 
>  cc859bf8
> [   80.275218] 9bc0: cc859c2c cc859bd0 bf18b3b8 bf18791c 41c0  
> 00201000 
> [   80.283756] 9be0: 00201000  00201000  ccd02518 ccb31200 
> cc15fbd4 cc15fbd4
> [   80.292293] 9c00: cce624d0 1000   00

Re: Install to or Recover RAID Array Subvolume Root?

2016-04-19 Thread Nicholas D Steeves
On 19 April 2016 at 07:14, Austin S. Hemmelgarn  wrote:
> The closest I've ever seen for Debian to a Gentoo stage3 installation (the
> developers discourage usage of stage2 installs these days unless you're
> bootstrapping _everything_ yourself) is debbootstrap, and I could never get
> that to work reliably.

Fair, I remember Gentoo stage3+documentation was more straightforward
than debbootstrap+documentation.  There are a couple of wrappers
around debbootstrap and better documentation now, but I think there
are still rare times when bootstrapping from unstable or testing will
fail because a major transition is in progress (like the libc5 ->
libc6 transition, the ulibc -> glibc one, possibly the multiarch or
multilib transition, sometimes a GCC one, etc.)

> FWIW, the installer wasn't the only reason I switched to Gentoo, the two
> bigger ones for me were wanting newer software versions and needing
> different sets of features enabled on packages than the default builds, I
> ended up switching at the point that I was building more locally than I was
> installing from the repositories.

Ah yes, a convenient stream of fresh updates and the power of USE
flags :-)  For my needs, security-only updates + a backport whenever I
need a more up-to-date package of saves me time.  That's what the
choice of tool comes down to, right?  Does what you need it to, and
saves you time.

Best regards,
Nicholas
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Клиентские базы тел +79133913837 Skype: prodawez389 Email: ammanakuw-7...@yopmail.com

2016-04-19 Thread ammanakuw-7...@yopmail.com
Соберем для Вас по интернет базу данных потенциальных клиентов для Вашего 
Бизнеса.
По базе можно звонить, писать, слать факсы и email, 
вести любые прямые активные продажи Ваших товаров и услуг
Узнайте подробнее по 
тел +79133913837 (whatsapp,viber,telegram) 
Skype: prodawez389 
Email: ammanakuw-7...@yopmail.com


Re: Install to or Recover RAID Array Subvolume Root?

2016-04-19 Thread Nicholas D Steeves
On 18 April 2016 at 23:06, David Alcorn  wrote:
> Nicolas:
>
> My flash drive uses BTRFS and I am comfortable with your instructions
> with one exception.  What does "update /etc/default/grub" mean?
>
> Currently, I am waiting for a scrub to verify that all is in good
> order before fixing the problem.

I meant that more as a general precaution and good habit.  The most
common check/change would be to make sure the "resume=foo" option
matches the UUID or /dev/sdX of the swap partition; it's mostly
relevant to laptop users.

More to the point, as Austin and Chris mentioned the tricky bit is
going to get GRUB to boot from raid6 profile btrfs if your /boot is
part of your btrfs volume.  I honestly don't know if it will work...
Do you have a separate /boot partition?  What is your /dev/sda being
used for?  UEFI firmware loads GRUB's EFI payload, which loads the
different stages of grub that allow file system access, which is
necessary for grub to be able to find the kernel.  The EFI payload is
installed to your FAT-formatted ESP partition, which is usually
mounted to /boot/grub/efi/EFI.  I also suspect that without a separate
/boot partition GRUB won't be able to find the kernel
(/boot/vmlinuz-4.4.0-1-amd64).  If I remember correctly GRUB's stage1
talks to your motherboard's firmware, stage2 enables filesystem access
(/boot/grub/x86_64-efi/btrfs.mod), and stage3 loads the kernel.  En
bref, if GRUB has insufficient support for btrfs' raid6 profile then
grub will either be unable to access btrfs.mod, or btrfs.mod will be
unable to enable access /boot/vmlinuz-4.4.0-1-amd64.

I suspect the following worst-case scenario if you don't have a
partition you can use for /boot, and didn't leave any unallocated
space on any of your drives, and if you can't shrink something like a
swap partition to make room for /boot:  No need to backup/restore if
you have a usb port to dedicate to /boot.  A more exotic solution
would be using a small SATADOM to hold it, but then you lose a SATA
port ;-)  After sending the rootfs of your USB flash installation to a
subvolume of your raid6, you can manually use the GRUB command line on
your existing USB stick to attempt to boot the rootfs subvolume of
your raid6.

Cheers,
Nicholas
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: btrfs-image and btrfs send related queries

2016-04-19 Thread Henk Slager
> I have /dev/sdb , /dev/sdc. Using wipefs -fa I cleared both devices and
> created btrfs on /dev/sdb. Mounted and written some files and unmounted it.
>
> Then I ran btrfs-image /dev/sdc /img1.img and got the dump.

It looks like you imaged the wrong device, that might clarify the IO
errors later on

> Once image created I again ran wipefs -fa /dev/sdb.
>
> Then I ran btrfs-image -r /img1.img /dev/sdc and mounted /dev/sdc.
>
> ls to dumped filesystem shows the file size as original and no file content.
> I guess btrfs-image doesn't modify files stat so i feel it is showing size
> as original.
>
> However running cat on some of files give i/o error
>
> qd67:/btrfs1 # cat shadow.h
> cat: shadow.h: Input/output error
>
> These errors are not on all files on other files, since dump doesn't
> contains any data it just prompts for cat as below.
>
> qd67:/btrfs1 # cat stab.h
> qd67:/btrfs1 # cat stdio_ext.h
>
> Not sure why i/o errors are coming.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Install to or Recover RAID Array Subvolume Root?

2016-04-19 Thread Austin S. Hemmelgarn

On 2016-04-18 16:34, Nicholas D Steeves wrote:

On 18 April 2016 at 11:52, Austin S. Hemmelgarn  wrote:

On 2016-04-18 11:39, Chris Murphy wrote:


On Mon, Apr 18, 2016 at 9:15 AM, Austin S. Hemmelgarn
 wrote:

Like I said in one of my earlier e-mails though, these kind of limitations
are part of why I switched to Gentoo, there's no GUI installer, but you can
put the system together however the hell you want (which is especially nice
with BTRFS, because none of the installers out there will let you use BTRFS
on top of LVM, which is useful for things like BTRFS raid1 on top of
DM-RAID0).


Limitations?  ;-)  Debian has a variety of bootstrapping methods,
though I forget if they're more analogous to starting a Gentoo
installation from stage2 or stage3...I haven't used Gentoo since 2002.
Please consult the following doc for one of the methods:
https://www.debian.org/releases/stable/amd64/apds03.html.en
The closest I've ever seen for Debian to a Gentoo stage3 installation 
(the developers discourage usage of stage2 installs these days unless 
you're bootstrapping _everything_ yourself) is debbootstrap, and I could 
never get that to work reliably.


FWIW, the installer wasn't the only reason I switched to Gentoo, the two 
bigger ones for me were wanting newer software versions and needing 
different sets of features enabled on packages than the default builds, 
I ended up switching at the point that I was building more locally than 
I was installing from the repositories.

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: RAID5 Unable to remove Failing HD

2016-04-19 Thread Lionel Bouton
Hi,

Le 19/04/2016 11:13, Anand Jain a écrit :
>
>>> # btrfs device delete 3 /mnt/store/
>>> ERROR: device delete by id failed: Inappropriate ioctl for device
>>>
>>> Were the patch sets above for btrfs-progs or for the kernel ?
>> [...]
>
>  By the way, For Lionel issue, delete missing should work ?
>  which does not need any additional patch.

Delete missing works with 4.1.15 and btrfs-progs 4.5.1 (see later), but
the device can't be marked missing online so there's no way to maintain
redundancy without downtime. I was a little surprised: I half-expected
something like this because reading this list, RAID recovery seems to
still be a pain point but this isn't documented anywhere and after
looking around the relevant information seems to only be in this thread
(and many come from md and don't read this list, so won't expect this
behavior at all).
While I was waiting for directions the system crashed with a kernel
panic (clearly linked to IO errors according to the kernel panic but I
couldn't get all the stacktrace) and the system wasn't able to boot
properly (kernel panic shortly after the system mounted the filesystem
on each boot) until I removed the faulty drive (apparently it was
somehow readable enough to be recognized, but not enough to be usable).
After removing the faulty drive delete missing worked and a balance is
currently running (by the way it seems the drive bay was faulty: the
drive was not firmly fixed and it's cage could move a bit around in the
chassis and it was the only one, I didn't expect this and from
experience it's probably a factor in the hardware failure).

There may have been fixes since 4.1.15 to prevent the kernel panic
(there was only one device with IO errors, so ideally it shouldn't be
able to bring down the kernel) so it may not be worth further analysis.
That said I'll have 2 new drives next week (one replacement, one spare)
and I have a chassis lying around where I could try to replicate
failures with various kernels on a RAID1 filesystem built with a brand
new drive and the faulty drive (until the faulty drive completely dies
which they usually do in my experience) so if someone wants some tests
done with 4.6-rcX or even 4.6-rcX + patches I can spend some time on it
next week.

Lionel
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: RAID5 Unable to remove Failing HD

2016-04-19 Thread Duncan
Anand Jain posted on Tue, 19 Apr 2016 17:13:04 +0800 as excerpted:

>>> # btrfs device delete 3 /mnt/store/
>>> ERROR: device delete by id failed: Inappropriate ioctl for device
>>>
>>> Were the patch sets above for btrfs-progs or for the kernel ?
>>
>> Looks like you're primarily interested in the concern2 patches, device
>> delete by devid.
>>
>> A quick search of the list back-history reveals that an updated patch
>> set, 00/15 now (look for [PATCH 00/15] Device delete by id ), was
>> posted by dserba on the 15th of February.  It was the kernel patches
>> and was slated for the kernel 4.6 dev cycle.  However, the patch set
>> was pulled at that time due to test failures, tho they were suspected
>> to actually be from something else.
> 
>   Thanks Duncan. Yep the reported issue did not point to any of the
>   patch in that set. But I am keeping a tab open for new users /test
>   cases, anything that is found will help.
> 
>   By the way, For Lionel issue, delete missing should work ?
>   which does not need any additional patch.

Were the issues with btrfs delete missing fixed?  There were some issues 
with it actually trying to delete a device literally named "missing" or 
something the like, and of course not finding it to delete, a kernel 
cycle or two ago, and AFAIK delete by ID was originally proposed as a fix 
for that.  If I was reading the comments correctly, however, I think the 
problem there was introduced with the switch to libblockdev or some such, 
however, so he might be able to get around that by using older releases, 
as long as the filesystem isn't using newer features that would block 
that.

So delete missing may or may not work now, I've lost track.  But he was 
reluctant to unmount and reboot, and of course with btrfs not yet 
offlining failed devices, it doesn't know it's actually missing, yet.  So 
even if delete missing does work for him, it's not going to work until a 
reboot and remount degraded, and he was hoping to avoid that with the 
delete by ID.




-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: btrfs-image and btrfs send related queries

2016-04-19 Thread sri
Henk Slager  gmail.com> writes:

> 
> On Mon, Apr 18, 2016 at 4:26 PM, Roman Mamedov  romanrm.net> 
wrote:
> > On Mon, 18 Apr 2016 16:13:28 +0200
> > Henk Slager  gmail.com> wrote:
> >
> >> (your email keeps ending up in gmail spam folder)
> >>
> >> On Mon, Apr 18, 2016 at 9:24 AM, sri  
yahoo.co.in> wrote:
> >> > I tried btrfs-image and created image file and ran btrfs-image -r 
to a
> >> > different disk. Once recovered and mounted, I can able to see 
data is
> >> > not zeroed out as mentioned in btrfs-image man page.
> >>
> >> "different disk"  you mention, that is important info. If you doe 
the
> >> restore to a image file, that image file is sparse and all data 
blocks
> >> are read as zeros.
> >>
> >> However, if you restore to a block device, then you can assume it 
just
> >> writes the device blocks for metadata and leaves the rest 
untouched.
> >> So trim whole device first or brute-force overwrite completely with
> >> zeros.
> >>
> >> So maybe the man pages needs some correction / extra notes.
> >>
> >> > I tried on same machine.
> >
> > Does btrfs-image store/restore the FS UUID? If it does, then 
potentially both
> > the source FS and the restored one were visible at the same time to 
the kernel
> > with identical UUIDs, and maybe it was actually accessing/mounting 
the source
> > one.
> 
> Very good point! The UUID's are the same. I remember I used a VM to
> separate the source FS from the restored FS
> 
> Also, the assumption I made about restoring to a block device is not
> correct: If you restore to a loopdev that has a file with all
> non-zeros as backing-store, the files in the mounted restored FS are
> read as zeros.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" 
in
> the body of a message to majordomo  vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 
thank you for inputs. Actually I have tried in the following way.
I have /dev/sdb , /dev/sdc. Using wipefs -fa I cleared both devices and 
cre=
ated btrfs on /dev/sdb. Mounted and written some files and unmounted it.
Then I ran btrfs-image /dev/sdc /img1.img and got the dump.
Once image created I again ran wipefs -fa /dev/sdb.
Then I ran btrfs-image -r /img1.img /dev/sdc and mounted /dev/sdc.
ls to dumped filesystem shows the file size as original and no file 
content=
. I guess btrfs-image doesn't modify files stat so i feel it is showing 
siz=
e as original.
However running cat on some of files give i/o error
qd67:/btrfs1 # cat shadow.hcat: shadow.h: Input/output error
These errors are not on all files on other files, since dump doesn't 
contai=
ns any data it just prompts for cat as below.
qd67:/btrfs1 # cat stab.hqd67:/btrfs1 # cat stdio_ext.h
Not sure why i/o errors are coming.



--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: RAID5 Unable to remove Failing HD

2016-04-19 Thread Anand Jain



# btrfs device delete 3 /mnt/store/
ERROR: device delete by id failed: Inappropriate ioctl for device

Were the patch sets above for btrfs-progs or for the kernel ?


Looks like you're primarily interested in the concern2 patches, device
delete by devid.

A quick search of the list back-history reveals that an updated patch
set, 00/15 now (look for [PATCH 00/15] Device delete by id ), was posted
by dserba on the 15th of February.  It was the kernel patches and was
slated for the kernel 4.6 dev cycle.  However, the patch set was pulled
at that time due to test failures, tho they were suspected to actually be
from something else.


 Thanks Duncan. Yep the reported issue did not point to any
 of the patch in that set. But I am keeping a tab open for
 new users /test cases, anything that is found will help.

 By the way, For Lionel issue, delete missing should work ?
 which does not need any additional patch.

Thanks, Anand

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [resend] btrfs-send -c fails: reproduction case

2016-04-19 Thread Duncan
Zachary Vance posted on Sun, 17 Apr 2016 18:46:13 -0700 as excerpted:

> I was already making sure all -c references were both present and
> unmodified, I think the confusion is mostly around whether the parent
> required to use -c, and whether it's an implicit reference volume in
> particular. If it's required, it's impossible to preserve de-duplication
> after deleting the original parent which would be really bad.

AFAIK, send -c does require a parent, but if there's clones and no parent 
given, it will try to pick its own from the list of clones given.

Which explains the error complaining about being unable to find a parent, 
when given only clone sources that (based on your reproducer) had nothing 
in common with the snapshots listed as clones.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Kernel crash if both devices in raid1 are failing

2016-04-19 Thread Duncan
Dmitry Katsubo posted on Tue, 19 Apr 2016 07:45:40 +0200 as excerpted:

> Actually btrfs restore has recovered many files, however I was not able
> to run in fully unattended mode as it complains about "looping a lot".
> Does it mean that files are corrupted / not correctly restored?

As long as you tell it to keep going each time, the loop complaints 
shouldn't be an issue.  The problem is that the loop counter is measuring 
loops on a particular directory, because that's what it has available to 
measure.  But if you had a whole bunch of files in that dir, it's /going/ 
to loop a lot, to restore all of them.

I have one cache directory with over 200K files in it.  They're all text 
messages from various technical lists and newsgroups (like this list, 
which I view as a newsgroup using gmane.org's list2news service) so 
they're quite small, about 5 KiB on average by my quick calculation, but 
that's still a LOT of files for a single dir, even if they're only using 
just over a GiB of space.

I ended up doing a btrfs restore on that filesystem (/home), because 
while I had a backup, restore was getting more recent copies of stuff 
back, and that dir looped a *LOT* the first time it happened, now several 
years ago, before they actually added the always option.

The second time it happened, about a year ago, restore worked much 
better, and I was able to use the always option.  But AFAIK, always only 
applies to that dir.  If you have multiple dirs with the problem, you'll 
still get asked for the next one.  But it did vastly improve the 
situation for me, giving me only a handful of prompts instead of the very 
many I had before the option was there.

(The main problem triggering the need to run restore for me, turned out 
to be hardware.  I've had no issues since I replaced that failing ssd, 
and with a bit of luck, won't be running restore again for a few years, 
now.)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: RAID5 Unable to remove Failing HD

2016-04-19 Thread Duncan
Lionel Bouton posted on Mon, 18 Apr 2016 10:59:35 +0200 as excerpted:

> Hi,
> 
> Le 10/02/2016 10:00, Anand Jain a écrit :
>>
>> Thanks for the report. Fixes are in the following patch sets
>>
>>  concern1:
>>  Btrfs to fail/offline a device for write/flush error:
>>[PATCH 00/15] btrfs: Hot spare and Auto replace
>>
>>  concern2:
>>  User should be able to delete a device when device has failed:
>>[PATCH 0/7] Introduce device delete by devid
>>
>>  If you were able to tryout these patches, pls lets know.
> 
> Just found out this thread after digging for a problem similar to mine.
> 
> I just got the same error when trying to delete a failed hard drive on a
> RAID1 filesystem with a total of 4 devices.
> 
> # btrfs device delete 3 /mnt/store/
> ERROR: device delete by id failed: Inappropriate ioctl for device
> 
> Were the patch sets above for btrfs-progs or for the kernel ?

Looks like you're primarily interested in the concern2 patches, device 
delete by devid.

A quick search of the list back-history reveals that an updated patch 
set, 00/15 now (look for [PATCH 00/15] Device delete by id ), was posted 
by dserba on the 15th of February.  It was the kernel patches and was 
slated for the kernel 4.6 dev cycle.  However, the patch set was pulled 
at that time due to test failures, tho they were suspected to actually be 
from something else.

I haven't updated to kernel 4.6 git yet (tho I'm on 4.5 and generally do 
run git post rc4 or so, which was just released), so I'll probably update 
shortly) so can't check whether it ultimately made it in or not, but if 
it's not in 4.6 it certainly won't be in anything earlier as stable 
patches must be in devel mainline first.

So I'd say check 4.6 devel, and if it's not there as appears to be 
likely, you'll have to grab the patches off the list and apply them 
yourself.

> Currently the kernel is 4.1.15-r1 from Gentoo. I used btrfs-progs-4.3.1
> (the Gentoo stable version) but it didn't support delete by devid so I
> upgraded to btrfs-progs-4.5.1 which supports it but got the same
> "inappropriate ioctl for device" error when I used the devid.

FWIW, I'm a gentooer also, but I'm on ~amd64 not stable, and as I said I 
run current stable and later devel kernels.  I also often update the 
(often unfortunately lagging, even on ~arch) btrfs-progs ebuild to the 
latest as announced here and normally run that.

And FWIW I run btrfs raid1 mode also, but on only two ssds, which 
decomplicates things since btrfs raid1 is only 2-way-mirroring anyway.  I 
also partition up the ssds and run multiple independent btrfs, the 
largest only 24 GiB usable size (24 GiB partitions on two devices, 
raid1), so my data eggs aren't all in one btrfs basket and it's easier to 
recover from just one btrfs failing.  As an example, my / is only 8 GiB 
and contains everything installed by portage but a few bits of /var which 
need to be writable at runtime, because I keep my / mounted read-only by 
default, only mounting it writable to update.  An 8 GiB root is easy to 
duplicate elsewhere for backup, and indeed, my first backup is another 
set of 8 GiB partitions in btrfs raid1, on the same ssds, with the second 
backup being an 8 GiB reiserfs partition on spinning rust, with all three 
bootable from grub (separately installed to each of the three physical 
devices, each of which has its own /boot, with the one that's booted 
selected from grub), should it be needed.

> I don't have any drive available right now for replacing this one (so no
> btrfs dev replace possible right now). The filesystem's data could fit
> on only 2 of the 4 drives (in fact I just added 2 old drives that were
> previously used with md and rebalanced, which is most probably what
> triggered one of the new drives failure). So I can't use replace and
> would prefer not to lose redundancy while waiting for new drives to get
> there.

I did have to use btrfs replace for one of the ssds, but as it happens I 
did have a spare, as the old netbook I intended to put it in died before 
I got it installed.  And the failing ssd wasn't entirely failed, just 
needing more and more frequent scrubs as sectors failed, and the replace 
(replaces as I have multiple btrfs on the pair of ssds) went quite 
well... and fast on the ssds. =:^) 

> So the obvious thing to do in this circumstance is to delete the drive,
> forcing the filesystem to create the missing replicas in the process and
> only reboot if needed (no hotplug). Unfortunately I'm not sure of the
> conditions where this is possible (which kernel version supports this if
> any ?). If there is a minimum kernel version where device delete works,
> can https://btrfs.wiki.kernel.org/index.php/Gotchas be updated ? I don't
> have a wiki account yet but I'm willing to do it myself if I can get
> reliable information.

As I said, it'd be 4.6 if it's even there.  Otherwise you'll have to 
apply the patches yourself.

> I can reboot this system and I expect the current drive to a