Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.
On Wed, Jan 21, 2015 at 03:47:54PM +0800, Qu Wenruo wrote: To David: I'm a little curious about why inode_cache needs to be delayed to next transaction. In btrfs_remount() we have s_umount mutex, and we synced the whole filesystem already, so there should be no running transaction and we can just set any mount option into fs_info. See our discussion under the noinode_cache option: http://www.mail-archive.com/linux-btrfs%40vger.kernel.org/msg30075.html http://www.mail-archive.com/linux-btrfs%40vger.kernel.org/msg30414.html What do you think about reverting the whole patchset and rework the sysfs interface? IMO reverting should be the last option, we have a minimal fix to the sync deadlock and you've proposed the per-trasaction mount options to replace the pending inode_change. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs: Don't call btrfs_start_transaction() on frozen fs to avoid deadlock.
On Wed, Jan 21, 2015 at 03:04:02PM +0800, Miao Xie wrote: Pending changes are *not* only mount options. Feature change and label change are also pending changes if using sysfs. My miss, I don't notice feature and label change by sysfs. But the implementation of feature and label change by sysfs is wrong, we can not change them without write permission. Label change does not happen if the fs is readonly. If the filesystem is RW and label is changed through sysfs, then remount to RO will sync the filesystem and the new label will be saved. The sysfs features write handler is missing that protection though, I'll send a patch. For freeze, it's not the same problem since the fs will be unfreeze sooner or later and transaction will be initiated. You can not assume the operations of the users, they might freeze the fs and then shutdown the machine. The semantics of freezing should make the on-device image consistent, but still keep some changes in memory. For example, if we change the features/label through sysfs, and then umount the fs, It is different from pending change. No, now features/label changing using sysfs both use pending changes to do the commit. See BTRFS_PENDING_COMMIT bit. So freeze - change features/label - sync will still cause the deadlock in the same way, and you can try it yourself. As I said above, the implementation of sysfs feature and label change is wrong, it is better to separate them from the pending mount option change, make the sysfs feature and label change be done in the context of transaction after getting the write permission. If so, we needn't do anything special when sync the fs. That would mean to drop the write support of sysfs files that change global filesystem state (label and features right now). This would leave only the ioctl way to do that. I'd like to keep the sysfs write support though for ease of use from scripts and languages not ioctl-friendly. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 3.19-rc] btrfs: add read-only check to sysfs handler of features
We don't want to trigger the change on a read-only filesystem, similar to what the label handler does. Signed-off-by: David Sterba dste...@suse.cz --- This emerged during the discussions about pending changes problems. fs/btrfs/sysfs.c | 3 +++ 1 file changed, 3 insertions(+) diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c index 92db3f648df4..00b9c46e351c 100644 --- a/fs/btrfs/sysfs.c +++ b/fs/btrfs/sysfs.c @@ -119,6 +119,9 @@ static ssize_t btrfs_feature_attr_store(struct kobject *kobj, if (!fs_info) return -EPERM; + if (fs_info-sb-s_flags MS_RDONLY) + return -EROFS; + ret = kstrtoul(skip_spaces(buf), 0, val); if (ret) return ret; -- 2.1.3 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/1] Btrfs: move super_kobj and device_dir_kobj from fs_info to btrfs_fs_devices
On Tue, Jan 20, 2015 at 12:02:27AM +0800, Anand Jain wrote: This patch will provide a framework and help to create attributes from the structure btrfs_fs_devices which are available even before fs_info is created. So by moving the parent kobject super_kobj from fs_info to btrfs_fs_devices, it will help to create attributes from the btrfs_fs_devices as well. Ok. Just to note, this does not change any of the existing btrfs sysfs external kobject names and its attributes and not even the life cycle of them. Changes are internal only. And to ensure the same, this path has been tested with various device operations and, checking and comparing the sysfs kobjects and attributes with sysfs kobject and attributes with out this patch, and they remain same. Good. Reviewed-by: David Sterba dste...@suse.cz -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 2/2 RESEND] btrfs: introduce shrinker for rb_tree that keeps valid btrfs_devices
On Thu, Jan 15, 2015 at 04:53:08PM +0800, Gui Hecheng wrote: The following patch: btrfs: remove empty fs_devices to prevent memory runout introduces @valid_dev_root aiming at recording @btrfs_device objects that have corresponding block devices with btrfs. But if a block device is broken or unplugged, no one tells the @valid_dev_root to cleanup the dead objects. To recycle the memory occuppied by those deads, we could rely on the shrinker. The shrinker's scan function will traverse the @valid_dev_root and trys to open the devices one by one, if it fails or encounters a non-btrfs it will remove the dead @btrfs_device. I don't see why shrinker is used here. linux.git/linux/shrinker.h: A callback you can register to apply pressure to ageable caches. How is guaranteed that it will take action at the right time? -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Moving snapshots
On Fri, Jan 23, 2015 at 5:05 AM, Matthias Urlichs matth...@urlichs.de wrote: Hello, how do I move a (read-only) snapshot? If you want to move a read-only snapshot to a different directory, '..' changes, and therefore is not a read-only operation. Simply creating another read-only snap from the first one and then deleting the source works, except that it destroy's the snapshot's identity -- which means that it can't be used as a parent for btrfs receive any more. :-( # btrfs property set -ts /path/to/snapshot ro false # mv /path/to/snapshot /new/path # btrfs property set -ts /new/path ro true worked for me -- -- Matthias Urlichs -- Noah -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4] btrfs: add regression test for remount with thread_pool resized
On Fri, Jan 23, 2015 at 08:36:37PM +0800, Eryu Guan wrote:
On Fri, Jan 23, 2015 at 03:28:59PM +0800, Xing Gu wrote:
Regression test for a btrfs issue of resizing 'thread_pool' when
remount the fs.
This regression was introduced by the following linux kernel commit:
btrfs: Added btrfs_workqueue_struct implemented ordered
execution based on kernel workqueue
08a9ff3264181986d1d692a4e6fce3669700c9f8
And it was fixed by the following linux kernel commit:
btrfs: fix crash in remount(thread_pool=) case
800ee2247f483b6d05ed47ef3bbc90b56451746c
Signed-off-by: Xing Gu gux.f...@cn.fujitsu.com
---
v3-v4: Remove the check for a bug message.
v2-v3: Add the regresssion commit in description.
Remove dmesg -c /dev/null.
Echo Silence is golden to indicate that an empty output
file is expected.
v1-v2: Add the fix commit in description.
Add the check for a bug message.
---
tests/btrfs/082 | 65
+
tests/btrfs/082.out | 2 ++
tests/btrfs/group | 1 +
3 files changed, 68 insertions(+)
create mode 100755 tests/btrfs/082
create mode 100644 tests/btrfs/082.out
diff --git a/tests/btrfs/082 b/tests/btrfs/082
new file mode 100755
index 000..7a6e4b5
--- /dev/null
+++ b/tests/btrfs/082
@@ -0,0 +1,65 @@
+#!/bin/bash
+# FS QA Test No. btrfs/082
+#
+# Regression test for a btrfs issue of resizing 'thread_pool' when
+# remount the fs.
+#
+# This regression was introduced by the following linux kernel commit:
+# btrfs: Added btrfs_workqueue_struct implemented ordered
+# execution based on kernel workqueue
+# 08a9ff3264181986d1d692a4e6fce3669700c9f8
+# And it was fixed by the following linux kernel commit:
+# btrfs: fix crash in remount(thread_pool=) case
+# 800ee2247f483b6d05ed47ef3bbc90b56451746c
+#
+#---
+# Copyright (c) 2015 Fujitsu. All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#
+#---
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo QA output created by $seq
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+
+_cleanup()
+{
+rm -f $tmp.*
+}
+
+trap _cleanup ; exit \$status 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs /dev/null
Redirect stderr to /dev/null too, otherwise test fails if scratch device
supports trim.
btrfs/082 0s ... - output mismatch (see
/root/xfstests/results//btrfs/082.out.bad)
--- tests/btrfs/082.out 2015-01-23 20:27:15.603689913 +0800
+++ /root/xfstests/results//btrfs/082.out.bad 2015-01-23
20:33:46.514259626 +0800
@@ -1,2 +1,3 @@
QA output created by 082
+Performing full device TRIM (5.00GiB) ...
Silence is golden
I think the internal btrfs mkfs call needs to filter that so that
error messages are still caught by the test.
In fact, I'll go as far as to suggest that outputting operational
status information on stderr is a bug/regression in mkfs.btrfs.
Cheers,
Dave.
--
Dave Chinner
da...@fromorbit.com
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 3.19-rc5: Bug 91911: [REGRESSION] rm command hangs big time with deleting a lot of files at once
On Fri, Jan 23, 2015 at 03:01:28PM +0100, Martin Steigerwald wrote: Hi! Anyone seen this? Reported as: https://bugzilla.kernel.org/show_bug.cgi?id=91911 I have seen something like this since 3.15. I've also seen its cousin, which gets stuck in evict_inode, but the stacks of the hanging processes start from renameat2() instead of unlinkat(). I haven't seen the renameat2() variant of this bug since 3.18-rc6. I just want to get rid of some 127000+ akonadi lost+found files, any delete command I start just gets rid of some thousands and then hangs. merkaba:~ btrfs fi df /home Data, RAID1: total=160.92GiB, used=111.09GiB System, RAID1: total=32.00MiB, used=48.00KiB Metadata, RAID1: total=5.99GiB, used=2.49GiB GlobalReserve, single: total=512.00MiB, used=0.00B merkaba:~ btrfs fi sh /home Label: 'home' uuid: […] Total devices 2 FS bytes used 113.58GiB devid1 size 170.00GiB used 166.94GiB path /dev/mapper/msata- home devid2 size 170.00GiB used 166.94GiB path /dev/mapper/sata- home Btrfs v3.18 merkaba:/home/ms/.local/share/akonadi#1 find file_lost+found | wc -l 110070 merkaba:/home/ms/.local/share/akonadi find file_lost+found -delete [4] 2660 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 101645 merkaba:/home/ms/.local/share/akonadi find file_lost+found -delete [5] 2663 merkaba:/home/ms/.local/share/akonadi find file_lost+found -delete [6] 2664 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 91369 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 89844 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 88042 merkaba:/home/ms/.local/share/akonadi find file_lost+found -delete [7] 2671 merkaba:/home/ms/.local/share/akonadi uname -a Linux merkaba 3.19.0-rc5-tp520-trim-all-bgroups+ #18 SMP PREEMPT Mon Jan 19 09:58:33 CET 2015 x86_64 GNU/Linux merkaba:/home/ms/.local/share/akonadi find file_lost+found -delete [8] 2694 merkaba:/home/ms/.local/share/akonadi find file_lost+found -delete [9] 2700 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 67278 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 65244 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 63713 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 62725 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 62213 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 61213 merkaba:/home/ms/.local/share/akonadi find file_lost+found -delete [10] 2715 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 60470 merkaba:/home/ms/.local/share/akonadi find file_lost+found -delete [11] 2718 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 53303 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 51396 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 51396 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 51396 merkaba:/home/ms/.local/share/akonadi ps aux | grep find ms2647 0.4 0.2 43096 36204 pts/3D+ 14:45 0:00 find file_lost+found -delete root 2651 0.3 0.2 42568 35688 pts/0DN 14:45 0:00 find file_lost+found -delete root 2654 2.7 0.2 44544 35652 pts/0DN 14:46 0:05 find file_lost+found -delete root 2657 0.3 0.2 44016 35048 pts/0DN 14:46 0:00 find file_lost+found -delete root 2660 2.1 0.1 39136 32280 pts/0DN 14:46 0:03 find file_lost+found -delete root 2663 0.2 0.1 36760 29988 pts/0DN 14:46 0:00 find file_lost+found -delete root 2664 3.3 0.1 36760 29888 pts/0DN 14:46 0:05 find file_lost+found -delete root 2671 0.9 0.1 33856 26984 pts/0DN 14:46 0:01 find file_lost+found -delete root 2694 1.1 0.1 32404 25380 pts/0DN 14:47 0:01 find file_lost+found -delete root 2700 4.0 0.1 30952 24064 pts/0DN 14:47 0:04 find file_lost+found -delete root 2715 0.3 0.1 26200 19332 pts/0DN 14:47 0:00 find file_lost+found -delete root 2718 4.1 0.1 26068 19068 pts/0DN 14:47 0:02 find file_lost+found -delete root 2840 0.0 0.0 12672 1592 pts/0S+ 14:49 0:00 grep find merkaba:/home/ms/.local/share/akonadi ps aux | grep rm root 113 0.0 0.0 0 0 ?S 14:41 0:00 [acpi_thermal_pm] root 290 0.0 0.0 0 0 ?S 14:41 0:00 [btrfs- rmw] root 803 0.0 0.0 0 0 ?S 14:41 0:00 [btrfs- rmw] root 835 0.0 0.0 0 0 ?S 14:41 0:00 [btrfs- rmw] dirmngr 1565 0.0 0.0 17880 2192 ?Ss 14:41 0:00 /usr/bin/dirmngr --daemon --sh ms2615 0.4 0.2 36276 32432 pts/1D+ 14:43 0:01 rm -r file_lost+found root 2842 0.0 0.0 12676 1544
3.19-rc5: Bug 91911: [REGRESSION] rm command hangs big time with deleting a lot of files at once
Hi! Anyone seen this? Reported as: https://bugzilla.kernel.org/show_bug.cgi?id=91911 I just want to get rid of some 127000+ akonadi lost+found files, any delete command I start just gets rid of some thousands and then hangs. merkaba:~ btrfs fi df /home Data, RAID1: total=160.92GiB, used=111.09GiB System, RAID1: total=32.00MiB, used=48.00KiB Metadata, RAID1: total=5.99GiB, used=2.49GiB GlobalReserve, single: total=512.00MiB, used=0.00B merkaba:~ btrfs fi sh /home Label: 'home' uuid: […] Total devices 2 FS bytes used 113.58GiB devid1 size 170.00GiB used 166.94GiB path /dev/mapper/msata- home devid2 size 170.00GiB used 166.94GiB path /dev/mapper/sata- home Btrfs v3.18 merkaba:/home/ms/.local/share/akonadi#1 find file_lost+found | wc -l 110070 merkaba:/home/ms/.local/share/akonadi find file_lost+found -delete [4] 2660 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 101645 merkaba:/home/ms/.local/share/akonadi find file_lost+found -delete [5] 2663 merkaba:/home/ms/.local/share/akonadi find file_lost+found -delete [6] 2664 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 91369 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 89844 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 88042 merkaba:/home/ms/.local/share/akonadi find file_lost+found -delete [7] 2671 merkaba:/home/ms/.local/share/akonadi uname -a Linux merkaba 3.19.0-rc5-tp520-trim-all-bgroups+ #18 SMP PREEMPT Mon Jan 19 09:58:33 CET 2015 x86_64 GNU/Linux merkaba:/home/ms/.local/share/akonadi find file_lost+found -delete [8] 2694 merkaba:/home/ms/.local/share/akonadi find file_lost+found -delete [9] 2700 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 67278 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 65244 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 63713 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 62725 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 62213 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 61213 merkaba:/home/ms/.local/share/akonadi find file_lost+found -delete [10] 2715 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 60470 merkaba:/home/ms/.local/share/akonadi find file_lost+found -delete [11] 2718 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 53303 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 51396 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 51396 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 51396 merkaba:/home/ms/.local/share/akonadi ps aux | grep find ms2647 0.4 0.2 43096 36204 pts/3D+ 14:45 0:00 find file_lost+found -delete root 2651 0.3 0.2 42568 35688 pts/0DN 14:45 0:00 find file_lost+found -delete root 2654 2.7 0.2 44544 35652 pts/0DN 14:46 0:05 find file_lost+found -delete root 2657 0.3 0.2 44016 35048 pts/0DN 14:46 0:00 find file_lost+found -delete root 2660 2.1 0.1 39136 32280 pts/0DN 14:46 0:03 find file_lost+found -delete root 2663 0.2 0.1 36760 29988 pts/0DN 14:46 0:00 find file_lost+found -delete root 2664 3.3 0.1 36760 29888 pts/0DN 14:46 0:05 find file_lost+found -delete root 2671 0.9 0.1 33856 26984 pts/0DN 14:46 0:01 find file_lost+found -delete root 2694 1.1 0.1 32404 25380 pts/0DN 14:47 0:01 find file_lost+found -delete root 2700 4.0 0.1 30952 24064 pts/0DN 14:47 0:04 find file_lost+found -delete root 2715 0.3 0.1 26200 19332 pts/0DN 14:47 0:00 find file_lost+found -delete root 2718 4.1 0.1 26068 19068 pts/0DN 14:47 0:02 find file_lost+found -delete root 2840 0.0 0.0 12672 1592 pts/0S+ 14:49 0:00 grep find merkaba:/home/ms/.local/share/akonadi ps aux | grep rm root 113 0.0 0.0 0 0 ?S 14:41 0:00 [acpi_thermal_pm] root 290 0.0 0.0 0 0 ?S 14:41 0:00 [btrfs- rmw] root 803 0.0 0.0 0 0 ?S 14:41 0:00 [btrfs- rmw] root 835 0.0 0.0 0 0 ?S 14:41 0:00 [btrfs- rmw] dirmngr 1565 0.0 0.0 17880 2192 ?Ss 14:41 0:00 /usr/bin/dirmngr --daemon --sh ms2615 0.4 0.2 36276 32432 pts/1D+ 14:43 0:01 rm -r file_lost+found root 2842 0.0 0.0 12676 1544 pts/0S+ 14:49 0:00 grep rm Until it eventually completes: merkaba:/home/ms/.local/share/akonadi find file_lost+found -delete [20] 2858 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 8998 merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l 240 merkaba:/home/ms/.local/share/akonadi [20] + done find file_lost+found -delete merkaba:/home/ms/.local/share/akonadi find file_lost+found | wc -l
Re: btrfs convert running out of space
Marc Joliet posted on Fri, 23 Jan 2015 08:54:41 +0100 as excerpted: Am Fri, 23 Jan 2015 04:34:19 + (UTC) schrieb Duncan 1i5t5.dun...@cox.net: Gareth Pye posted on Fri, 23 Jan 2015 08:58:08 +1100 as excerpted: What are the chances that splitting all the large files up into sub gig pieces, finish convert, then recombine them all will work? [...] Option 2: Since new files should be created using the desired target mode (raid1 IIRC), you may actually be able to move them off and immediately back on, so they appear as new files and thus get created in the desired mode. With current coreutils, wouldn't that also work if he moves the files to another (temporary) subvolume? (And with future coreutils, by copying the files without using reflinks and then removing the originals.) If done correctly, yes. However, off the filesystem is far simpler to explain over email or the like, and is much less ambiguous in terms of OK, but did you do it 'correctly' if it doesn't end up helping. If it doesn't work, it doesn't work. If move to a different subvolume under specific conditions in terms of reflinking and the like doesn't work, there's always the question of whether it /really/ didn't work, or if somehow the instructions weren't clear enough and thus failure was simply the result of a failure to fully meet the technical requirements. Of course if I was doing it myself, and if I was absolutely sure of the technical details in terms of what command I had to use to be /sure/ it didn't simply reflink and thus defeat the whole exercise, I'd likely use the shortcut. But in reality, if it didn't work I'd be second-guessing myself and would probably move everything entirely off and back on to be sure, and knowing that, I'd probably do it the /sure/ way in the first place, avoiding the chance of having to redo it to prove to myself that I'd done it correctly. Of course, having demonstrated to myself that it worked, if I ever had the problem again, I might try the shortcut, just to demonstrate to my own satisfaction the full theory that the effect of the shortcut was the same as the effect of doing it the longer and more fool-proof way. But of course I'd rather not have the opportunity to try that second-half proof. =:^) Make sense? =:^) -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a master -- and if you use the program, he is your master. Richard Stallman -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] Btrfs: Add code to support file creation time
On Thu, Jan 22, 2015 at 12:01:10PM -0800, Zach Brown wrote: @@ -5729,6 +5744,7 @@ static struct inode *btrfs_new_inode(struct btrfs_trans_handle *trans, struct btrfs_path *path; struct btrfs_inode_ref *ref; struct btrfs_key key[2]; + struct timespec current_time; u32 sizes[2]; int nitems = name ? 2 : 1; unsigned long ptr; @@ -5824,7 +5840,13 @@ static struct inode *btrfs_new_inode(struct btrfs_trans_handle *trans, inode_init_owner(inode, dir, mode); inode_set_bytes(inode, 0); - inode-i_mtime = inode-i_atime = inode-i_ctime = CURRENT_TIME; + + current_time = CURRENT_TIME; + inode-i_mtime = current_time; + inode-i_atime = current_time; + inode-i_ctime = current_time; + BTRFS_I(inode)-i_otime = current_time; I might have just set mtime to CURRENT_TIME then the rest to mtime. It'd be a touch less noisy. Author's choice, though :). Ok, less churn sounds better. I'll do a v2 anyway because I left a fixup uncommitted (use of btrfs_inode_otime). Any plans to add it to send/recv? 766702ef (Alexander Block 2012-07-28 14:11:31 +0200 2480) /* TODO Add otime support when the otime patches get into upstream */ This is part of the send protocol update, otime and other changes. I don't remember how many pieces are still missing to do the version bump. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Recovery Operation With Multiple Devices
On Fri, Jan 23, 2015 at 06:53:42PM +1100, Brett King wrote: Hi All, Just wondering how 'btrfs recovery' operates, when the source device given is one of many in an MD array - I can't find anything documentation beyond a single device use case. Does it automatically include all devices in the relevant MD array as occurs when mounting, or does it only restore the data which happened to be written to the specific, single device given ? Neither. :) It automatically includes all devices as occurs when running btrfsck. OK, it's a relatively pointless distinction, but the mechanisms are slightly different. From an inverse perspective, how can I restore all data including snapshots, which are spread across a damaged MD FS to a new (MD) FS ? btrfs restore -l will show you the list of tree roots, which (for trees numbered 256 and up) correspond to subvolumes and snapshots. You can then use -r to select the tree to restore. Can send / receive do this perhaps ? Yes, but only if you can mount the FS. Hugo. -- Hugo Mills | I know of three kinds: hot, cool, and hugo@... carfax.org.uk | what-time-does-the-tune-start? http://carfax.org.uk/ | PGP: 65E74AC0 | Chris Dollin signature.asc Description: Digital signature
[PATCH RFC v3 4/5] btrfs: Use btrfs_test_trans_opt() to handle SPACE_CACHE if it's under transaction protect.
Convert btrfs_test_opt() to btrfs_test_trans_opt() if it's called under
transaction protection.
This will ensure SPACE_CACHE bit is consistent during transaction.
Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com
---
fs/btrfs/extent-tree.c | 2 +-
fs/btrfs/transaction.c | 7 ---
2 files changed, 5 insertions(+), 4 deletions(-)
diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
index 1511658..c968d12 100644
--- a/fs/btrfs/extent-tree.c
+++ b/fs/btrfs/extent-tree.c
@@ -3260,7 +3260,7 @@ again:
spin_lock(block_group-lock);
if (block_group-cached != BTRFS_CACHE_FINISHED ||
- !btrfs_test_opt(root, SPACE_CACHE) ||
+ !btrfs_test_trans_opt(trans, SPACE_CACHE) ||
block_group-delalloc_bytes) {
/*
* don't bother trying to write stuff out _if_
diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 846e1b8..aec5a5a 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -1507,7 +1507,8 @@ static noinline int create_pending_snapshots(struct
btrfs_trans_handle *trans,
return ret;
}
-static void update_super_roots(struct btrfs_root *root)
+static void update_super_roots(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root)
{
struct btrfs_root_item *root_item;
struct btrfs_super_block *super;
@@ -1523,7 +1524,7 @@ static void update_super_roots(struct btrfs_root *root)
super-root = root_item-bytenr;
super-generation = root_item-generation;
super-root_level = root_item-level;
- if (btrfs_test_opt(root, SPACE_CACHE))
+ if (btrfs_test_trans_opt(trans, SPACE_CACHE))
super-cache_generation = root_item-generation;
if (root-fs_info-update_uuid_tree_gen)
super-uuid_tree_generation = root_item-generation;
@@ -1987,7 +1988,7 @@ int btrfs_commit_transaction(struct btrfs_trans_handle
*trans,
switch_commit_roots(cur_trans, root-fs_info);
assert_qgroups_uptodate(trans);
- update_super_roots(root);
+ update_super_roots(trans, root);
btrfs_set_super_log_root(root-fs_info-super_copy, 0);
btrfs_set_super_log_root_level(root-fs_info-super_copy, 0);
--
2.2.2
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH RFC v3 1/5] Revert btrfs: add support for processing pending changes related commits
This reverts commit 572d9ab7845 ~ a6f69dc8018.
This pending commits patches introduce deadlock with freeze, and fix for
it will introduce extra checks on freeze and read only case.
For mount option change, later patches will introduce copy-n-update
method and rwsem protects to keep mount options consistent during
transaction.
For sysfs interface to change label/features, it will keep the same
behavior as 'btrfs pro set', so pending changes are also not needed.
Revert them to a clean base for later changes.
Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com
Conflicts:
fs/btrfs/ctree.h
fs/btrfs/super.c
---
fs/btrfs/ctree.h | 49 +
fs/btrfs/disk-io.c | 8 +++-
fs/btrfs/inode-map.c | 2 +-
fs/btrfs/super.c | 14 +++---
fs/btrfs/sysfs.c | 34 +-
fs/btrfs/transaction.c | 38 ++
fs/btrfs/transaction.h | 2 --
7 files changed, 35 insertions(+), 112 deletions(-)
diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 7e60741..5f99743 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -1411,11 +1411,6 @@ struct btrfs_fs_info {
*/
u64 last_trans_log_full_commit;
unsigned long mount_opt;
- /*
-* Track requests for actions that need to be done during transaction
-* commit (like for some mount options).
-*/
- unsigned long pending_changes;
unsigned long compress_type:4;
int commit_interval;
/*
@@ -2113,6 +2108,7 @@ struct btrfs_ioctl_defrag_range_args {
#define BTRFS_MOUNT_CHECK_INTEGRITY_INCLUDING_EXTENT_DATA (1 21)
#define BTRFS_MOUNT_PANIC_ON_FATAL_ERROR (1 22)
#define BTRFS_MOUNT_RESCAN_UUID_TREE (1 23)
+#defineBTRFS_MOUNT_CHANGE_INODE_CACHE (1 24)
#define BTRFS_DEFAULT_COMMIT_INTERVAL (30)
#define BTRFS_DEFAULT_MAX_INLINE (8192)
@@ -2138,49 +2134,6 @@ struct btrfs_ioctl_defrag_range_args {
}
/*
- * Requests for changes that need to be done during transaction commit.
- *
- * Internal mount options that are used for special handling of the real
- * mount options (eg. cannot be set during remount and have to be set during
- * transaction commit)
- */
-
-#define BTRFS_PENDING_SET_INODE_MAP_CACHE (0)
-#define BTRFS_PENDING_CLEAR_INODE_MAP_CACHE(1)
-#define BTRFS_PENDING_COMMIT (2)
-
-#define btrfs_test_pending(info, opt) \
- test_bit(BTRFS_PENDING_##opt, (info)-pending_changes)
-#define btrfs_set_pending(info, opt) \
- set_bit(BTRFS_PENDING_##opt, (info)-pending_changes)
-#define btrfs_clear_pending(info, opt) \
- clear_bit(BTRFS_PENDING_##opt, (info)-pending_changes)
-
-/*
- * Helpers for setting pending mount option changes.
- *
- * Expects corresponding macros
- * BTRFS_PENDING_SET_ and CLEAR_ + short mount option name
- */
-#define btrfs_set_pending_and_info(info, opt, fmt, args...)\
-do { \
- if (!btrfs_raw_test_opt((info)-mount_opt, opt)) { \
- btrfs_info((info), fmt, ##args);\
- btrfs_set_pending((info), SET_##opt); \
- btrfs_clear_pending((info), CLEAR_##opt); \
- } \
-} while(0)
-
-#define btrfs_clear_pending_and_info(info, opt, fmt, args...) \
-do { \
- if (btrfs_raw_test_opt((info)-mount_opt, opt)) { \
- btrfs_info((info), fmt, ##args);\
- btrfs_set_pending((info), CLEAR_##opt); \
- btrfs_clear_pending((info), SET_##opt); \
- } \
-} while(0)
-
-/*
* Inode flags
*/
#define BTRFS_INODE_NODATASUM (1 0)
diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c
index 8c63419..2d3c8b7 100644
--- a/fs/btrfs/disk-io.c
+++ b/fs/btrfs/disk-io.c
@@ -2832,11 +2832,9 @@ retry_root_backup:
btrfs_set_opt(fs_info-mount_opt, SSD);
}
- /*
-* Mount does not set all options immediatelly, we can do it now and do
-* not have to wait for transaction commit
-*/
- btrfs_apply_pending_changes(fs_info);
+ /* Set the real inode map cache flag */
+ if (btrfs_test_opt(tree_root, CHANGE_INODE_CACHE))
+ btrfs_set_opt(tree_root-fs_info-mount_opt, INODE_MAP_CACHE);
#ifdef CONFIG_BTRFS_FS_CHECK_INTEGRITY
if (btrfs_test_opt(tree_root, CHECK_INTEGRITY)) {
diff --git a/fs/btrfs/inode-map.c b/fs/btrfs/inode-map.c
index 74faea3..81efd83 100644
--- a/fs/btrfs/inode-map.c
+++ b/fs/btrfs/inode-map.c
@@ -178,7 +178,7 @@ static void start_caching(struct btrfs_root *root)
[PATCH RFC v3 2/5] btrfs: Make btrfs_parse_options() parse mount option in a atomic way
Current btrfs_parse_options() is not atomic, which can set and clear a
bit, especially for nospace_cache case.
For example, if a fs is mounted with nospace_cache,
btrfs_parse_options() will set SPACE_CACHE bit first(since
cache_generation is non-zeo) and clear the SPACE_CACHE bit due to
nospace_cache mount option.
So under heavy operations and remount a nospace_cache btrfs, there is a
windows for commit to create space cache.
This bug can be reproduced by fstest/btrfs/071 073 074 with
nospace_cache mount option. It has about 50% chance to create space
cache, and about 10% chance to create wrong space cache, which can't
pass btrfsck.
This patch will do the mount option parse in a copy-and-update method.
First copy the mount_opt from fs_info to new_opt, and only update
options in new_opt. At last, copy the new_opt back to
fs_info-mount_opt.
This patch is already good enough to fix the above nospace_cache +
remount bug, but need later patch to make sure mount options does not
change during transaction.
Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com
---
fs/btrfs/ctree.h | 16 +++
fs/btrfs/inode-map.c | 3 +-
fs/btrfs/super.c | 115 +++
3 files changed, 71 insertions(+), 63 deletions(-)
diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
index 5f99743..26bb47b 100644
--- a/fs/btrfs/ctree.h
+++ b/fs/btrfs/ctree.h
@@ -2119,18 +2119,18 @@ struct btrfs_ioctl_defrag_range_args {
#define btrfs_test_opt(root, opt) ((root)-fs_info-mount_opt \
BTRFS_MOUNT_##opt)
-#define btrfs_set_and_info(root, opt, fmt, args...)\
+#define btrfs_set_and_info(fs_info, val, opt, fmt, args...)\
{ \
- if (!btrfs_test_opt(root, opt)) \
- btrfs_info(root-fs_info, fmt, ##args); \
- btrfs_set_opt(root-fs_info-mount_opt, opt); \
+ if (!btrfs_raw_test_opt(val, opt)) \
+ btrfs_info(fs_info, fmt, ##args); \
+ btrfs_set_opt(val, opt);\
}
-#define btrfs_clear_and_info(root, opt, fmt, args...) \
+#define btrfs_clear_and_info(fs_info, val, opt, fmt, args...) \
{ \
- if (btrfs_test_opt(root, opt)) \
- btrfs_info(root-fs_info, fmt, ##args); \
- btrfs_clear_opt(root-fs_info-mount_opt, opt); \
+ if (btrfs_raw_test_opt(val, opt)) \
+ btrfs_info(fs_info, fmt, ##args); \
+ btrfs_clear_opt(val, opt); \
}
/*
diff --git a/fs/btrfs/inode-map.c b/fs/btrfs/inode-map.c
index 81efd83..d1edab5 100644
--- a/fs/btrfs/inode-map.c
+++ b/fs/btrfs/inode-map.c
@@ -178,7 +178,8 @@ static void start_caching(struct btrfs_root *root)
root-root_key.objectid);
if (IS_ERR(tsk)) {
btrfs_warn(root-fs_info, failed to start inode caching task);
- btrfs_clear_and_info(root, CHANGE_INODE_CACHE,
+ btrfs_clear_and_info(root-fs_info, root-fs_info-mount_opt,
+ CHANGE_INODE_CACHE,
disabling inode map caching);
}
}
diff --git a/fs/btrfs/super.c b/fs/btrfs/super.c
index b0c45b2..490fe1f 100644
--- a/fs/btrfs/super.c
+++ b/fs/btrfs/super.c
@@ -395,10 +395,13 @@ int btrfs_parse_options(struct btrfs_root *root, char
*options)
int ret = 0;
char *compress_type;
bool compress_force = false;
+ unsigned long new_opt;
+
+ new_opt = info-mount_opt;
cache_gen = btrfs_super_cache_generation(root-fs_info-super_copy);
if (cache_gen)
- btrfs_set_opt(info-mount_opt, SPACE_CACHE);
+ btrfs_set_opt(new_opt, SPACE_CACHE);
if (!options)
goto out;
@@ -422,7 +425,7 @@ int btrfs_parse_options(struct btrfs_root *root, char
*options)
switch (token) {
case Opt_degraded:
btrfs_info(root-fs_info, allowing degraded mounts);
- btrfs_set_opt(info-mount_opt, DEGRADED);
+ btrfs_set_opt(new_opt, DEGRADED);
break;
case Opt_subvol:
case Opt_subvolid:
@@ -434,7 +437,7 @@ int btrfs_parse_options(struct btrfs_root *root, char
*options)
*/
break;
case Opt_nodatasum:
- btrfs_set_and_info(root, NODATASUM,
+ btrfs_set_and_info(info, new_opt, NODATASUM,
[PATCH RFC v3 3/5] btrfs: Introduce per-transaction mount_opt to keep mount option consistent during transaction.
Before this patch, mount_opt is not consistent during a transaction.
btrfs_parse_options() can race with transaction.
Now each transaction will keep a copy of fs_info-mount_opt upon
creation, and new btrfs_test_trans_opt() macro is introduced to get the
mount_opt in the transaction.
Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com
---
fs/btrfs/transaction.c | 1 +
fs/btrfs/transaction.h | 4
2 files changed, 5 insertions(+)
diff --git a/fs/btrfs/transaction.c b/fs/btrfs/transaction.c
index 295a135..846e1b8 100644
--- a/fs/btrfs/transaction.c
+++ b/fs/btrfs/transaction.c
@@ -210,6 +210,7 @@ loop:
return -EROFS;
}
+ cur_trans-mount_opt = fs_info-mount_opt;
atomic_set(cur_trans-num_writers, 1);
extwriter_counter_init(cur_trans, type);
init_waitqueue_head(cur_trans-writer_wait);
diff --git a/fs/btrfs/transaction.h b/fs/btrfs/transaction.h
index fd400a3..4052879 100644
--- a/fs/btrfs/transaction.h
+++ b/fs/btrfs/transaction.h
@@ -52,6 +52,7 @@ struct btrfs_transaction {
struct list_head list;
struct extent_io_tree dirty_pages;
unsigned long start_time;
+ unsigned long mount_opt;
wait_queue_head_t writer_wait;
wait_queue_head_t commit_wait;
struct list_head pending_snapshots;
@@ -126,6 +127,9 @@ struct btrfs_pending_snapshot {
struct list_head list;
};
+#define btrfs_test_trans_opt(trans, opt) \
+ (btrfs_raw_test_opt(trans-transaction-mount_opt, opt))
+
static inline void btrfs_set_inode_last_trans(struct btrfs_trans_handle *trans,
struct inode *inode)
{
--
2.2.2
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH RFC v3 0/5] mount options consistent enhancement
Patchset to solve the previous found deadlock and enhance mount options consistence. Unlike previous pending_changes, which uses transaction commits to ensure mount option doesn't change during transaction. This patch use the RCU-like concept, which will copy the mount_opt from fs_info into btrfs_transaction, and each btrfs_test_opt() for specific mount option bit(SPACE_CACHE/INODE_MAP_CACHE) should be changed to btrfs_test_trans_opt(). So mount option during transaction won't be changed, and also, no extra read_only or freeze check is needed, the commit routine is also untouched. Qu Wenruo (5): Revert btrfs: add support for processing pending changes related commits btrfs: Make btrfs_parse_options() parse mount option in a atomic way btrfs: Introduce per-transaction mount_opt to keep mount option consistent during transaction. btrfs: Use btrfs_test_trans_opt() to handle SPACE_CACHE if it's under transaction protect. btrfs: Use btrfs_test_trans_opt() to handle INODE_CACHE if it's under transaction protect. fs/btrfs/ctree.h| 64 +++--- fs/btrfs/disk-io.c | 6 --- fs/btrfs/extent-tree.c | 2 +- fs/btrfs/free-space-cache.c | 2 +- fs/btrfs/inode-map.c| 5 +- fs/btrfs/super.c| 129 ++-- fs/btrfs/sysfs.c| 34 +++- fs/btrfs/transaction.c | 43 ++- fs/btrfs/transaction.h | 6 ++- 9 files changed, 107 insertions(+), 184 deletions(-) -- 2.2.2 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: Recovery Operation With Multiple Devices
-Original message- From: Hugo Mills h...@carfax.org.uk Sent: Fri 01-23-2015 08:48 pm Subject:Re: Recovery Operation With Multiple Devices Attachment: signature.asc To: Brett King brett.k...@commandict.com.au; CC: linux-btrfs@vger.kernel.org; On Fri, Jan 23, 2015 at 06:53:42PM +1100, Brett King wrote: Hi All, Just wondering how 'btrfs recovery' operates, when the source device given is one of many in an MD array - I can't find anything documentation beyond a single device use case. Does it automatically include all devices in the relevant MD array as occurs when mounting, or does it only restore the data which happened to be written to the specific, single device given ? Neither. :) It automatically includes all devices as occurs when running btrfsck. Fantastic - that is very powerful and exactly what I need. OK, it's a relatively pointless distinction, but the mechanisms are slightly different. From an inverse perspective, how can I restore all data including snapshots, which are spread across a damaged MD FS to a new (MD) FS ? btrfs restore -l will show you the list of tree roots, which (for trees numbered 256 and up) correspond to subvolumes and snapshots. You can then use -r to select the tree to restore. Great info, will keep that in the tool box ! Can send / receive do this perhaps ? Yes, but only if you can mount the FS. Hugo. Excellent, much appreciated. -- Hugo Mills | I know of three kinds: hot, cool, and hugo@... carfax.org.uk | what-time-does-the-tune-start? http://carfax.org.uk/ | PGP: 65E74AC0 | Chris Dollin -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
btrfs send and interrupts
Hi, root@data:/daten/backup/email/xmit# btrfs send foo | ssh wherever btrfs receive /mnt/mail/xmit/ At subvol foo At subvol foo [ some time passes and I need to do something else on that volume ] ^Z [1]+ Stopped btrfs send foo | ssh -p50022 surf btrfs receive /mnt/mail/xmit/ root@data:/daten/backup/email/xmit# bg [1]+ btrfs send foo | ssh -p50022 surf btrfs receive /mnt/mail/xmit/ root@data:/daten/backup/email/xmit# [ Immediately afterwards, this happens: ] ERROR: crc32 mismatch in command. At subvol foo At subvol foo ERROR: creating subvolume foo failed. File exists [1]+ Exit 1 btrfs send foo | ssh -p50022 surf btrfs receive /mnt/mail/xmit/ root@data:/daten/backup/email/xmit# Yowch. Please make sure that the simple act of backgrounding a data transfer doesn't abort it. That was ten hours in, now I have to repeat the whole thing. :-/ Thank you. -- -- Matthias Urlichs signature.asc Description: Digital signature
Moving snapshots
Hello, how do I move a (read-only) snapshot? Simply creating another read-only snap from the first one and then deleting the source works, except that it destroy's the snapshot's identity -- which means that it can't be used as a parent for btrfs receive any more. :-( -- -- Matthias Urlichs signature.asc Description: Digital signature
Re: Recovery Operation With Multiple Devices
On 2015/01/23 09:53, Brett King wrote: Hi All, Just wondering how 'btrfs recovery' operates I'm assuming you're referring to a different set of commands or general scrub/recovery processes. AFAIK there is no btrfs recovery command. , when the source device given is one of many in an MD array - I can't find anything documentation beyond a single device use case. btrfs doesn't know what an md array or member is, therefore your results aren't going to be well-defined. Depending on the type of md array the member was in, your data may be mostly readable (RAID1) or completely/mostly non-interpretable (RAID5/6/10/0) until md fixes the array. Does it automatically include all devices in the relevant MD array as occurs when mounting, or does it only restore the data which happened to be written to the specific, single device given ? As above, btrfs is not md-aware. It will attempt to work with what it is given. It might not understand anything it sees as it will not have a good description of what it is looking at. Imagine being given instructions on how to get somewhere only to find that the first 20 instructions and every second instruction thereafter was skipped and there's a 50% chance the destination doesn't exist. From an inverse perspective, how can I restore all data including snapshots, which are spread across a damaged MD FS to a new (MD) FS ? Your best bet is to restore the md array. More details are needed for anyone to assist - for example what RAID-type was the array set up with, how many disks were in the array, and how it failed. Also, technically this is the wrong place to ask for advice about restoring md arrays. ;) Can send / receive do this perhaps ? Send/receive is for sending good data to a destination that can accept it. This, as above, depends on the data being readable/available. Very likely the data will be unreadable from a single disk unless the md array was RAID1. Thanks in advance ! -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- __ Brendan Hide http://swiftspirit.co.za/ http://www.webafrica.co.za/?AFF1E97 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH] btrfs-progs:qgroup:make large size aligned
On Fri, Jan 23, 2015 at 09:50:49AM +0800, Fan Chengniang wrote: problem: when the size is too big, the output format will be unaligned. the __update__columns_max_len function has been updated to fix this problem Signed-off-by: Fan Chengniang fancn.f...@cn.fujitsu.com --- In my patch [PATCH v3] make btrfs qgroups show human readable sizes I forget to update the function __update__columns_max_len which may cause large size unaligned. Thanks, applied. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3] Btrfs: Add code to support file creation time
From: chandan r chandanrm...@gmail.com
This patch adds a new member to the 'struct btrfs_inode' structure to hold
the file creation time.
Signed-off-by: chandan chandanrm...@gmail.com
[refreshed, removed btrfs_inode_otime]
Signed-off-by: David Sterba dste...@suse.cz
---
V3:
- removed forgotten use of btrfs_inode_otime
- simplified current time assignments (Zach)
fs/btrfs/btrfs_inode.h | 3 +++
fs/btrfs/delayed-inode.c | 10 ++
fs/btrfs/inode.c | 25 +++--
3 files changed, 36 insertions(+), 2 deletions(-)
diff --git a/fs/btrfs/btrfs_inode.h b/fs/btrfs/btrfs_inode.h
index 4aadadcfab20..de5e4f2adfea 100644
--- a/fs/btrfs/btrfs_inode.h
+++ b/fs/btrfs/btrfs_inode.h
@@ -185,6 +185,9 @@ struct btrfs_inode {
struct btrfs_delayed_node *delayed_node;
+ /* File creation time. */
+ struct timespec i_otime;
+
struct inode vfs_inode;
};
diff --git a/fs/btrfs/delayed-inode.c b/fs/btrfs/delayed-inode.c
index 116eb4bed8d3..82f0c7c95474 100644
--- a/fs/btrfs/delayed-inode.c
+++ b/fs/btrfs/delayed-inode.c
@@ -1769,6 +1769,11 @@ static void fill_stack_inode_item(struct
btrfs_trans_handle *trans,
inode-i_ctime.tv_sec);
btrfs_set_stack_timespec_nsec(inode_item-ctime,
inode-i_ctime.tv_nsec);
+
+ btrfs_set_stack_timespec_sec(inode_item-otime,
+BTRFS_I(inode)-i_otime.tv_sec);
+ btrfs_set_stack_timespec_nsec(inode_item-otime,
+BTRFS_I(inode)-i_otime.tv_nsec);
}
int btrfs_fill_inode(struct inode *inode, u32 *rdev)
@@ -1810,6 +1815,11 @@ int btrfs_fill_inode(struct inode *inode, u32 *rdev)
inode-i_ctime.tv_sec = btrfs_stack_timespec_sec(inode_item-ctime);
inode-i_ctime.tv_nsec = btrfs_stack_timespec_nsec(inode_item-ctime);
+ BTRFS_I(inode)-i_otime.tv_sec =
+ btrfs_stack_timespec_sec(inode_item-otime);
+ BTRFS_I(inode)-i_otime.tv_nsec =
+ btrfs_stack_timespec_nsec(inode_item-otime);
+
inode-i_generation = BTRFS_I(inode)-generation;
BTRFS_I(inode)-index_cnt = (u64)-1;
diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c
index 87f1d9b8727e..5da80af467a6 100644
--- a/fs/btrfs/inode.c
+++ b/fs/btrfs/inode.c
@@ -3535,6 +3535,11 @@ static void btrfs_read_locked_inode(struct inode *inode)
inode-i_ctime.tv_sec = btrfs_timespec_sec(leaf, inode_item-ctime);
inode-i_ctime.tv_nsec = btrfs_timespec_nsec(leaf, inode_item-ctime);
+ BTRFS_I(inode)-i_otime.tv_sec =
+ btrfs_timespec_sec(leaf, inode_item-otime);
+ BTRFS_I(inode)-i_otime.tv_nsec =
+ btrfs_timespec_nsec(leaf, inode_item-otime);
+
inode_set_bytes(inode, btrfs_inode_nbytes(leaf, inode_item));
BTRFS_I(inode)-generation = btrfs_inode_generation(leaf, inode_item);
BTRFS_I(inode)-last_trans = btrfs_inode_transid(leaf, inode_item);
@@ -3669,6 +3674,11 @@ static void fill_inode_item(struct btrfs_trans_handle
*trans,
btrfs_set_token_timespec_nsec(leaf, item-ctime,
inode-i_ctime.tv_nsec, token);
+ btrfs_set_token_timespec_sec(leaf, item-otime,
+BTRFS_I(inode)-i_otime.tv_sec, token);
+ btrfs_set_token_timespec_nsec(leaf, item-otime,
+ BTRFS_I(inode)-i_otime.tv_nsec, token);
+
btrfs_set_token_inode_nbytes(leaf, item, inode_get_bytes(inode),
token);
btrfs_set_token_inode_generation(leaf, item, BTRFS_I(inode)-generation,
@@ -5256,7 +5266,10 @@ static struct inode *new_simple_dir(struct super_block
*s,
inode-i_op = btrfs_dir_ro_inode_operations;
inode-i_fop = simple_dir_operations;
inode-i_mode = S_IFDIR | S_IRUGO | S_IWUSR | S_IXUGO;
- inode-i_mtime = inode-i_atime = inode-i_ctime = CURRENT_TIME;
+ inode-i_mtime = CURRENT_TIME;
+ inode-i_atime = inode-i_mtime;
+ inode-i_ctime = inode-i_mtime;
+ BTRFS_I(inode)-i_otime = inode-i_mtime;
return inode;
}
@@ -5824,7 +5837,12 @@ static struct inode *btrfs_new_inode(struct
btrfs_trans_handle *trans,
inode_init_owner(inode, dir, mode);
inode_set_bytes(inode, 0);
- inode-i_mtime = inode-i_atime = inode-i_ctime = CURRENT_TIME;
+
+ inode-i_mtime = CURRENT_TIME;
+ inode-i_atime = inode-i_mtime;
+ inode-i_ctime = inode-i_mtime;
+ BTRFS_I(inode)-i_otime = inode-i_mtime;
+
inode_item = btrfs_item_ptr(path-nodes[0], path-slots[0],
struct btrfs_inode_item);
memset_extent_buffer(path-nodes[0], 0, (unsigned long)inode_item,
@@ -8574,6 +8592,9 @@ struct inode *btrfs_alloc_inode(struct super_block *sb)
ei-delayed_node = NULL;
+ ei-i_otime.tv_sec = 0;
+ ei-i_otime.tv_nsec = 0;
+
inode = ei-vfs_inode;
Re: btrfs send and interrupts
On 2015/01/23 11:58, Matthias Urlichs wrote: Hi, root@data:/daten/backup/email/xmit# btrfs send foo | ssh wherever btrfs receive /mnt/mail/xmit/ At subvol foo At subvol foo [ some time passes and I need to do something else on that volume ] ^Z [1]+ Stopped btrfs send foo | ssh -p50022 surf btrfs receive /mnt/mail/xmit/ root@data:/daten/backup/email/xmit# bg [1]+ btrfs send foo | ssh -p50022 surf btrfs receive /mnt/mail/xmit/ root@data:/daten/backup/email/xmit# [ Immediately afterwards, this happens: ] ERROR: crc32 mismatch in command. At subvol foo At subvol foo ERROR: creating subvolume foo failed. File exists [1]+ Exit 1 btrfs send foo | ssh -p50022 surf btrfs receive /mnt/mail/xmit/ root@data:/daten/backup/email/xmit# Yowch. Please make sure that the simple act of backgrounding a data transfer doesn't abort it. That was ten hours in, now I have to repeat the whole thing. :-/ Thank you. Interesting case. I'm not sure of the merits/workaround needed to do this. It appears even using cat into netcat (nc) causes netcat to quit if you background the operation. A workaround for future: I *strongly* recommend using screen for long-lived operations. This would have avoided the problem. Perhaps you were sitting in front of the server and it wasn't much of a concern at the time - but most admins work remotely. Never mind ^z, what about other occurrences such as if the power/internet goes out at your office/home and the server is on another continent? Your session dies and you lose 10 hours of work/waiting. With a screen session, that is no longer true. -- __ Brendan Hide http://swiftspirit.co.za/ http://www.webafrica.co.za/?AFF1E97 -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 3.19-rc5: Bug 91911: [REGRESSION] rm command hangs big time with deleting a lot of files at once
On Fri, 23 Jan 2015 15:01:28 +0100, Martin Steigerwald wrote: Hi! Anyone seen this? Reported as: https://bugzilla.kernel.org/show_bug.cgi?id=91911 You might be interested in: https://git.kernel.org/cgit/linux/kernel/git/josef/btrfs-next.git/commit/?h=evict-softlockupid=29249e14d6e3379a5c4bb098dd4beddfefbc606f and https://git.kernel.org/cgit/linux/kernel/git/josef/btrfs-next.git/commit/?h=evict-softlockupid=e4a58b71ff981b098ac3371f4d573dc6a90006ce I'm sure everyone would love to hear how this works out for you ;-) -h -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PULL] [PATCH 0/3] Btrfs, fixes for freezing vs pending changes
On Fri, Jan 23, 2015 at 10:32:16AM +0800, Qu Wenruo wrote: 1) mount option change problem. In fact, there is no need to start a transaction to change mount option, since it doesn't change anything on-disk. The commit is to flush all the data that should see the same state of the mount option. What we need is just to keep the mount option doesn't change during transaction. Yes, that's better. So I prefer to add a rwsem to protect mount_opt, each btrfs_transaction will hold the read lock on it and upon btrfs_put_transaction(), read unlock it. btrfs_parse_option() should wait for write lock to change it. Ok. BTW, current btrfs_parse_options() is not atomic, and for nospace_cache mount option, SPACE_CACHE bit is always first set and later cleared, which created a window btrfs_commit_transaction() can create space cache. I'll solve it by using copy-n-update method. That's a unrelated fix to the pending changes, but a needed one. 2) Sysfs label/feature change problem For this problem, I agree with Miao to keep the behavior the same as btrfs pro set command, since it will write something on disk. And since btrfs_ioctl_set_fslabel() is synchronized, I didn't see the necessity to change it to async using sysfs. The calling context is different from 'btrfs prop set' and direct ioctl call. Details in the reply to your RFC mail. What do you think about this idea? Although, I'm afraid this may revert all your pending_changes patches :- Given the timeframe of dev cycle, I think the minimal fix to avoid the sync deadlock will probably go in, as is in the for-linus branch right now. Next, the lock-protected mount options would replace current way of handling the inode_cache and we can see if we find different way how to get rid of the async commit. -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH RFC v3 1/5] Revert btrfs: add support for processing pending changes related commits
On Fri, Jan 23, 2015 at 05:31:41PM +0800, Qu Wenruo wrote:
For mount option change, later patches will introduce copy-n-update
method and rwsem protects to keep mount options consistent during
transaction.
That's a better approach, for the mount options.
For sysfs interface to change label/features, it will keep the same
behavior as 'btrfs pro set', so pending changes are also not needed.
This still leaves the transaction commit inside the syfs handler, that
was one of the points not to do that.
The callstack looks safe from, eg. the label handler:
[169148.523158] WARNING: CPU: 1 PID: 2044 at fs/btrfs/sysfs.c:394
btrfs_label_store+0x135/0x190 [btrfs]()
[169148.533925] Modules linked in: btrfs dm_flakey rpcsec_gss_krb5 loop [last
unloaded: btrfs]
[169148.536950] CPU: 1 PID: 2044 Comm: bash Tainted: GW
3.19.0-rc5-default+ #211
[169148.536952] Hardware name: Intel Corporation Santa Rosa platform/Matanzas,
BIOS TSRSCRB1.86C.0047.B00.0610170821 10/17/06
[169148.536954] 018a 88007a753dc8 81a9898b
018a
[169148.536963] 88007a753e08 81077f65
880077fb0100
[169148.536972] 880075dc 880077fbff00 0009
880075dc06d0
[169148.536980] Call Trace:
[169148.536983] [81a9898b] dump_stack+0x4f/0x6c
[169148.536991] [81077f65] warn_slowpath_common+0x95/0xe0
[169148.537000] [81077fca] warn_slowpath_null+0x1a/0x20
[169148.537005] [a0052b65] btrfs_label_store+0x135/0x190 [btrfs]
[169148.537030] [813ed8b7] kobj_attr_store+0x17/0x20
[169148.537037] [812147ff] sysfs_kf_write+0x4f/0x70
[169148.537044] [81213cc8] kernfs_fop_write+0x128/0x180
[169148.537051] [8119f404] vfs_write+0xd4/0x1d0
[169148.537059] [8119f7b9] SyS_write+0x59/0xd0
[169148.537070] [81a9f9d2] system_call_fastpath+0x12/0x17
Lockep shows these locks held:
[169148.537296] 4 locks held by bash/2044:
[169148.537309] #0: (sb_writers#5){.+.+.+}, at: [8119f4e0]
vfs_write+0x1b0/0x1d0
[169148.537319] #1: (of-mutex){+.+.+.}, at: [81213c2e]
kernfs_fop_write+0x8e/0x180
[169148.537330] #2: (s_active#214){.+.+.+}, at: [81213c36]
kernfs_fop_write+0x96/0x180
[169148.537342] #3: (tasklist_lock){.+.+..}, at: [810b9ed4]
debug_show_all_locks+0x44/0x1e0
#3 is from lockdep
#2 is not really a lock, annotated vfs atomic counter
#0 is annotated atomic, the freezing barrier
#1 is a kernfs mutex that, afaics it's per file, but I don't like to see
the lock dependency here. That's a lock we can see now, but it's outside
of btrfs or the vfs. It's a matter of precaution.
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: 3.19-rc5: Bug 91911: [REGRESSION] rm command hangs big time with deleting a lot of files at once
On Fri, Jan 23, 2015 at 9:38 AM, Holger Hoffstätte holger.hoffstae...@googlemail.com wrote: On Fri, 23 Jan 2015 15:01:28 +0100, Martin Steigerwald wrote: Hi! Anyone seen this? Reported as: https://urldefense.proofpoint.com/v1/url?u=https://bugzilla.kernel.org/show_bug.cgi?id%3D91911k=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0Ar=6%2FL0lzzDhu0Y1hL9xm%2BQyA%3D%3D%0Am=fHpzX%2FNpDGcY26Q1T%2B0VtIyZz0GPMnG3qvE%2FVJ%2Blsf0%3D%0As=66b06542cc5a9585c9913a0b76885b4baba0a8028f70149803982388c9333d71 You might be interested in: https://urldefense.proofpoint.com/v1/url?u=https://git.kernel.org/cgit/linux/kernel/git/josef/btrfs-next.git/commit/?h%3Devict-softlockup%26id%3D29249e14d6e3379a5c4bb098dd4beddfefbc606fk=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0Ar=6%2FL0lzzDhu0Y1hL9xm%2BQyA%3D%3D%0Am=fHpzX%2FNpDGcY26Q1T%2B0VtIyZz0GPMnG3qvE%2FVJ%2Blsf0%3D%0As=137b101ca50e644b95dc2b14758cf1472e9cc0097b4f89f7edf8e2845455b20f and https://urldefense.proofpoint.com/v1/url?u=https://git.kernel.org/cgit/linux/kernel/git/josef/btrfs-next.git/commit/?h%3Devict-softlockup%26id%3De4a58b71ff981b098ac3371f4d573dc6a90006cek=ZVNjlDMF0FElm4dQtryO4A%3D%3D%0Ar=6%2FL0lzzDhu0Y1hL9xm%2BQyA%3D%3D%0Am=fHpzX%2FNpDGcY26Q1T%2B0VtIyZz0GPMnG3qvE%2FVJ%2Blsf0%3D%0As=3462ceb04ffed8316e21ffe16eee4e3274f9771a58326cd1bbb9c1cfc1eef9ba I'm sure everyone would love to hear how this works out for you ;-) These are a little different. Josef is fixing softlockups (CPUs pegged in R state) while this bug report is for sleeping procs. The fact that things come back sound like we're pegged waiting for IO? Do you have a lot of writing going on at the same time? -chris -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v4] btrfs: add regression test for remount with thread_pool resized
On Fri, Jan 23, 2015 at 03:28:59PM +0800, Xing Gu wrote:
Regression test for a btrfs issue of resizing 'thread_pool' when
remount the fs.
This regression was introduced by the following linux kernel commit:
btrfs: Added btrfs_workqueue_struct implemented ordered
execution based on kernel workqueue
08a9ff3264181986d1d692a4e6fce3669700c9f8
And it was fixed by the following linux kernel commit:
btrfs: fix crash in remount(thread_pool=) case
800ee2247f483b6d05ed47ef3bbc90b56451746c
Signed-off-by: Xing Gu gux.f...@cn.fujitsu.com
---
v3-v4: Remove the check for a bug message.
v2-v3: Add the regresssion commit in description.
Remove dmesg -c /dev/null.
Echo Silence is golden to indicate that an empty output
file is expected.
v1-v2: Add the fix commit in description.
Add the check for a bug message.
---
tests/btrfs/082 | 65
+
tests/btrfs/082.out | 2 ++
tests/btrfs/group | 1 +
3 files changed, 68 insertions(+)
create mode 100755 tests/btrfs/082
create mode 100644 tests/btrfs/082.out
diff --git a/tests/btrfs/082 b/tests/btrfs/082
new file mode 100755
index 000..7a6e4b5
--- /dev/null
+++ b/tests/btrfs/082
@@ -0,0 +1,65 @@
+#!/bin/bash
+# FS QA Test No. btrfs/082
+#
+# Regression test for a btrfs issue of resizing 'thread_pool' when
+# remount the fs.
+#
+# This regression was introduced by the following linux kernel commit:
+# btrfs: Added btrfs_workqueue_struct implemented ordered
+# execution based on kernel workqueue
+# 08a9ff3264181986d1d692a4e6fce3669700c9f8
+# And it was fixed by the following linux kernel commit:
+# btrfs: fix crash in remount(thread_pool=) case
+# 800ee2247f483b6d05ed47ef3bbc90b56451746c
+#
+#---
+# Copyright (c) 2015 Fujitsu. All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc., 51 Franklin St, Fifth Floor, Boston, MA 02110-1301 USA
+#
+#---
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo QA output created by $seq
+
+here=`pwd`
+tmp=/tmp/$$
+status=1 # failure is the default!
+
+_cleanup()
+{
+rm -f $tmp.*
+}
+
+trap _cleanup ; exit \$status 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs /dev/null
Redirect stderr to /dev/null too, otherwise test fails if scratch device
supports trim.
btrfs/082 0s ... - output mismatch (see
/root/xfstests/results//btrfs/082.out.bad)
--- tests/btrfs/082.out 2015-01-23 20:27:15.603689913 +0800
+++ /root/xfstests/results//btrfs/082.out.bad 2015-01-23
20:33:46.514259626 +0800
@@ -1,2 +1,3 @@
QA output created by 082
+Performing full device TRIM (5.00GiB) ...
Silence is golden
...
(Run 'diff -u tests/btrfs/082.out
/root/xfstests/results//btrfs/082.out.bad' to see the entire diff)
Others look good to me.
Thanks,
Eryu
+
+_scratch_mount -o thread_pool=6
+_scratch_mount -o remount,thread_pool=10
+
+echo Silence is golden
+status=0
+exit
diff --git a/tests/btrfs/082.out b/tests/btrfs/082.out
new file mode 100644
index 000..2977f14
--- /dev/null
+++ b/tests/btrfs/082.out
@@ -0,0 +1,2 @@
+QA output created by 082
+Silence is golden
diff --git a/tests/btrfs/group b/tests/btrfs/group
index e2b..fd2fa76 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
@@ -84,3 +84,4 @@
079 auto rw metadata
080 auto snapshot
081 auto quick clone
+082 auto quick remount
--
1.9.3
--
To unsubscribe from this list: send the line unsubscribe linux-btrfs in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] fstests: btrfs/079: Fix wrong value passed to available space check.
On Wed, Jan 21, 2015 at 02:08:37PM +0800, Qu Wenruo wrote: Wrong value is passed to _require_fs_space, which should be in unit of kilobyte(1024), but passed in unit of gigabyte(1024^3). Fix it. Signed-off-by: Qu Wenruo quwen...@cn.fujitsu.com Reviewed-by: David Sterba dste...@suse.cz -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
