[PATCH] Btrfs: disk-io: replace root args iff only fs_info used

2014-11-21 Thread Daniel Dressler
This is the 3rd independent patch of a larger project to cleanup btrfs's internal usage of btrfs_root. Many functions take btrfs_root only to grab the fs_info struct. By requiring a root these functions cause programmer overhead. That these functions can accept any valid root is not obvious until

Re: Changing label few times killed filesystem?

2014-11-21 Thread Boris Chernov
On 2014-11-21 04:35, Roman Mamedov wrote: On Fri, 21 Nov 2014 01:27:17 + Boris Chernov aqs1...@hotmail.com wrote: I have changed file system label few times in total. When I tried to mount it after that, it became not mountable: # mount /dev/sdb1 /mnt mount: Not a directory I'd say

Re: [RFC PATCH 0/6] btrfs: implement swap file support

2014-11-21 Thread Christoph Hellwig
On Tue, Nov 18, 2014 at 11:22:35PM -0800, Omar Sandoval wrote: Here's a nice little bit of insanity I put together in that direction -- consider it a discussion point more than a patch. It does two things: - Uses an ITER_BVEC iov_iter to do direct_IO for swap_readpage. This makes

[PATCH v2 2/5] nfs: don't dirty ITER_BVEC pages read through direct I/O

2014-11-21 Thread Omar Sandoval
As with the generic blockdev code, kernel pages shouldn't be dirtied by the direct I/O path. Signed-off-by: Omar Sandoval osan...@osandov.com --- fs/nfs/direct.c | 5 - 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/fs/nfs/direct.c b/fs/nfs/direct.c index 10bf072..a67fa2c

[PATCH v2 5/5] btrfs: enable swap file support

2014-11-21 Thread Omar Sandoval
Implement the swap file a_ops on btrfs. Activation simply checks for a usable swap file: it must be fully allocated (no holes), support direct I/O (so no compressed or inline extents) and should be nocow (I'm not sure about that last one). Signed-off-by: Omar Sandoval osan...@osandov.com ---

[PATCH v2 1/5] direct-io: don't dirty ITER_BVEC pages on read

2014-11-21 Thread Omar Sandoval
Reads through the iov_iter infrastructure for kernel pages shouldn't be dirtied by the direct I/O code. This is based on Dave Kleikamp's and Ming Lei's previously posted patches. Cc: Dave Kleikamp dave.kleik...@oracle.com Cc: Ming Lei ming@canonical.com Signed-off-by: Omar Sandoval

Re: [RFC PATCH 0/6] btrfs: implement swap file support

2014-11-21 Thread Omar Sandoval
On Fri, Nov 21, 2014 at 02:06:57AM -0800, Christoph Hellwig wrote: On Tue, Nov 18, 2014 at 11:22:35PM -0800, Omar Sandoval wrote: Here's a nice little bit of insanity I put together in that direction -- consider it a discussion point more than a patch. It does two things: - Uses an

Re: 3.18.0-rc4 - WARNING: CPU: 3 PID: 3889 at fs/btrfs/extent-tree.c:3785 btrfs_free_reserved_data_space+0xc8/0xe0 [btrfs]()

2014-11-21 Thread Torbjørn
On 12. nov. 2014 09:29, Torbjørn wrote: Hi, After upgrade to 3.18.0-rc4 get a lot of these warnings. The system seems to otherwise behave normally. I have not observed any actual issues. --snip-- Just to close off this email thread. After upgrading to 3.18.0-rc5 these messages have

[PATCH v2 4/5] btrfs: don't allow -C or +c chattrs on a swap file

2014-11-21 Thread Omar Sandoval
swap_activate will check for a compressed or copy-on-write file; we shouldn't allow it to become either once it has already been activated. Signed-off-by: Omar Sandoval osan...@osandov.com --- fs/btrfs/ioctl.c | 50 +++--- 1 file changed, 31

[PATCH v2 3/5] swap: use direct I/O for SWP_FILE swap_readpage

2014-11-21 Thread Omar Sandoval
On Mon, Nov 17, 2014 at 07:48:17AM -0800, Christoph Hellwig wrote: With the new iov_iter infrastructure that supprots direct I/O to kernel pages please get rid of the -readpage hack first. I'm still utterly disapoined that this crap ever got merged. Cc: Mel Gorman mgor...@suse.de

[PATCH v2 0/5] btrfs: implement swap file support

2014-11-21 Thread Omar Sandoval
This patch series, based on 3.18-rc5, implements support for swap files on BTRFS. The standard swap file implementation uses the filesystem's implementation of bmap() to get a list of physical blocks on disk, which the swap file code then does I/O on directly without going through the filesystem.

Re: [PATCH v2 0/5] btrfs: implement swap file support

2014-11-21 Thread Omar Sandoval
Sorry for the noise, looks like Christoph got back to me on the previous RFC just before I sent this out -- disregard this for now. -- Omar -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at

Re: [PATCH v2 0/5] btrfs: implement swap file support

2014-11-21 Thread Christoph Hellwig
On Fri, Nov 21, 2014 at 02:15:31AM -0800, Omar Sandoval wrote: Sorry for the noise, looks like Christoph got back to me on the previous RFC just before I sent this out -- disregard this for now. If the NFS people are fine with this version I'd certainly welcome it as a first step. Additional

Re: [PATCH] btrfs: remove empty fs_devices to prevent memory runout

2014-11-21 Thread Anand Jain
Hi Gui, Thanks for attempting this. There was this patch previously attempted for the same problem, which I had to nack.. [PATCH 1/2] btrfs: device list could grow infinite I haven't seen your fix in full yet, But looks like you are using dev_t to suffice the problem of identifying

Re: BTRFS messes up snapshot LV with origin

2014-11-21 Thread Robert White
On 11/20/2014 10:22 PM, Duncan wrote: But while other filesystems might allow un-UUIDs (heh, UUUIDs or U3IDs =:^), because they're no longer unique, requiring them to be unique just as the label says cannot be considered a bug. It's simply stricter enforcement of the rules, which are, after

Re: Changing label few times killed filesystem?

2014-11-21 Thread Duncan
Chris Murphy posted on Thu, 20 Nov 2014 19:20:22 -0700 as excerpted: On Thu, Nov 20, 2014 at 6:27 PM, Boris Chernov aqs1...@hotmail.com wrote: Since it failed after checking extents I decided to try --init-extent-tree: There might be hope yet if you didn't use --repair which is said

Re: BTRFS messes up snapshot LV with origin

2014-11-21 Thread Duncan
Robert White posted on Fri, 21 Nov 2014 03:35:05 -0800 as excerpted: On 11/20/2014 10:22 PM, Duncan wrote: But while other filesystems might allow un-UUIDs (heh, UUUIDs or U3IDs =:^), because they're no longer unique, requiring them to be unique just as the label says cannot be considered a

Re: btrfs send and an existing backup

2014-11-21 Thread Austin S Hemmelgarn
On 2014-11-20 09:10, Duncan wrote: Bardur Arantsson posted on Thu, 20 Nov 2014 14:17:52 +0100 as excerpted: If you have no other backups, I would really recommend that you *don't* use btrfs for your backup, or at least have a *third* backup which isn't on btrfs -- there are *still* problems

Re: soft lockup - CPU#0 stuck - Kernel 3.17.2

2014-11-21 Thread Patrick Schmid
On 15.11.2014 00:47, Chris Mason wrote: Ok, I think this is related to the new fair read/writer lock implementation in the generic kernel code. btrfs_clear_path_blocking() will end up taking locks outside of the strict locking order the rest of Btrfs uses. This used to be fine because

Re: scrub implies failing drive - smartctl blissfully unaware

2014-11-21 Thread Ian Armstrong
On Fri, 21 Nov 2014 09:05:32 +0200, Brendan Hide wrote: On 2014/11/21 06:58, Zygo Blaxell wrote: I also notice you are not running regular SMART self-tests (e.g. by smartctl -t long) and the last (and first, and only!) self-test the drive ran was ~12000 hours ago. That means most of your

Re: soft lockup - CPU#0 stuck - Kernel 3.17.2

2014-11-21 Thread Chris Mason
On Fri, Nov 21, 2014 at 8:01 AM, Patrick Schmid sch...@phys.ethz.ch wrote: On 15.11.2014 00:47, Chris Mason wrote: Ok, I think this is related to the new fair read/writer lock implementation in the generic kernel code. btrfs_clear_path_blocking() will end up taking locks outside of the

Re: scrub implies failing drive - smartctl blissfully unaware

2014-11-21 Thread Phillip Susi
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/20/2014 5:45 PM, Robert White wrote: Nice attempt at saving face, but wrong as _always_. The CONFIG_PATA_ACPI option has been in the kernel since 2008 and lots of people have used it. If you search for ACPI ide you'll find people

Re: scrub implies failing drive - smartctl blissfully unaware

2014-11-21 Thread Phillip Susi
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/20/2014 6:08 PM, Robert White wrote: Well you should have _actually_ trimmed your response down to not pressing send. _Many_ motherboards have complete RAID support at levels 0, 1, 10, and five 5. A few have RAID6. Some of them even

Re: [PATCH v2 1/5] direct-io: don't dirty ITER_BVEC pages on read

2014-11-21 Thread Dave Kleikamp
On 11/21/2014 04:08 AM, Omar Sandoval wrote: Reads through the iov_iter infrastructure for kernel pages shouldn't be dirtied by the direct I/O code. This is based on Dave Kleikamp's and Ming Lei's previously posted patches. Acked-by: Dave Kleikamp dave.kleik...@oracle.com Cc: Ming Lei

Re: [PATCH v2] Btrfs: qgroup: add BUILD_BUG to report pointer cast breakage

2014-11-21 Thread David Sterba
On Thu, Nov 13, 2014 at 12:54:39AM +0900, Daniel Dressler wrote: Our ulist data structure stores at max 64bit values. qgroup has used this structure to store pointers. In the future when we upgrade to 128bit this casting of pointers to uint64_t will break. What are we going to upgrade to 128

Re: [PATCH] Btrfs: ctree: reduce args where only fs_info used

2014-11-21 Thread David Sterba
On Wed, Nov 12, 2014 at 01:43:09PM +0900, Daniel Dressler wrote: This patch is part of a larger project to cleanup btrfs's internal usage of struct btrfs_root. Many functions take btrfs_root only to grab a pointer to fs_info. Thanks for picking up the project. A mere formality, can you

Re: [PATCH] Btrfs: delayed-inode: replace root args iff only fs_info used

2014-11-21 Thread David Sterba
On Mon, Nov 17, 2014 at 10:05:02PM +0900, Daniel Dressler wrote: This is the second independent patch of a larger project to cleanup btrfs's internal usage of btrfs_root. Many functions take btrfs_root only to grab the fs_info struct. By requiring a root these functions cause programmer

Re: [PATCH] Btrfs: ctree: reduce args where only fs_info used

2014-11-21 Thread Daniel Dressler
Ah thanks David for looking at this. Sorry for the thin paragraphs my vim was warming too early about long lines. I will reformat it to break at 74 chars. No problem, I'll redo everything so it is one function per patch. Now fair warning: there are about 102 functions to cleanup. I was a bit

Re: [PATCH] Btrfs: disk-io: replace root args iff only fs_info used

2014-11-21 Thread David Sterba
On Fri, Nov 21, 2014 at 05:15:07PM +0900, Daniel Dressler wrote: This is the 3rd independent patch of a larger project to cleanup btrfs's internal usage of btrfs_root. Many functions take btrfs_root only to grab the fs_info struct. By requiring a root these functions cause programmer

Re: [PATCH] Btrfs: disk-io: replace root args iff only fs_info used

2014-11-21 Thread Daniel Dressler
Thank you David this is helpful feedback. What would a cover letter be like? Would that be a separate email to the list, or maybe the first email in a patch series? Sorry I've twice looked for the integration repo. I found some that look like it could be but those had older commits. Could you

Re: [PATCH v2 4/5] btrfs: don't allow -C or +c chattrs on a swap file

2014-11-21 Thread David Sterba
On Fri, Nov 21, 2014 at 02:08:30AM -0800, Omar Sandoval wrote: @@ -293,14 +293,21 @@ static int btrfs_ioctl_setflags(struct file *file, void __user *arg) } } else { You can put the condition here, instead of shifting the nested block. } else if

Re: scrub implies failing drive - smartctl blissfully unaware

2014-11-21 Thread Chris Murphy
On Fri, Nov 21, 2014 at 5:55 AM, Ian Armstrong bt...@iarmst.co.uk wrote: In my situation what I've found is that if I scrub let it fix the errors then a second pass immediately after will show no errors. If I then leave it a few days try again there will be errors, even in old files which

Re: scrub implies failing drive - smartctl blissfully unaware

2014-11-21 Thread Zygo Blaxell
On Fri, Nov 21, 2014 at 09:05:32AM +0200, Brendan Hide wrote: On 2014/11/21 06:58, Zygo Blaxell wrote: You have one reallocated sector, so the drive has lost some data at some time in the last 49000(!) hours. Normally reallocations happen during writes so the data that was lost was data you

Re: BTRFS messes up snapshot LV with origin

2014-11-21 Thread Zygo Blaxell
On Fri, Nov 21, 2014 at 06:22:57AM +, Duncan wrote: After all, an LVM block-level snapshot takes the same space as a file containing the same raw data, and if there's room for the data in an LVM snapshot, given a different layout, there's room for exactly the same amount of data as a

Re: [PATCH v2 5/5] btrfs: enable swap file support

2014-11-21 Thread David Sterba
On Fri, Nov 21, 2014 at 02:08:31AM -0800, Omar Sandoval wrote: Implement the swap file a_ops on btrfs. Activation simply checks for a usable swap file: it must be fully allocated (no holes), support direct I/O (so no compressed or inline extents) and should be nocow (I'm not sure about that

Re: scrub implies failing drive - smartctl blissfully unaware

2014-11-21 Thread Chris Murphy
On Fri, Nov 21, 2014 at 10:42 AM, Zygo Blaxell zblax...@furryterror.org wrote: I run 'smartctl -t long' from cron overnight (or whenever the drives are most idle). You can also set up smartd.conf to launch the self tests; however, the syntax for test scheduling is byzantine compared to cron

Re: BTRFS messes up snapshot LV with origin

2014-11-21 Thread Chris Murphy
On Thu, Nov 20, 2014 at 11:22 PM, Duncan 1i5t5.dun...@cox.net wrote: When I have such a filesystem level problem, I simply dd from the backing device to some other location, generally to a file that's on a different filesystem (preferrably non-btrfs, I use reiserfs as I've found it very

Re: [PATCH v2 5/5] btrfs: enable swap file support

2014-11-21 Thread Filipe David Manana
On Fri, Nov 21, 2014 at 6:00 PM, David Sterba dste...@suse.cz wrote: On Fri, Nov 21, 2014 at 02:08:31AM -0800, Omar Sandoval wrote: Implement the swap file a_ops on btrfs. Activation simply checks for a usable swap file: it must be fully allocated (no holes), support direct I/O (so no

[PATCH] Btrfs: make sure logged extents complete in the current transaction V3

2014-11-21 Thread Josef Bacik
Liu Bo pointed out that my previous fix would lose the generation update in the scenario I described. It is actually much worse than that, we could lose the entire extent if we lose power right after the transaction commits. Consider the following write extent 0-4k log extent in log tree commit

[PATCH 1/4] fs: split update_time() into update_time() and write_time()

2014-11-21 Thread Theodore Ts'o
In preparation for adding support for the lazytime mount option, we need to be able to separate out the update_time() and write_time() inode operations. Currently, only btrfs and xfs uses update_time(). We needed to preserve update_time() because btrfs wants to have a special

[PATCH 2/4] vfs: add support for a lazytime mount option

2014-11-21 Thread Theodore Ts'o
Add a new mount option which enables a new lazytime mode. This mode causes atime, mtime, and ctime updates to only be made to the in-memory version of the inode. The on-disk times will only get updated when (a) if the inode needs to be updated for some non-time related change, (b) if userspace

[PATCH 3/4] vfs: don't let the dirty time inodes get more than a day stale

2014-11-21 Thread Theodore Ts'o
Guarantee that the on-disk timestamps will be no more than 24 hours stale. Signed-off-by: Theodore Ts'o ty...@mit.edu --- fs/fs-writeback.c | 1 + fs/inode.c | 7 ++- include/linux/fs.h | 1 + 3 files changed, 8 insertions(+), 1 deletion(-) diff --git a/fs/fs-writeback.c

Re: [PATCH 1/4] fs: split update_time() into update_time() and write_time()

2014-11-21 Thread Chris Mason
On Fri, Nov 21, 2014 at 2:59 PM, Theodore Ts'o ty...@mit.edu wrote: In preparation for adding support for the lazytime mount option, we need to be able to separate out the update_time() and write_time() inode operations. Currently, only btrfs and xfs uses update_time(). We needed to preserve

Re: [PATCH 3/4] vfs: don't let the dirty time inodes get more than a day stale

2014-11-21 Thread Andreas Dilger
On Nov 21, 2014, at 1:59 PM, Theodore Ts'o ty...@mit.edu wrote: Guarantee that the on-disk timestamps will be no more than 24 hours stale. Signed-off-by: Theodore Ts'o ty...@mit.edu --- fs/fs-writeback.c | 1 + fs/inode.c | 7 ++- include/linux/fs.h | 1 + 3 files changed, 8

Re: scrub implies failing drive - smartctl blissfully unaware

2014-11-21 Thread Robert White
On 11/21/2014 07:11 AM, Phillip Susi wrote: On 11/20/2014 5:45 PM, Robert White wrote: If you search for ACPI ide you'll find people complaining in 2008-2010 about windows error messages indicating the device is present in their system but no OS driver is available. Nope... not finding it.

Re: [PATCH 3/4] vfs: don't let the dirty time inodes get more than a day stale

2014-11-21 Thread Theodore Ts'o
On Fri, Nov 21, 2014 at 02:19:07PM -0600, Andreas Dilger wrote: - if (inode-i_sb-s_flags MS_LAZYTIME) { + if ((inode-i_sb-s_flags MS_LAZYTIME) + (!inode-i_ts_dirty_day || +inode-i_ts_dirty_day == days_since_boot)) { spin_lock(inode-i_lock);

Re: scrub implies failing drive - smartctl blissfully unaware

2014-11-21 Thread Robert White
On 11/21/2014 01:12 PM, Robert White wrote: (wrong links included in post...) Dangit... those two links were bad... wrong clipboard... /sigh... I'll just stand on the pasted text from the driver. 8-) -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a

Re: [PATCH 1/4] fs: split update_time() into update_time() and write_time()

2014-11-21 Thread Theodore Ts'o
Out of curiosity, why does btrfs_update_time() need to call btrfs_root_readonly()? Why can't it just depend on the __mnt_want_write() call in touch_atime()? Surely if there are times when it's not OK to write into a btrfs file system and mnt_is_readonly() returns false, the VFS is going to get

[RFC][PATCH] dm: add dm-power-fail target

2014-11-21 Thread Josef Bacik
Hello, I'm hoping some FS guys can weigh in and verify my approach for testing power fail conditions, and the DM guys to of course verify I didn't completely fail at making a DM target. All suggestions welcome, I want to have a nice robust tool for testing power fail so we can be sure our

Re: BTRFS messes up snapshot LV with origin

2014-11-21 Thread Duncan
Chris Murphy posted on Fri, 21 Nov 2014 11:23:45 -0700 as excerpted: On Thu, Nov 20, 2014 at 11:22 PM, Duncan 1i5t5.dun...@cox.net wrote: When I have such a filesystem level problem, I simply dd from the backing device to some other location, generally to a file that's on a different

Re: BTRFS messes up snapshot LV with origin

2014-11-21 Thread Duncan
Zygo Blaxell posted on Fri, 21 Nov 2014 12:56:23 -0500 as excerpted: It's not a bug as long as I can completely control which devices are searched for UUIDs, and the system behaves sanely when multiple UUIDs are found through automatic discovery; otherwise, it's not only a bug, it's a DoS

Re: BTRFS messes up snapshot LV with origin

2014-11-21 Thread Duncan
Duncan posted on Fri, 21 Nov 2014 22:49:06 + as excerpted: Chris Murphy posted... That's not true for thin volume snapshots. They take up next to no space upon creation, they don't need space reserved in advance. Thus the mention of compression if necessary. Thin-volume snapshots are

Re: [PATCH] Btrfs: disk-io: replace root args iff only fs_info used

2014-11-21 Thread David Sterba
On Sat, Nov 22, 2014 at 01:37:10AM +0900, Daniel Dressler wrote: What would a cover letter be like? Would that be a separate email to the list, or maybe the first email in a patch series? It's a separate mail that does not carry any diff but an overview of the following patches. The patches are

Re: BTRFS messes up snapshot LV with origin

2014-11-21 Thread Duncan
Duncan posted on Fri, 21 Nov 2014 23:41:49 + as excerpted: Duncan posted on Fri, 21 Nov 2014 22:49:06 + as excerpted: Chris Murphy posted... That's not true for thin volume snapshots. They take up next to no space upon creation, they don't need space reserved in advance. Thus

Re: [PATCH] Btrfs: ctree: reduce args where only fs_info used

2014-11-21 Thread David Sterba
On Sat, Nov 22, 2014 at 01:03:32AM +0900, Daniel Dressler wrote: No problem, I'll redo everything so it is one function per patch. Now fair warning: there are about 102 functions to cleanup. I was a bit worried that many patches would cause too much maintainer overhead but it is no problem for

Re: scrub implies failing drive - smartctl blissfully unaware

2014-11-21 Thread Zygo Blaxell
On Fri, Nov 21, 2014 at 11:06:19AM -0700, Chris Murphy wrote: On Fri, Nov 21, 2014 at 10:42 AM, Zygo Blaxell zblax...@furryterror.org wrote: I run 'smartctl -t long' from cron overnight (or whenever the drives are most idle). You can also set up smartd.conf to launch the self tests;

Re: scrub implies failing drive - smartctl blissfully unaware

2014-11-21 Thread Ian Armstrong
On Fri, 21 Nov 2014 10:45:21 -0700 Chris Murphy wrote: On Fri, Nov 21, 2014 at 5:55 AM, Ian Armstrong bt...@iarmst.co.uk wrote: In my situation what I've found is that if I scrub let it fix the errors then a second pass immediately after will show no errors. If I then leave it a few