Re: btrfs_direct_IO oops

2016-10-10 Thread Dave Jones
On Mon, Oct 10, 2016 at 04:43:57AM +0100, Al Viro wrote: > Very interesting. Could you slap something like > diff --git a/lib/iov_iter.c b/lib/iov_iter.c > index 0ce3411..1ef00e7 100644 > --- a/lib/iov_iter.c > +++ b/lib/iov_iter.c > @@ -682,8 +682,9 @@ static void pipe_advance(struct iov_

Re: Is is possible to submit binary image as fstest test case?

2016-10-09 Thread Dave Chinner
On Fri, Oct 07, 2016 at 06:05:51PM +0200, David Sterba wrote: > On Fri, Oct 07, 2016 at 08:18:38PM +1100, Dave Chinner wrote: > > On Thu, Oct 06, 2016 at 04:12:56PM +0800, Qu Wenruo wrote: > > > So I'm wondering if I can just upload a zipped raw image as part

Re: Making statfs return qgroup info (for container use case etc.)

2016-10-09 Thread Dave Chinner
On Fri, Oct 07, 2016 at 06:58:47PM +0200, David Sterba wrote: > On Fri, Oct 07, 2016 at 09:40:11AM +1100, Dave Chinner wrote: > > XFS does this with directory tree quotas. It was implmented 10 years > > ago or so, IIRC... > > Sometimes, the line between a historical remark

Re: btrfs_direct_IO oops

2016-10-09 Thread Dave Jones
On Sat, Oct 08, 2016 at 07:20:08PM -0400, Dave Jones wrote: > On Sat, Oct 08, 2016 at 07:29:03PM +0100, Al Viro wrote: > > On Sat, Oct 08, 2016 at 02:08:06PM -0400, Dave Jones wrote: > > > That code: matches this dissembly: > > > > > >

Re: btrfs_direct_IO oops

2016-10-08 Thread Dave Jones
On Sat, Oct 08, 2016 at 07:29:03PM +0100, Al Viro wrote: > On Sat, Oct 08, 2016 at 02:08:06PM -0400, Dave Jones wrote: > > That code: matches this dissembly: > > > > for (i = seg + 1; i < iter->nr_segs; i++) { > > *whoa* > > OK, t

btrfs_direct_IO oops

2016-10-08 Thread Dave Jones
Found this in logs this morning. First time I've seen this one. Might be related to some direct IO related changes I made in Trinity that is tickling some new path. Oops: [#1] PREEMPT SMP DEBUG_PAGEALLOC CPU: 2 PID: 25313 Comm: trinity-c18 Not tainted 4.8.0-think+ #7 task: 88040f7b1c00 t

Re: [PATCH] generic: make 17[1-4] work well when btrfs compression is enabled

2016-10-07 Thread Dave Chinner
1 > +dd if=/dev/zero of=$testdir/eat_my_space >> $seqres.full 2>&1 Please don't replace xfs_io writes using a specific data pattern with dd calls that write zeros. Indeed, we don't use dd for new tests anymore - xfs_io should be used. Write a function that fills all the re

Re: Is is possible to submit binary image as fstest test case?

2016-10-07 Thread Dave Chinner
On Fri, Oct 07, 2016 at 05:26:27PM +0800, Qu Wenruo wrote: > > > At 10/07/2016 05:18 PM, Dave Chinner wrote: > >On Thu, Oct 06, 2016 at 04:12:56PM +0800, Qu Wenruo wrote: > >>Hi, > >> > >>Just as the title says, for some case(OK, btrfs again) we need to

Re: Is is possible to submit binary image as fstest test case?

2016-10-07 Thread Dave Chinner
works or debug-only sysfs hooks are for. The XFS kernel code has both, xfstests use both, and they pretty much do away with the need for custom binary filesystem images for such testing... Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "

Re: Making statfs return qgroup info (for container use case etc.)

2016-10-06 Thread Dave Chinner
; (e.g. by passing in an option at mount time - such as qgroup level > maybe?) , instead of the global filesystem data in f_bfree f_blocks etc. XFS does this with directory tree quotas. It was implmented 10 years ago or so, IIRC... Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from

Re: Problem with btrfs_create() getting EBUSY; cause is stale highest_objectid at mount - kernel.org bug 173001

2016-10-06 Thread Dave Olson
Dave Olson wrote: > Dave Olson wrote: > > > The full details are in > > https://bugzilla.kernel.org/show_bug.cgi?id=173001 > > > > Basicly, logrotate has rotated a file to a new name, tries to open a new > > file with the original name, and gets EBUSY. Th

Re: Problem with btrfs fs in 4.1.25 (also 4.8.0) - getting EBUSY on file create - kernel.org bug 173001

2016-10-03 Thread Dave Olson
Dave Olson wrote: > The full details are in > https://bugzilla.kernel.org/show_bug.cgi?id=173001 > > Basicly, logrotate has rotated a file to a new name, tries to open a new > file with the original name, and gets EBUSY. The file is not created. > > Later on the file c

Problem with btrfs fs in 4.1.25 - getting EBUSY on file create - kernel.org bug 173001

2016-09-27 Thread Dave Olson
st of the BTRFS config options, and there are no BTRFS messages related to the failure. The file is on the root filesystem. Dave Olson ol...@cumulusnetworks.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org

Re: [RFC] Preliminary BTRFS Encryption

2016-09-15 Thread Dave Chinner
en only users with permission to unlock the subvolume key can access it. Once the generic file encryption is solid and fulfils the needs of most users, then you can look to solving the less common threat models that neither dmcrypt or per-file encryption address. Only if the generic code cannot be ex

Re: [PATCH v6 1/6] fstests: common: Introduce _post_mount_hook for btrfs

2016-09-14 Thread Dave Chinner
ex 23c007a..631397f 100644 > --- a/common/rc > +++ b/common/rc > @@ -321,6 +321,27 @@ _overlay_scratch_unmount() > $UMOUNT_PROG $SCRATCH_MNT > } > > +_run_btrfs_post_mount_hook() > +{ > + mnt_point=$1 > + for n in $ALWAYS_ENABLE_BTRFS_FEATURE; do What&#

Re: [PATCH 2/3] writeback: allow for dirty metadata accounting

2016-09-12 Thread Dave Chinner
ly hide memcg from the writeback implementations similar to the way memcg is completely hidden from the shrinker reclaim implementations... Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/3] writeback: allow for dirty metadata accounting

2016-09-12 Thread Dave Chinner
On Mon, Sep 12, 2016 at 10:24:12AM -0400, Josef Bacik wrote: > Dave your reply got eaten somewhere along the way for me, so all i > have is this email. I'm going to respond to your stuff here. No worries, I'll do a 2-in-1 reply :P > On 09/12/2016 03:34 AM, Jan Kara wrote: &g

Re: [PATCH 2/3] writeback: allow for dirty metadata accounting

2016-09-11 Thread Dave Chinner
how tracking of information such as the global amount of dirty metadata is useful for diagnostics, but I'm not convinced we should be using it for globally scoped external control of deeply integrated and highly specific internal filesystem functionality. Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/3] ioctl_xfs_ioc_getfsmap.2: document XFS_IOC_GETFSMAP ioctl

2016-09-09 Thread Dave Chinner
On Thu, Sep 08, 2016 at 11:07:16PM -0700, Darrick J. Wong wrote: > On Fri, Sep 09, 2016 at 09:38:06AM +1000, Dave Chinner wrote: > > On Tue, Aug 30, 2016 at 12:09:49PM -0700, Darrick J. Wong wrote: > > > > I recall for FIEMAP that some filesystems may not have files alig

Re: lockdep warning in btrfs in 4.8-rc3

2016-09-08 Thread Dave Jones
s so I could see if this was corruption or two unrelated keys. I'll make it do that in case it happens again. Dave -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/3] ioctl_xfs_ioc_getfsmap.2: document XFS_IOC_GETFSMAP ioctl

2016-09-08 Thread Dave Chinner
k I like this better. Everyone else, please chime in. :) That's pretty much the structure I was going to suggest - it matches the fiemap pattern. i.e control parameters are separated from record data. I'd dump a bit more reserved space in the structure, though; we've got heaps o

Re: [PATCH] fstests: common: Enhance _exclude_scratch_mount_option to handle multiply options and generic fs type

2016-09-06 Thread Dave Chinner
hy we review changes. If it's not obvious to the reviewer why the mount option is excluded, or it's not documented in the commit message, then the reviewer should be asking for it to be added. Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the l

Re: [PATCH] fstests: common: Enhance _exclude_scratch_mount_option to handle multiply options and generic fs type

2016-09-06 Thread Dave Chinner
gt; { > for opt in $*; do > if echo $MOUNT_OPTIONS | grep -qw "$opt"; then > _notrun "mount option \"$opt\" not allowed in this > test" > fi > done > } > > (Note that the c

Re: Will Btrfs have an official command to "uncow" existing files?

2016-08-22 Thread Dave Chinner
On Mon, Aug 22, 2016 at 04:06:13PM -0700, Darrick J. Wong wrote: > [add Dave and Christoph to cc] > > On Mon, Aug 22, 2016 at 04:14:19PM -0400, Jeff Mahoney wrote: > > On 8/21/16 2:59 PM, Tomokhov Alexander wrote: > > > Btrfs wiki FAQ gives a link to example Pyt

Re: checksum error in metadata node - best way to move root fs to new drive?

2016-08-11 Thread Dave T
What I have gathered so far is the following: 1. my RAM is not faulty and I feel comfortable ruling out a memory error as having anything to do with the reported problem. 2. my storage device does not seem to be faulty. I have not figured out how to do more definitive testing, but smartctl report

Re: checksum error in metadata node - best way to move root fs to new drive?

2016-08-10 Thread Dave T
Apologies. I have to make a correction to the message I just sent. Disregard that message and use this one: On Wed, Aug 10, 2016 at 5:15 PM, Chris Murphy wrote: > 1. Report 'btrfs check' without --repair, let's see what it complains > about and if it might be able to plausibly fix this. First,

Re: checksum error in metadata node - best way to move root fs to new drive?

2016-08-10 Thread Dave T
see below On Wed, Aug 10, 2016 at 5:15 PM, Chris Murphy wrote: > 1. Report 'btrfs check' without --repair, let's see what it complains > about and if it might be able to plausibly fix this. First, a small part of the dmesg output: [ 172.772283] Btrfs loaded [ 172.772632] BTRFS: device label

Re: checksum error in metadata node - best way to move root fs to new drive?

2016-08-10 Thread Dave T
Thanks for all the responses, guys! I really appreciate it. This information is very helpful. I will be working through the suggestions (e.g., check without repair) for the next hour or so. I'll report back when I have something to report. My drive is a Samsung 950 Pro nvme drive, which in most re

checksum error in metadata node - best way to move root fs to new drive?

2016-08-09 Thread Dave T
btrfs scrub returned with uncorrectable errors. Searching in dmesg returns the following information: BTRFS warning (device dm-0): checksum error at logical N on /dev/mapper/[crypto] sector: y metadata node (level 2) in tree 250 it also says: unable to fixup (regular) error at logical NN

btrfs check --repair error: check_owner_ref: Assertion `rec->is_root` failed.

2016-08-09 Thread Dave T
m-crypt? Look elsewhere for help? On Tue, Aug 9, 2016 at 6:54 PM, Dave T wrote: > The original problem from 2 days ago just happened again. I ran btrfs > rescue zero-log (again) and the root filesystem mounted but it was > read-only on first boot. I rebooted again and everything seems normal.

Re: system locked up with btrfs-transaction consuming 100% CPU

2016-08-09 Thread Dave T
Linux btrfs-progs v4.6.1 On Tue, Aug 9, 2016 at 2:07 PM, Dave T wrote: > My system locked up with btrfs-transaction consuming 100% CPU and NMI > watchdog reporting BUG: soft lockup with btrfs-transaction:314. > > This comes 2 days after a serious event involving BTRFS where my > sy

Re: system locked up with btrfs-transaction consuming 100% CPU

2016-08-09 Thread Dave T
had to troubleshoot it in the past. Now (because of 4 years of problem-free operation) I'm using it on a critical production system. I have backups, but I cannot allow these problems to go unresolved. On Tue, Aug 9, 2016 at 5:32 PM, Duncan <1i5t5.dun...@cox.net> wrote: > Dave T poste

system locked up with btrfs-transaction consuming 100% CPU

2016-08-09 Thread Dave T
However, I have no clue about what the problem might be (except that it seemingly involves btrfs). What other information can I provide? On Sun, Aug 7, 2016 at 6:44 PM, Dave wrote: > I am running Arch Linux on a system with full disk encryption and the > storage is a Samsung 950 Pro NVMe dri

Re: [PATCH 13/17] xfs: test swapext with reflink

2016-08-08 Thread Dave Chinner
entire file and counts lines. > Seeing as XFS records the extent count in the inode, we might as well use it. perhaps put a special xfs case in _count_extents() that does this rather than FIEMAP? Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

btrfs open_ctree failed - root fs will not mount

2016-08-07 Thread Dave
I am running Arch Linux on a system with full disk encryption and the storage is a Samsung 950 Pro NVMe drive (512 GB). The computer is a couple months old. No bad behavior until now. (I'm only using 21 GB of the 512 space on the disk.) btrfs-progs v4.5.1 Today I was using my system normally

Re: [4.8] btrfs heats my room with lock contention

2016-08-04 Thread Dave Chinner
On Thu, Aug 04, 2016 at 10:28:44AM -0400, Chris Mason wrote: > > > On 08/04/2016 02:41 AM, Dave Chinner wrote: > > > >Simple test. 8GB pmem device on a 16p machine: > > > ># mkfs.btrfs /dev/pmem1 > ># mount /dev/pmem1 /mnt/scratch > ># dbench -t 60

[4.8] btrfs heats my room with lock contention

2016-08-03 Thread Dave Chinner
, they only drop to ~1500-2000MB/s as they hit internal limits. Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org

Re: A lot warnings in dmesg while running thunderbird

2016-07-24 Thread Dave Chinner
this into an xfstests regression test and submit it? Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] generic/371: run write(2) and fallocate(2) in parallel

2016-07-21 Thread Dave Chinner
8 auto quick richacl > > 369 auto quick richacl > > 370 auto quick richacl > > +371 auto enospc prealloc stress > > So we can add 'quick' group and remove 'stress'. I'd leave the test in stress if we can run it based on time as per my above sugges

Re: [PATCH] fstests: btrfs: Test send on heavily deduped file

2016-07-21 Thread Dave Chinner
On Thu, Jul 21, 2016 at 10:05:25AM +0800, Qu Wenruo wrote: > > > At 07/21/2016 07:37 AM, Dave Chinner wrote: > >On Wed, Jul 20, 2016 at 03:40:29PM +0800, Qu Wenruo wrote: > >>At 07/20/2016 03:01 PM, Eryu Guan wrote: > >>>On Tue, Jul 19, 2016

Re: [PATCH] fstests: btrfs: Test send on heavily deduped file

2016-07-20 Thread Dave Chinner
nish in about 5min with default mount > option, but for 'nodatasum' it can take up to 2 hours. > So should it belong to 'auto'? Yes. Also, keep in mind that runtime is dependent on the type of storage you are testing on. The general idea is that the "< 30s quick, < 5m auto" rule is based on how long the test takes to run on a local single spindle SATA drive, as that is the basic hardware we'd expect people to be testing against. This means that a test that takes 20s on your SSD might not be a "quick" test because it takes 5 minutes on spinning rust Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH] fstests: btrfs: Test send on heavily deduped file

2016-07-20 Thread Dave Chinner
On Wed, Jul 20, 2016 at 03:01:00PM +0800, Eryu Guan wrote: > For running tests, "./check -g auto -x dangerous" might fit your need. Yes, that's precisely the way the dangerous group is intended to be used: as a exclusion filter that gets applied to other test group definitio

Re: [PATCH 19/20] xfs: run xfs_repair at the end of each test

2016-07-06 Thread Dave Chinner
test device takes 10s, this means the test harness overhead increases by a factor of 3. i.e. test takes 1s to run, checking the filesystem between tests now takes 30s. i.e. this will badly blow out the run time of the test suite on aged test devices What does this overhead actually gain u

Re: sharing page cache pages between multiple mappings

2016-05-19 Thread Dave Chinner
up. At > that point we switch to sharing pages with the read-write copy. Unless I'm missing something here (quite possible!), I'm not sure we can fix that problem with page cache sharing or reflink. It implies we are sharing pages in a downwards direction - private overlay pages/mappin

Re: [PATCH] fstests: btrfs: Test fiemap ioctl on completely deduped file

2016-05-11 Thread Dave Chinner
at large. e.g. tests/xfs/136.out has 7800 lines in its golden output file. There are quite a few tests with large amounts of output: $ find . -name *.out -exec ls -s {} \; |sort -nr |head -5 144 ./tests/xfs/136.out 124 ./tests/generic/324.out 120 ./tests/xfs/165.out 116 ./tests/xfs/107.out 92 ./te

Re: [PATCH 0/2] scop GFP_NOFS api

2016-05-03 Thread Dave Chinner
On Sun, May 01, 2016 at 08:19:44AM +1000, NeilBrown wrote: > On Sat, Apr 30 2016, Dave Chinner wrote: > > Indeed, blocking the superblock shrinker in reclaim is a key part of > > balancing inode cache pressure in XFS. If the shrinker starts > > hitting dirty inodes, it bl

Re: [PATCH 0/2] scop GFP_NOFS api

2016-04-29 Thread Dave Chinner
in place, I'd then make the changes to the generic superblock shrinker code to enable finer grained reclaim and optimise the XFS shrinkers to make use of it... Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe li

Re: [PATCH 0/2] scop GFP_NOFS api

2016-04-29 Thread Dave Chinner
n > > context. I haven't converted any of the FS myself because that is way > > beyond my area of expertise but I would be happy to help with further > > changes on the MM front as well as in some more generic code paths. > > > > Dave had an idea on how to further i

Re: [PATCH 2/2] mm, debug: report when GFP_NO{FS,IO} is used explicitly from memalloc_no{fs,io}_{save,restore} context

2016-04-27 Thread Dave Chinner
On Wed, Apr 27, 2016 at 10:03:11AM +0200, Michal Hocko wrote: > On Wed 27-04-16 08:58:45, Dave Chinner wrote: > > On Tue, Apr 26, 2016 at 01:56:12PM +0200, Michal Hocko wrote: > > > From: Michal Hocko > > > > > > THIS PATCH IS FOR TESTING ONLY AND NOT MEANT

Re: [PATCH 1/2] mm: add PF_MEMALLOC_NOFS

2016-04-26 Thread Dave Chinner
#x27;t actually care about in XFS at all. That way I can carry all the XFS changes in the XFS tree and not have to worry about when this stuff gets merged or conflicts with the rest of the work that is being done to the mm/ code and whatever tree that eventually lands in... Cheers, Dave. --

Re: [PATCH 2/2] mm, debug: report when GFP_NO{FS,IO} is used explicitly from memalloc_no{fs,io}_{save,restore} context

2016-04-26 Thread Dave Chinner
ing to restart the flood of false positive lockdep warnings we've silenced over the years, so perhaps lockdep needs to be made smarter as well... Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the b

BTRFS: assertion failed: num_extents, file: fs/btrfs/extent-tree.c, line: 5584

2016-04-20 Thread Dave Jones
Don't think I've reported this one before. It's on the same box I've been seeing the btrfs_destroy_inode WARN_ON's on though. Dave BTRFS: assertion failed: num_extents, file: fs/btrfs/extent-tree.c, line: 5584 [ cut here ] kernel BUG

Re: btrfs_destroy_inode WARN_ON.

2016-04-01 Thread Dave Jones
On Fri, Apr 01, 2016 at 02:12:27PM -0400, Dave Jones wrote: > BUG: workqueue lockup - pool cpus=1 node=0 flags=0x0 nice=0 stuck for 30s! > Showing busy workqueues and worker pools: > workqueue events: flags=0x0 > pwq 6: cpus=3 node=0 flags=0x0 nice=0 active=1/256 &

Re: btrfs_destroy_inode WARN_ON.

2016-04-01 Thread Dave Jones
On Sun, Mar 27, 2016 at 09:14:00PM -0400, Dave Jones wrote: > > WARNING: CPU: 2 PID: 32570 at fs/btrfs/inode.c:9261 > btrfs_destroy_inode+0x389/0x3f0 [btrfs] > > CPU: 2 PID: 32570 Comm: rm Not tainted 4.5.0-think+ #14 > > c039baf9 ef72

Re: fallocate mode flag for "unshare blocks"?

2016-03-31 Thread Dave Chinner
On Thu, Mar 31, 2016 at 06:34:17PM -0400, J. Bruce Fields wrote: > On Fri, Apr 01, 2016 at 09:20:23AM +1100, Dave Chinner wrote: > > On Thu, Mar 31, 2016 at 01:47:50PM -0600, Andreas Dilger wrote: > > > On Mar 31, 2016, at 12:08 PM, J. Bruce Fields > > > wrote: >

Re: fallocate mode flag for "unshare blocks"?

2016-03-31 Thread Dave Chinner
On Thu, Mar 31, 2016 at 01:47:50PM -0600, Andreas Dilger wrote: > On Mar 31, 2016, at 12:08 PM, J. Bruce Fields wrote: > > > > On Thu, Mar 31, 2016 at 10:18:50PM +1100, Dave Chinner wrote: > >> On Thu, Mar 31, 2016 at 12:54:40AM -0700, Christoph Hellwig wrote: > >&g

Re: fallocate mode flag for "unshare blocks"?

2016-03-31 Thread Dave Chinner
On Thu, Mar 31, 2016 at 12:54:40AM -0700, Christoph Hellwig wrote: > On Thu, Mar 31, 2016 at 12:18:13PM +1100, Dave Chinner wrote: > > On Wed, Mar 30, 2016 at 11:27:55AM -0700, Darrick J. Wong wrote: > > > Or is it ok that fallocate could block, potentially for a long time as &g

Re: fallocate mode flag for "unshare blocks"?

2016-03-30 Thread Dave Chinner
a long time. Yes, it's perfectly fine for fallocate to block for long periods of time. See what gfs2 does during preallocation of blocks - it ends up calling sb_issue_zerout() because it doesn't have unwritten extents, and hence can block for long periods of time.... Cheers, Dav

Re: good documentation on btrfs internals and on disk layout

2016-03-30 Thread Dave Stevens
Quoting Liu Bo : On Wed, Mar 30, 2016 at 01:58:03PM +, sri wrote: Hi, I could find very limited documentation related to on disk layout of btrfs and how all trees are related to each other. Except wiki which has very specific top level details I couldn't able to find more details on btrfs.

Re: btrfs_destroy_inode WARN_ON.

2016-03-27 Thread Dave Jones
On Thu, Mar 24, 2016 at 06:54:11PM -0400, Dave Jones wrote: > Just hit this on a tree from earlier this morning, v4.5-11140 or so. > > WARNING: CPU: 2 PID: 32570 at fs/btrfs/inode.c:9261 > btrfs_destroy_inode+0x389/0x3f0 [btrfs] > CPU: 2 PID: 32570 Comm: rm Not tainted 4

btrfs_destroy_inode WARN_ON.

2016-03-24 Thread Dave Jones
ut there's no indication in dmesg of anything awry. Spinning rust on SATA, nothing special. Dave -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 06/12] xfs/030: fix output on newer filesystems

2016-03-22 Thread Dave Chinner
e just filter then entire bad agbno in agfl line out? I'm going to commit the change anyway, as this is a separate issue that needs to be solved. Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in th

Re: [PATCH] fstests: _fail the tests if _scratch_mount failed to avoid fully filling root fs

2016-03-22 Thread Dave Chinner
I think the btrfs bug should be fixed. At minimum, the workaround to see if the filesytem can be mounted should be in btrfs's implementation of scratch_mkfs_sized Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs&qu

qgroup code slowing down rebalance

2016-03-19 Thread Dave Hansen
I have a medium-sized multi-device btrfs filesystem (4 disks, 16TB total) running under 4.5.0-rc5. I recently added a disk and needed to rebalance. I started a rebalance operation three days ago. It was on the order of 20% done after those three days. :) During this rebalance, the disks were pr

Re: qgroup code slowing down rebalance

2016-03-19 Thread Dave Hansen
On 03/17/2016 06:02 PM, Qu Wenruo wrote: > Dave Hansen wrote on 2016/03/17 09:36 -0700: >> On 03/16/2016 06:36 PM, Qu Wenruo wrote: >>> Dave Hansen wrote on 2016/03/16 13:53 -0700: >>>> I have a medium-sized multi-device btrfs filesystem (4 disks, 16TB >>&

Re: [PATCH v2] fstest: btrfs: test single 4k extent after subpagesize buffered writes

2016-03-19 Thread Dave Chinner
On Wed, Mar 16, 2016 at 09:13:57PM -0700, Liu Bo wrote: > On Tue, Mar 15, 2016 at 02:39:41PM +1100, Dave Chinner wrote: > > On Mon, Mar 07, 2016 at 04:27:59PM -0800, Liu Bo wrote: > > > This is to test if COW enabled btrfs can end up with single 4k extents > > > when

Re: qgroup code slowing down rebalance

2016-03-19 Thread Dave Hansen
On 03/16/2016 06:36 PM, Qu Wenruo wrote: > Dave Hansen wrote on 2016/03/16 13:53 -0700: >> I have a medium-sized multi-device btrfs filesystem (4 disks, 16TB >> total) running under 4.5.0-rc5. I recently added a disk and needed to >> rebalance. I started a rebalance operatio

Re: [PATCH 12/12] block: test fallocate for block devices

2016-03-14 Thread Dave Chinner
t just to a new bdev group. I think it's the right place to test it - we have all the infrastructure available to do it (i.e. xfs_io and various block devices) and we really need to make sure this stuff works, especially if we start to write filesystem code that depends on correct behaviour...

Re: [PATCH v2] fstest: btrfs: test single 4k extent after subpagesize buffered writes

2016-03-14 Thread Dave Chinner
> +$XFS_IO_PROG -c "fiemap -v" $tfile | awk '{if ($4 == 8) print $4}' Assumes page size is 4k. Also assumes that no individual extent in the file is 4k. Likely broken. > +# success, all done > +status=0 > +exit > diff --git a/tests/btrfs/027.out b/tests

Re: [PATCH v4 0/6] Btrfs in-band de-duplication test cases

2016-03-14 Thread Dave Chinner
ps to make it clear > which dedupe feature each test is aiming to validate? So just name the two groups appropriately: "ib-dedupe" and "oob-dedupe" or something like that. Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 RESEND 2/5] fstests: btrfs: Add basic test for btrfs in-band de-duplication

2016-02-28 Thread Dave Chinner
On Mon, Feb 29, 2016 at 10:04:35AM +0800, Qu Wenruo wrote: > Hi Dave, > > Thanks for the review. > > All comment are correct and I'll update the patchset soon. > > Only one small question below > > Dave Chinner wrote on 2016/02/29 09:26 +1100: > ... > &g

Re: [PATCH 1/2] fstests: btrfs/011 use replace_options

2016-02-28 Thread Dave Chinner
lid test. IOWs, it is up to the developers who use xfstests to make sure the tests relevant to their areas of expertise work correctly and are valid/viable tests, not me. To avoid this sort of issue in future, please review changes promptly. Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To

Re: [PATCH v2 RESEND 5/5] fstests: btrfs: Test inband dedup with balance.

2016-02-28 Thread Dave Chinner
rfs_util_prog balance cancel $SCRATCH_MNT &> /dev/null This needs to be in the cleanup function. Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 RESEND 4/5] fstests: btrfs: Add per inode dedup flag test

2016-02-28 Thread Dave Chinner
SCRATCH_MNT/dedup_file $file_size > +dedup_write_file $SCRATCH_MNT/no_dedup_file $file_size > +dedup_write_file $SCRATCH_MNT/dedup_dir/dedup_dir_default_file $file_size > +dedup_write_file $SCRATCH_MNT/no_dedup_dir/no_dedup_dir_default_file > $file_size > + > +print_result $SCRATC

Re: [PATCH v2 RESEND 3/5] fstests: btrfs: Add testcase for btrfs dedup enable disable race test

2016-02-28 Thread Dave Chinner
ather than having to wait for fsstress to complete. Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 RESEND 2/5] fstests: btrfs: Add basic test for btrfs in-band de-duplication

2016-02-28 Thread Dave Chinner
> + > + # Also check the md5sum to ensure data is not corrupted > + md5=$(_md5_checksum $SCRATCH_MNT/real_file) > + if [ $md5 != $init_md5 ]; then > + echo "File after in-band de-duplication is corrupted" > + fi Nope. Just echo the md5sum to the go

Re: [PATCH 1/2] fstests: generic test for directory fsync after rename operation

2016-02-17 Thread Dave Chinner
rectory a/b/" > +[ -e $SCRATCH_MNT/c/foo ] || echo "File foo is not at directory c/" > + > +# The new file named bar should also exist. > +[ -e $SCRATCH_MNT/a/bar ] || echo "File bar is missing" This can all be replaced simply by: ls -R $SCRATCH_MNT | _filter_sc

Re: [PATCH v4.2 00/32] xfstests: test the nfs/cifs/btrfs/xfs reflink/dedupe ioctls

2016-02-12 Thread Dave Chinner
On Thu, Feb 11, 2016 at 03:39:16PM -0800, Darrick J. Wong wrote: > Dave Chinner: I've renumbered the new tests and pushed to github[3] if > you'd like to pull. See the pull request at the end of this message. > > This is a patch set against the reflink/dedupe test cases in

Re: [PATCH 12/32] dio: unwritten conversion bug tests

2016-02-12 Thread Dave Chinner
On Fri, Feb 12, 2016 at 10:22:31AM -0500, Theodore Ts'o wrote: > On Fri, Feb 12, 2016 at 02:52:53PM +1100, Dave Chinner wrote: > > On Thu, Feb 11, 2016 at 03:40:37PM -0800, Darrick J. Wong wrote: > > > Check that we don't expose old disk contents when a directio write

Re: [PATCH 12/32] dio: unwritten conversion bug tests

2016-02-11 Thread Dave Chinner
tion 'main': aiocp.c:407:1: warning: format '%d' expects argument of type 'int', but argument 3 has type 'off_t' [-Wformat=] printf("tocopy=%d len=%d blk=%d\n", tocopy, length, aio_blksize); ^ Followup patch is fine, I'll commit as is. Chee

Re: [PATCH 12/23] xfs/122: support refcount/rmap data structures

2016-02-09 Thread Dave Chinner
On Mon, Feb 08, 2016 at 11:55:06PM -0800, Darrick J. Wong wrote: > On Tue, Feb 09, 2016 at 06:43:30PM +1100, Dave Chinner wrote: > > On Mon, Feb 08, 2016 at 05:13:03PM -0800, Darrick J. Wong wrote: > > > Include the refcount and rmap structures in the golden output. > >

Re: [PATCH 21/23] xfs: aio cow tests

2016-02-09 Thread Dave Chinner
" -f -c "pwrite -S 0x63 -b $((blksz * bsz)) 0 $((blksz * nr))" > "$TEST_DIR/moo" >> "$seqres.full" offset = block size times block size? I think some better names might be needed... Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscr

Re: [PATCH 19/23] xfs: test rmapbt functionality

2016-02-09 Thread Dave Chinner
t; +_supported_os Linux > +_supported_fs xfs > +_require_xfs_scratch_rmapbt > + > +echo "Format and mount" > +_scratch_mkfs -d size=$((2 * 4096 * 4096)) -l size=4194304 > "$seqres.full" > 2>&1 > +_scratch_mount >> "$seqres.full" 2&g

Re: [PATCH 18/23] xfs: test the automatic cowextsize extent garbage collector

2016-02-09 Thread Dave Chinner
blks" >> "$seqres.full" > +test $new_extents -lt $((internal_blks / 7)) || _fail "file2 badly > fragmented" I wouldn't use _fail like this, echo is sufficient to cause the test to fail. > +echo "Check for damage" > +umount "$SCRATCH_MNT&

Re: [PATCH 17/23] reflink: test CoW across a mixed range of block types with cowextsize set

2016-02-09 Thread Dave Chinner
1 $((blksz * f)) $blksz "$testdir/file3.chk" >> > "$seqres.full" > +done This looks like several tests use this setup. Factor? > +_scratch_remount > + > +echo "Compare files" > +md5sum "$testdir/file1" | _filter_scratch > +md5sum

Re: [PATCH 13/23] xfs: test fragmentation characteristics of copy-on-write

2016-02-09 Thread Dave Chinner
;file2 cowextsize > not set" > + > +echo "Set cowextsize and check flag" > +"$XFS_IO_PROG" -f -c "cowextsize 1048576" "$testdir/file3" | _filter_scratch > +_scratch_remount > +test $("$XFS_IO_PROG" -c "stat" "

Re: [PATCH 06/23] dio unwritten conversion bug tests

2016-02-08 Thread Dave Chinner
d the rw group because it's testing IO. Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 12/23] xfs/122: support refcount/rmap data structures

2016-02-08 Thread Dave Chinner
> xfs_name > +xfs_owner_info > +xfs_refcount_irec > +xfs_rmap_irec > xfs_alloctype_t > xfs_buf_cancel_t > xfs_bmbt_rec_32_t So this is going to cause failures on any userspace that doesn't know about these new types, right? Should these be conditional in some way? Cheers, Dave.

Re: [PATCH 10/23] xfs: more reflink tests

2016-02-08 Thread Dave Chinner
gt; +status=0 > +exit This is what _require_scratch_nocheck avoids. i.e. do this instead: _require_scratch_nocheck ..... "$XFS_REPAIR_PROG" -o force_geometry -n "$SCRATCH_DEV" >> "$seqres.full" 2>&1 status=$? exit Also, we really don't n

Re: [PATCH v4.1 00/23] xfstests: test the nfs/cifs/btrfs/xfs reflink/dedupe ioctls

2016-02-08 Thread Dave Chinner
On Mon, Feb 08, 2016 at 05:11:45PM -0800, Darrick J. Wong wrote: > Happy New Year! > > Dave Chinner: I've renumbered the new tests and pushed to github[3] if > you'd like to pull. Can you include the commit ID I should see at the head of the tree so I can confirm I'

Re: [PATCH] fstests: btrfs, test for send with clone operations

2016-02-04 Thread Dave Chinner
: _populate_fs() -d 1 -n 1 -f 1000 -s 0 -r $SCRATCH_MNT/a/b/ Can you look into optimising _populate_fs() to use multiple threads (say up to 4 by default) and "echo -n" to create files, and then convert all our open coded "create lots of files" loops in tests to use it? Chee

Re: [PATCH 00/35 v2] separate operations from flags in the bio/request structs

2016-01-06 Thread Dave Chinner
eak now or forever hold your peace" > review deadline? I say just ask Linus to pull it immediately after the next merge window closes Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: [PATCH 13/35] xfs: set bi_op to REQ_OP

2016-01-06 Thread Dave Chinner
e which patches your note is refering to here. The XFS change here looks fine. Acked-by: Dave Chinner -Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Ideas on unified real-ro mount option across all filesystems

2015-12-26 Thread Dave Chinner
the install of each new Irix release, the bootloader was always up-to-date with the on-disk features the kernel supported and so there was never a problem with mismatched feature support. However, Linux users can upgrade or change the kernel at any time independently of the bootloader, so it

Re: [PATCH 1/2] fstests: fix btrfs test failures after commit 27d077ec0bda

2015-12-23 Thread Dave Chinner
l problem that needs fixing. As always, we should be trying to fix the root cause of the problem, not working around them... Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Ideas to do custom operation just after mount?

2015-12-23 Thread Dave Chinner
;I don't understand why qgroup is enabled/disabled by ioctl. :( > > > mount option won't persist across systems/computers unless > remembered by human. So record the mount option you want persistent in the filesystem at mount time and don't turn it off until a &quo

Loss of connection to Half of the drives

2015-12-22 Thread Dave S
root on sdh [ 3000.168649] BTRFS: open_ctree failed [ 3071.430479] btrfs[4701]: segfault at 100108 ip 0046e06a sp 7fff44539688 error 6 in btrfs[40+c7000] [ 3147.025032] btrfs[4755]: segfault at 100108 ip 0044c503 sp 7fffa79bbf40 error 6 in btrfs[40+83000] Sincerely -Da

Re: [PATCH 06/11] reflink: more tests

2015-12-20 Thread Dave Chinner
dir", in lower case, makes it clear that it's a local variable, not the global test device mount point (i.e. test local variables are lower case, exported test harnes variables are upper case ;) Separate patch/pull req is fine. Cheers, dave. -- Dave Chinner da...@fromorbit.com

Re: Ideas to do custom operation just after mount?

2015-12-20 Thread Dave Chinner
support patches, and the rest of it is in my local work area in an incomplete state. Cheers, Dave. -- Dave Chinner da...@fromorbit.com -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: !PageLocked BUG_ON hit in clear_page_dirty_for_io

2015-12-10 Thread Dave Jones
On Thu, Dec 10, 2015 at 05:57:20PM -0500, Dave Jones wrote: > On Thu, Dec 10, 2015 at 04:30:24PM -0500, Chris Mason wrote: > > On Thu, Dec 10, 2015 at 02:35:55PM -0500, Dave Jones wrote: > > > On Thu, Dec 10, 2015 at 02:02:20PM -0500, Chris Mason wrote: > > > &

<    1   2   3   4   5   6   7   8   >