Re: [PATCH 12/17] reflink: test cross-mountpoint reflink and dedupe

2016-08-01 Thread Christoph Hellwig
On Thu, Jul 21, 2016 at 04:47:32PM -0700, Darrick J. Wong wrote: > Test sharing blocks via reflink and dedupe between two different > mountpoints of the same filesystem. This shouldn't work, since > we don't allow cross-mountpoint functions. > > Signed-off-by: Darrick J. Wong

Re: [PATCH 16/17] xfs/122: add the realtime rmapbt inode and btree fields

2016-08-01 Thread Christoph Hellwig
On Thu, Jul 21, 2016 at 04:48:00PM -0700, Darrick J. Wong wrote: > Add the on-disk structures added by the realtime rmapbt. > > Signed-off-by: Darrick J. Wong Looks fine, Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send the line

Re: [PATCH 05/17] common/reflink: actually test dedupe on scratch device

2016-08-01 Thread Christoph Hellwig
On Thu, Jul 21, 2016 at 04:46:48PM -0700, Darrick J. Wong wrote: > In _require_scratch_dedupe, test the scratch device, not the testdev. > > Signed-off-by: Darrick J. Wong Looks fine, Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send

Re: [PATCH 06/17] xfs: run xfs_repair at the end of each test

2016-08-01 Thread Christoph Hellwig
On Thu, Jul 21, 2016 at 04:46:54PM -0700, Darrick J. Wong wrote: > Run xfs_repair twice at the end of each test -- once to rebuild > the btree indices, and again with -n to check the rebuild work. This looks fine to me in general, but shouldn't we have specific tests that test the rebuilding in a

Re: [PATCH 07/17] xfs/128: cycle_mount the scratch device, not the test device

2016-08-01 Thread Christoph Hellwig
On Thu, Jul 21, 2016 at 04:47:01PM -0700, Darrick J. Wong wrote: > This test uses the scratch device, so cycle that, not the test dev. > This is also a xfs_fsr test, so put it in the fsr group. > > Signed-off-by: Darrick J. Wong Looks fine, Reviewed-by: Christoph

Re: [PATCH 09/17] common/dmerror: fix mount option issues

2016-08-01 Thread Christoph Hellwig
On Thu, Jul 21, 2016 at 04:47:13PM -0700, Darrick J. Wong wrote: > Calling _mount doesn't work when we want to add mount options > such as realtime devices. Since it's just a normal scratch device > mount except for the source device, just call _scratch_mount with > SCRATCH_DEV set to the dmerror

Re: [PATCH 10/17] xfs/179: use scratch device helpers

2016-08-01 Thread Christoph Hellwig
On Thu, Jul 21, 2016 at 04:47:19PM -0700, Darrick J. Wong wrote: > Use the helper functions for scratch devices. This fixes a problem > where xfs/179 fails when there's a realtime device. > > Signed-off-by: Darrick J. Wong Looks fine, Reviewed-by: Christoph Hellwig

Re: [PATCH 08/17] xfs/129: fix post-metadump remounting idiocy

2016-08-01 Thread Christoph Hellwig
On Thu, Jul 21, 2016 at 04:47:07PM -0700, Darrick J. Wong wrote: > Use the standard _scratch_mount to mount the filesystem from the restored > image, instead of trying to call mount directly. This is needed in case > we had custom mount options (like rtdev). > > Signed-off-by: Darrick J. Wong

Re: [PATCH 11/17] xfs/234: use scratch device helpers

2016-08-01 Thread Christoph Hellwig
On Thu, Jul 21, 2016 at 04:47:26PM -0700, Darrick J. Wong wrote: > Use the helper functions for scratch devices. This fixes a problem > where xfs/234 fails when there's a realtime device. > > Signed-off-by: Darrick J. Wong Looks fine, Reviewed-by: Christoph Hellwig

[PATCH 1/3] btrfs-progs: Introduce new send-dump object

2016-08-01 Thread Qu Wenruo
Introduce send-dump.[ch] which implements a new btrfs_send_ops to exam and output all operations inside a send stream. It has a better output format than the old and no longer compilable send-test tool, but still tries to be script friendly. Provides the basis for later "inspect-internal

[PATCH 2/3] btrfs-progs: inspect: Introduce dump-send-stream subcommand

2016-08-01 Thread Qu Wenruo
Introduce a new subcommand "dump-send-stream" for "inspect-internal" command group. It will exam and output all operations inside a send stream. It's quite a useful tool to learn and dig kernel send stream. Signed-off-by: Qu Wenruo ---

[PATCH 3/3] btrfs-progs: Remove send-test tool

2016-08-01 Thread Qu Wenruo
Since new "inspect dump-send" has better output and structure, it's time to remove old and function-weak send-test tool. New "inspect dump-send" can handle them all better. Signed-off-by: Qu Wenruo --- Makefile.in | 6 +- send-test.c | 447

Re: Fixup direct bi_rw modifiers

2016-08-01 Thread Christoph Hellwig
On Sat, Jul 30, 2016 at 04:45:48PM -0500, Shaun Tancheff wrote: > bi_rw should be using bio_set_op_attrs to set bi_rw. Looks fine, Reviewed-by: Christoph Hellwig Jens, what do you think about renaming bi_rw? There aren't too many users left, and any old code that would keep

Re: systemd KillUserProcesses=yes and btrfs scrub

2016-08-01 Thread Austin S. Hemmelgarn
On 2016-07-30 20:29, Chris Murphy wrote: On Sat, Jul 30, 2016 at 2:02 PM, Chris Murphy wrote: Short version: When systemd-logind login.conf KillUserProcesses=yes, and the user does "sudo btrfs scrub start" in e.g. GNOME Terminal, and Same thing with Xfce, so it's not

Re: [PATCH 04/17] xfs/122: list the new log redo items

2016-08-01 Thread Christoph Hellwig
On Thu, Jul 21, 2016 at 04:46:42PM -0700, Darrick J. Wong wrote: > List the new log redo items. These should have stable sizes. > > Signed-off-by: Darrick J. Wong Looks fine, Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send the line

Re: [PATCH 01/17] xfs/26[34]: remove duplicate tests

2016-08-01 Thread Christoph Hellwig
On Thu, Jul 21, 2016 at 04:46:21PM -0700, Darrick J. Wong wrote: > These two tests were accidentally double-added as xfs/30[78], but the > newer versions have fixed up helper usage and fewer whitespace > problems, so nuke the old tests. > > Signed-off-by: Darrick J. Wong

Re: [PATCH 02/17] xfs: use rmapbt-checking helper

2016-08-01 Thread Christoph Hellwig
On Thu, Jul 21, 2016 at 04:46:29PM -0700, Darrick J. Wong wrote: > Don't open-code _notrun checks for the rmapbt, just use the helper. > > Signed-off-by: Darrick J. Wong Looks fine, Reviewed-by: Christoph Hellwig -- To unsubscribe from this list: send the

Re: [PATCH 03/17] xfs/310: fix the size calculation for the huge device

2016-08-01 Thread Christoph Hellwig
On Thu, Jul 21, 2016 at 04:46:35PM -0700, Darrick J. Wong wrote: > Fix the calculation of the dmhuge size. The previous calculation > tried to calculate the size correctly, but got it wrong for 1k > block sizes. Therefore, clean the whole mess up. > > Signed-off-by: Darrick J. Wong

Re: A lot warnings in dmesg while running thunderbird

2016-08-01 Thread Chris Mason
On 07/24/2016 08:36 PM, Dave Chinner wrote: On Fri, Jul 08, 2016 at 12:02:35PM -0400, Chris Mason wrote: Can you please run the attached test program: gcc -o short-write short-write.c -lpthread ./short-write some-new-file-on-btrfs Hi Chris, this seems like a useful thing to be testing on a

Re: qcow2 becomes 37P in size while qemu crashes

2016-08-01 Thread Chris Mason
On 07/23/2016 04:05 PM, Chris Murphy wrote: Using btrfs-debug-tree, I'm finding something a bit odd about some of the items in this 39P file. Seems normal item 71 key (994163 EXTENT_DATA 43689967616) itemoff 12467 itemsize 53 extent data disk byte 617349906432 nr 80805888

Re: systemd KillUserProcesses=yes and btrfs scrub

2016-08-01 Thread Chris Murphy
On Mon, Aug 1, 2016 at 9:46 AM, Chris Murphy wrote: > - The kernel workers are killed off within ~5 seconds of an > uninterrupted scrub. i.e. the kernel workers are doing the same work. They aren't being killed sooner as a result of logging out from the DE. The only

Re: systemd KillUserProcesses=yes and btrfs scrub

2016-08-01 Thread Chris Murphy
OK I've created a new volume that's sufficiently large I can tell if the kernel workers doing the scrub are also being killed off. First, I do a scrub without logging out to get a time for an uninterrupted scrub. And then initiate a scrub which I start timing, but then logout of the DE and watch

Re: systemd KillUserProcesses=yes and btrfs scrub

2016-08-01 Thread Austin S. Hemmelgarn
On 2016-08-01 11:46, Chris Murphy wrote: OK I've created a new volume that's sufficiently large I can tell if the kernel workers doing the scrub are also being killed off. First, I do a scrub without logging out to get a time for an uninterrupted scrub. And then initiate a scrub which I start

Re: systemd KillUserProcesses=yes and btrfs scrub

2016-08-01 Thread Chris Murphy
On Mon, Aug 1, 2016 at 10:08 AM, Austin S. Hemmelgarn wrote: > > MD and DM RAID handle this by starting kernel threads to do the scrub. They > then store the info about the scrub in the array itself, so you can query it > externally. If you watch, neither of those commands

Re: systemd KillUserProcesses=yes and btrfs scrub

2016-08-01 Thread Chris Murphy
On Mon, Aug 1, 2016 at 10:19 AM, Chris Murphy wrote: > So that makes me wonder how btrfs device add and remove will behave, > if issued in a DE which is then logged out of. Those commands do not > return to prompt until they complete. Strike add. That's fast. I'm

[PATCH] Btrfs: send, don't bug on inconsistent snapshots

2016-08-01 Thread fdmanana
From: Filipe Manana When doing an incremental send, if we find a new/modified/deleted extent, reference or xattr without having previously processed the corresponding inode item we end up exexuting a BUG_ON(). This is because whenever an extent, xattr or reference is added,

Re: qcow2 becomes 37P in size while qemu crashes

2016-08-01 Thread Chris Murphy
On Mon, Aug 1, 2016 at 9:26 AM, Chris Mason wrote: > > > On 07/23/2016 04:05 PM, Chris Murphy wrote: >> >> Using btrfs-debug-tree, I'm finding something a bit odd about some of >> the items in this 39P file. >> >> Seems normal >> >> item 71 key (994163 EXTENT_DATA 43689967616)

Corruption through subsequent compression?

2016-08-01 Thread Stephan Hopfmüller
Dear list, I've got into some trouble with my 40GB BTRFS formatted root partition in a 2015 Macbook 15": I was running low on free space so I remembered, that I had forgotten to enable compression while installing Antergos (a flavour of Arch Linux). So I looked into the Arch Wiki at

Re: systemd KillUserProcesses=yes and btrfs scrub

2016-08-01 Thread Austin S. Hemmelgarn
On 2016-08-01 12:19, Chris Murphy wrote: On Mon, Aug 1, 2016 at 10:08 AM, Austin S. Hemmelgarn wrote: MD and DM RAID handle this by starting kernel threads to do the scrub. They then store the info about the scrub in the array itself, so you can query it externally. If

Re: [PATCH 06/17] xfs: run xfs_repair at the end of each test

2016-08-01 Thread Darrick J. Wong
On Sun, Jul 31, 2016 at 11:27:19PM -0700, Christoph Hellwig wrote: > On Thu, Jul 21, 2016 at 04:46:54PM -0700, Darrick J. Wong wrote: > > Run xfs_repair twice at the end of each test -- once to rebuild > > the btree indices, and again with -n to check the rebuild work. > > This looks fine to me

Re: systemd KillUserProcesses=yes and btrfs scrub

2016-08-01 Thread Chris Murphy
On Mon, Aug 1, 2016 at 10:58 AM, Austin S. Hemmelgarn wrote: > On 2016-08-01 12:19, Chris Murphy wrote: >> >> On Mon, Aug 1, 2016 at 10:08 AM, Austin S. Hemmelgarn >> wrote: >> >>> >>> MD and DM RAID handle this by starting kernel threads to do the

Re: systemd KillUserProcesses=yes and btrfs scrub

2016-08-01 Thread Austin S. Hemmelgarn
On 2016-08-01 13:15, Chris Murphy wrote: On Mon, Aug 1, 2016 at 10:58 AM, Austin S. Hemmelgarn wrote: On 2016-08-01 12:19, Chris Murphy wrote: On Mon, Aug 1, 2016 at 10:08 AM, Austin S. Hemmelgarn wrote: MD and DM RAID handle this by starting

Re: Fixup direct bi_rw modifiers

2016-08-01 Thread Jens Axboe
On 08/01/2016 09:17 AM, Jens Axboe wrote: On 08/01/2016 05:47 AM, Christoph Hellwig wrote: On Sat, Jul 30, 2016 at 04:45:48PM -0500, Shaun Tancheff wrote: bi_rw should be using bio_set_op_attrs to set bi_rw. Looks fine, Reviewed-by: Christoph Hellwig Added, thanks Shaun.

Re: Fixup direct bi_rw modifiers

2016-08-01 Thread Jens Axboe
On 08/01/2016 05:47 AM, Christoph Hellwig wrote: On Sat, Jul 30, 2016 at 04:45:48PM -0500, Shaun Tancheff wrote: bi_rw should be using bio_set_op_attrs to set bi_rw. Looks fine, Reviewed-by: Christoph Hellwig Added, thanks Shaun. Jens, what do you think about renaming

Re: systemd KillUserProcesses=yes and btrfs scrub

2016-08-01 Thread Chris Murphy
Aug 01 09:29:59 localhost.localdomain sudo[1875]:chris : TTY=pts/1 ; PWD=/home/chris ; USER=root ; COMMAND=/sbin/btrfs scrub start /mnt/x Aug 01 09:30:16 localhost.localdomain systemd[1]: user@1000.service: Killing process 1883 (btrfs) with signal SIGKILL. Aug 01 09:43:34 localhost.localdomain

Re: FIDEDUPERANGE with src_length == 0

2016-08-01 Thread Darrick J. Wong
On Thu, Jul 14, 2016 at 11:16:47AM -0700, Omar Sandoval wrote: > On Thu, Jul 14, 2016 at 02:12:58PM -0400, Chris Mason wrote: > > > > > > On 07/14/2016 02:06 PM, Darrick J. Wong wrote: > > > On Wed, Jul 13, 2016 at 03:19:38PM +0200, David Sterba wrote: > > > > On Tue, Jul 12, 2016 at 10:26:43PM

Re: systemd KillUserProcesses=yes and btrfs scrub

2016-08-01 Thread Chris Murphy
On Mon, Aug 1, 2016 at 11:19 AM, Austin S. Hemmelgarn wrote: > On 2016-08-01 13:15, Chris Murphy wrote: >> I've been using balance with &, and when I logout, the btrfs command >> continues to flip between status D and R, just like before logout and >> it appears to

Re: systemd KillUserProcesses=yes and btrfs scrub

2016-08-01 Thread Chris Murphy
All of these have status R and D for their duration, and while all get a SIGKILL from systemd on logout, none of the processes change status or die until their kernel task is done. And each of these operations complete successfully with no worse for the wear. btrfs balance & btrfs dev rem & btrfs

Re: Btrfs send to send out metadata and data separately

2016-08-01 Thread Filipe Manana
On Fri, Jul 29, 2016 at 1:40 PM, Qu Wenruo wrote: > Hi Filipe, and maintainers, > > I'm recently working on the root fix to free send from calling backref walk. > > My current idea is to send data and metadata separately, and only do clone > detection inside the send

Re: Btrfs send to send out metadata and data separately

2016-08-01 Thread Qu Wenruo
At 08/02/2016 02:00 AM, Filipe Manana wrote: On Fri, Jul 29, 2016 at 1:40 PM, Qu Wenruo wrote: Hi Filipe, and maintainers, I'm recently working on the root fix to free send from calling backref walk. My current idea is to send data and metadata separately, and only