Re: What if TRIM issued a wipe on devices that don't TRIM?

2018-12-06 Thread ronnie sahlberg
Hi, I am more of a SCSI guy than ATA so forgive where I am ignorant. The SCSI equivalent to TRIM is called UNMAP. UNMAP is unfortunately only a "hint" to the device so if the device for any reason is busy, it can just do a NO-OP, leave the data as is and still return status SUCCESS. That is not

Re: [PATCH 00/10] btrfs: Support for DAX devices

2018-12-06 Thread Johannes Thumshirn
On 05/12/2018 13:28, Goldwyn Rodrigues wrote: > This is a support for DAX in btrfs. I understand there have been > previous attempts at it. However, I wanted to make sure copy-on-write > (COW) works on dax as well. > > Before I present this to the FS folks I wanted to run this through the > btrfs.

HELP unmountable partition after btrfs balance to RAID0

2018-12-06 Thread Thomas Mohr
Dear developers of BTRFS, we have a problem. We wanted to convert a file system to a RAID0 with two partitions. Unfortunately we had to reboot the server during the balance operation before it could complete. Now following happens: A mount attempt of the array fails with following error code

Re: [PATCH 07/10] dax: export functions for use with btrfs

2018-12-06 Thread Goldwyn Rodrigues
On 6:52 05/12, Christoph Hellwig wrote: > If you want to export these at all they have to be EXPORT_SYMBOL_GPL. > Understood. > But I'd really like to avoid seeing another duplicate DAX I/O path. > Please try to adopt the existing iomap-based infrastructure for your > needs first. This is not

Re: [PATCH 00/10] btrfs: Support for DAX devices

2018-12-06 Thread Goldwyn Rodrigues
On 11:07 06/12, Johannes Thumshirn wrote: > On 05/12/2018 13:28, Goldwyn Rodrigues wrote: > > This is a support for DAX in btrfs. I understand there have been > > previous attempts at it. However, I wanted to make sure copy-on-write > > (COW) works on dax as well. > > > > Before I present this to

Re: [PATCH 01/10] btrfs: add btrfs_delete_ref_head helper

2018-12-06 Thread Nikolay Borisov
On 3.12.18 г. 17:20 ч., Josef Bacik wrote: > From: Josef Bacik > > We do this dance in cleanup_ref_head and check_ref_cleanup, unify it > into a helper and cleanup the calling functions. > > Signed-off-by: Josef Bacik > Reviewed-by: Omar Sandoval Reviewed-by: Nikolay Borisov > --- > fs/

Re: [PATCH 02/10] btrfs: add cleanup_ref_head_accounting helper

2018-12-06 Thread Nikolay Borisov
On 3.12.18 г. 17:20 ч., Josef Bacik wrote: > From: Josef Bacik > > We were missing some quota cleanups in check_ref_cleanup, so break the > ref head accounting cleanup into a helper and call that from both > check_ref_cleanup and cleanup_ref_head. This will hopefully ensure that > we don't sc

Re: [PATCH 06/10] btrfs: update may_commit_transaction to use the delayed refs rsv

2018-12-06 Thread Nikolay Borisov
On 3.12.18 г. 17:20 ч., Josef Bacik wrote: > Any space used in the delayed_refs_rsv will be freed up by a transaction > commit, so instead of just counting the pinned space we also need to > account for any space in the delayed_refs_rsv when deciding if it will > make a different to commit the t

Re: What if TRIM issued a wipe on devices that don't TRIM?

2018-12-06 Thread Austin S. Hemmelgarn
On 2018-12-06 01:11, Robert White wrote: (1) Automatic and selective wiping of unused and previously used disk blocks is a good security measure, particularly when there is an encryption layer beneath the file system. (2) USB attached devices _never_ support TRIM and they are the most likely

Re: [PATCH 0/3] btrfs: use offset_in_page and PAGE_ALIGNED

2018-12-06 Thread David Sterba
On Wed, Dec 05, 2018 at 03:23:02PM +0100, Johannes Thumshirn wrote: > Use the offset_in_page() and PAGE_ALIGNED() macros instead of open-coding them > throughout btrfs. > > This series also includes a patch for 'make coccicheck' which is marked as an > RFC and I've CCed Julia in the hoping to get

Re: [PATCH 00/10][V2] Delayed refs rsv

2018-12-06 Thread David Sterba
On Mon, Dec 03, 2018 at 10:20:28AM -0500, Josef Bacik wrote: > v1->v2: > - addressed the comments from the various reviewers. > - split "introduce delayed_refs_rsv" into 5 patches. The patches are the same > together as they were, just split out more logically. They can't really be > bisected

Re: [PATCH 09/10] btrfs: don't run delayed refs in the end transaction logic

2018-12-06 Thread Nikolay Borisov
On 3.12.18 г. 17:20 ч., Josef Bacik wrote: > Over the years we have built up a lot of infrastructure to keep delayed > refs in check, mostly by running them at btrfs_end_transaction() time. > We have a lot of different maths we do to figure out how much, if we > should do it inline or async, etc

Re: [PATCH 08/10] btrfs: rework btrfs_check_space_for_delayed_refs

2018-12-06 Thread Nikolay Borisov
On 3.12.18 г. 17:20 ч., Josef Bacik wrote: > Now with the delayed_refs_rsv we can now know exactly how much pending > delayed refs space we need. This means we can drastically simplify IMO it will be helpful if there is a sentence here referring back to btrfs_update_delayed_refs_rsv to put you

Re: [RFC PATCH 3/3] coccinelle: api: add offset_in_page.cocci

2018-12-06 Thread Johannes Thumshirn
On 05/12/2018 15:46, Julia Lawall wrote: [...] >> +@r_patch depends on !context && patch && !org && !report@ >> +expression E; >> +type T; >> +@@ >> + >> +( >> +- E & ~PAGE_MASK >> ++ offset_in_page(E) >> +| >> +- E & (PAGE_SIZE - 1) >> ++ offset_in_page(E) > > The two lines above should be subsum

Re: [PATCH 08/10] btrfs: rework btrfs_check_space_for_delayed_refs

2018-12-06 Thread David Sterba
On Thu, Dec 06, 2018 at 06:52:21PM +0200, Nikolay Borisov wrote: > > > On 3.12.18 г. 17:20 ч., Josef Bacik wrote: > > Now with the delayed_refs_rsv we can now know exactly how much pending > > delayed refs space we need. This means we can drastically simplify > > IMO it will be helpful if there

Re: [PATCH 1/2] btrfs: catch cow on deleting snapshots

2018-12-06 Thread David Sterba
On Fri, Nov 30, 2018 at 12:19:18PM -0500, Josef Bacik wrote: > On Fri, Nov 30, 2018 at 05:14:54PM +, Filipe Manana wrote: > > On Fri, Nov 30, 2018 at 4:53 PM Josef Bacik wrote: > > > > > > From: Josef Bacik > > > > > > When debugging some weird extent reference bug I suspected that we were >

Re: [PATCH][v2] btrfs: run delayed items before dropping the snapshot

2018-12-06 Thread David Sterba
On Wed, Dec 05, 2018 at 12:12:21PM -0500, Josef Bacik wrote: > From: Josef Bacik > > With my delayed refs patches in place we started seeing a large amount > of aborts in __btrfs_free_extent > > BTRFS error (device sdb1): unable to find ref byte nr 91947008 parent 0 root > 35964 owner 1 offset

Re: [PATCH v2 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-12-06 Thread David Sterba
On Mon, Nov 12, 2018 at 10:33:33PM +0100, David Sterba wrote: > On Thu, Nov 08, 2018 at 01:49:12PM +0800, Qu Wenruo wrote: > > This patchset can be fetched from github: > > https://github.com/adam900710/linux/tree/qgroup_delayed_subtree_rebased > > > > Which is based on v4.20-rc1. > > Thanks, I'l

Re: [RFC PATCH 3/3] coccinelle: api: add offset_in_page.cocci

2018-12-06 Thread Julia Lawall
On Thu, 6 Dec 2018, Johannes Thumshirn wrote: > On 05/12/2018 15:46, Julia Lawall wrote: > [...] > >> +@r_patch depends on !context && patch && !org && !report@ > >> +expression E; > >> +type T; > >> +@@ > >> + > >> +( > >> +- E & ~PAGE_MASK > >> ++ offset_in_page(E) > >> +| > >> +- E & (PAGE_S

Re: [PATCH v2 0/6] btrfs: qgroup: Delay subtree scan to reduce overhead

2018-12-06 Thread Qu Wenruo
On 2018/12/7 上午3:35, David Sterba wrote: > On Mon, Nov 12, 2018 at 10:33:33PM +0100, David Sterba wrote: >> On Thu, Nov 08, 2018 at 01:49:12PM +0800, Qu Wenruo wrote: >>> This patchset can be fetched from github: >>> https://github.com/adam900710/linux/tree/qgroup_delayed_subtree_rebased >>> >>>

Re: BTRFS RAID filesystem unmountable

2018-12-06 Thread Qu Wenruo
On 2018/12/7 上午7:15, Michael Wade wrote: > Hi Qu, > > Me again! Having formatted the drives and rebuilt the RAID array I > seem to have be having the same problem as before (no power cut this > time [I bought a UPS]). But strangely, your super block shows it has log tree, which means either you

[PATCH] libbtrfsutil: fix unprivileged tests if kernel lacks support

2018-12-06 Thread Omar Sandoval
From: Omar Sandoval I apparently didn't test this on a pre-4.18 kernel. test_subvolume_info_unprivileged() checks for an ENOTTY, but this doesn't seem to work correctly with subTest(). test_subvolume_iterator_unprivileged() doesn't have a check at all. Add an explicit check to both before doing t

Re: [PATCH RESEND 0/8] btrfs-progs: sub: Relax the privileges of "subvolume list/show"

2018-12-06 Thread Omar Sandoval
On Tue, Nov 27, 2018 at 02:24:41PM +0900, Misono Tomohiro wrote: > Hello, > > This is basically the resend of > "[PATCH v2 00/20] btrfs-progs: Rework of "subvolume list/show" and relax the > root privileges of them" [1] > which I submitted in June. The aim of this series is to allow non-p

Re: What if TRIM issued a wipe on devices that don't TRIM?

2018-12-06 Thread Andrei Borzenkov
06.12.2018 16:04, Austin S. Hemmelgarn пишет: > > * On SCSI devices, a discard operation translates to a SCSI UNMAP > command.  As pointed out by Ronnie Sahlberg in his reply, this command > is purely advisory, may not result in any actual state change on the > target device, and is not guaranteed

System unable to mount partition after a power loss

2018-12-06 Thread Doni Crosby
All, I'm coming to you to see if there is a way to fix or at least recover most of the data I have from a btrfs filesystem. The system went down after both a breaker and the battery backup failed. I cannot currently mount the system, with the following error from dmesg: Note: The vda1 is just the

Re: System unable to mount partition after a power loss

2018-12-06 Thread Chris Murphy
On Thu, Dec 6, 2018 at 10:24 PM Doni Crosby wrote: > > All, > > I'm coming to you to see if there is a way to fix or at least recover > most of the data I have from a btrfs filesystem. The system went down > after both a breaker and the battery backup failed. I cannot currently > mount the system,

Re: System unable to mount partition after a power loss

2018-12-06 Thread Doni Crosby
> This is qemu-kvm? What's the cache mode being used? It's possible the > usual write guarantees are thwarted by VM caching. Yes it is a proxmox host running the system so it is a qemu vm, I'm unsure on the caching situation. > Old version of progs, I suggest upgrading to 4.17.1 and run I updated

Re: [PATCH 08/10] btrfs: rework btrfs_check_space_for_delayed_refs

2018-12-06 Thread Nikolay Borisov
On 6.12.18 г. 19:54 ч., David Sterba wrote: > On Thu, Dec 06, 2018 at 06:52:21PM +0200, Nikolay Borisov wrote: >> >> >> On 3.12.18 г. 17:20 ч., Josef Bacik wrote: >>> Now with the delayed_refs_rsv we can now know exactly how much pending >>> delayed refs space we need. This means we can drastic

Re: System unable to mount partition after a power loss

2018-12-06 Thread Qu Wenruo
On 2018/12/7 下午1:24, Doni Crosby wrote: > All, > > I'm coming to you to see if there is a way to fix or at least recover > most of the data I have from a btrfs filesystem. The system went down > after both a breaker and the battery backup failed. I cannot currently > mount the system, with the f

Re: [PATCH AUTOSEL 4.19 052/123] Btrfs: send, fix infinite loop due to directory rename dependencies

2018-12-06 Thread David Sterba
On Wed, Dec 05, 2018 at 04:34:44AM -0500, Sasha Levin wrote: > From: Robbie Ko > > [ Upstream commit a4390aee72713d9e73f1132bcdeb17d72fbbf974 ] ... > > Signed-off-by: Robbie Ko > Reviewed-by: Filipe Manana > [Wrote changelog with example and more clear explanation] > Signed-off-by: Filipe Man