Re: [PATCH] btrfs: Allow more disks missing for RAID10

2019-07-30 Thread Qu Wenruo
On 2019/7/26 下午6:39, David Sterba wrote: > On Fri, Jul 26, 2019 at 07:41:41AM +0800, Qu Wenruo wrote: >> >> >> On 2019/7/26 上午2:37, David Sterba wrote: >>> On Thu, Jul 18, 2019 at 02:27:49PM +0800, Qu Wenruo wrote: RAID10 can accept as much as half of its disks to be missing, as long as

Re: Bug Report: Btrfs can't allocate space for delete when block size arounds 512M

2019-07-30 Thread Hongzhi, Song
Hi, Anyone notice this? --Hongzhi On 7/17/19 4:34 PM, Hongzhi, Song wrote: Hi friends, *Description:*     One LTP testcase, fs_fill.c, fails on btrfs with kernel error when unlink files on Btrfs device:     "BTRFS warning (device loop0): could not allocate space for a delete; will trun

Re: Massive filesystem corruption since kernel 5.2 (ARCH)

2019-07-30 Thread Chris Murphy
On Tue, Jul 30, 2019 at 5:13 PM Graham Cobb wrote: > > On 30/07/2019 23:44, Swâmi Petaramesh wrote: > > Still, losing a given FS with subvols, snapshots etc, may be very > > annoying and very time consuming rebuilding. > > I believe that in one of the earlier mails, Qu said that you can > probably

Re: Massive filesystem corruption since kernel 5.2 (ARCH)

2019-07-30 Thread Graham Cobb
On 30/07/2019 23:44, Swâmi Petaramesh wrote: > Still, losing a given FS with subvols, snapshots etc, may be very > annoying and very time consuming rebuilding. I believe that in one of the earlier mails, Qu said that you can probably mount the corrupted fs readonly and read everything. If that is

Re: Massive filesystem corruption since kernel 5.2 (ARCH)

2019-07-30 Thread Swâmi Petaramesh
Le 30/07/2019 à 22:15, Chris Murphy a écrit : > I sympathize with the lack of resources. But no full disk backup > simply cannot be taken seriously in any computer science context. The > data cannot be that important by the user's own estimation if there > aren't backups. It's reasonable for resour

Re: Massive filesystem corruption since kernel 5.2 (ARCH)

2019-07-30 Thread Chris Murphy
On Tue, Jul 30, 2019 at 2:09 AM Swâmi Petaramesh wrote: > > On 7/29/19 9:10 PM, Chris Murphy wrote: > > We've discussed many times how both file system repair, and file > > system restore from backup, simply are not scalable for big file > > systems. It takes too long. > > So what would be the sol

Re: [RFC PATCH 09/17] progs: pass in a btrfs_mkfs_config to write_temp_extent_buffer

2019-07-30 Thread David Sterba
On Thu, Jul 25, 2019 at 11:33:56AM +0200, Johannes Thumshirn wrote: > Pass in a btrfs_mkfs_config to write_temp_extent_buffer(), this is needed > so we can grab the checksum type for checksum buffer verification in later > patches. > > Signed-off-by: Johannes Thumshirn Added to devel.

Re: [RFC PATCH 07/17] btrfs-progs: use btrfs_csum_data() in __csum_tree_block_size()

2019-07-30 Thread David Sterba
On Thu, Jul 25, 2019 at 11:33:54AM +0200, Johannes Thumshirn wrote: > Use the btrfs_csum_data() wrapper in __csum_tree_block_size() instead of > directly calling crc32c(). > > This helps us when plumbing new checksum algorithms into the FS. > > Signed-off-by: Johannes Thumshirn Added to devel.

Re: [RFC PATCH 05/17] btrfs-progs: add option for checksum type to mkfs

2019-07-30 Thread David Sterba
On Thu, Jul 25, 2019 at 11:33:52AM +0200, Johannes Thumshirn wrote: > Add an option to mkfs to specify which checksum algorithm will be used for > the filesystem. > > XXX this patch should go last in the series. If the code update and the mkfs option are split, I can merge the preparatory part in

Re: [PATCH] btrfs-progs: check: fix option parsing for -E

2019-07-30 Thread David Sterba
On Sun, Jul 28, 2019 at 08:47:36AM +0200, Adam Borowski wrote: > > On Mon, Jun 17, 2019 at 06:45:48PM +0200, Adam Borowski wrote: > > > It has a mandatory argument, thus it always crashed. > > > > Applied, thanks. > > Seems like this has fallen through the cracks -- could you please re-apply? Dun

Re: [RFC PATCH 2/4] btrfs: create structure to encode checksum type and length

2019-07-30 Thread David Sterba
On Thu, Jul 25, 2019 at 11:33:49AM +0200, Johannes Thumshirn wrote: > Create a structure to encode the type and length for the known on-disk > checksums. Also add a table and a convenience macro for adding the > checksum types to the table. > > This makes it easier to add new checksums later. > >

Re: [RFC PATCH 4/4] btrfs: sysfs: export supported checksums

2019-07-30 Thread David Sterba
On Thu, Jul 25, 2019 at 11:33:51AM +0200, Johannes Thumshirn wrote: > From: David Sterba > > Export supported checksum algorithms via sysfs. I wonder if we should also export the implementation that would be used. This could be crross referenced with /proc/crypto, but having it in one place woul

[PATCH 2/2] btrfs: delete debugfs code

2019-07-30 Thread David Sterba
Replaced by the sysfs exports that provide a more fine grained interface for filesystem debugging. Signed-off-by: David Sterba --- fs/btrfs/sysfs.c | 36 fs/btrfs/sysfs.h | 5 - 2 files changed, 41 deletions(-) diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/s

[PATCH 1/2] btrfs: sysfs: add debugging exports

2019-07-30 Thread David Sterba
Add 'debug' directories to global sysfs and per-filesystem. This will replace the debugfs directory. The sysfs location is simpler and builds on top of the existing file hierarchy so there will hopefully be no more questions about the sample debugfs file. The directory is called 'debug' and only p

[PATCH 0/2] Sysfs updates

2019-07-30 Thread David Sterba
Export the potential debugging data in the per-filesystem directories we already have, so we can drop debugfs. The new directories depend on CONFIG_BTRFS_DEBUG so they're not affecting normal builds. David Sterba (2): btrfs: sysfs: add debugging exports btrfs: delete debugfs code fs/btrfs/sy

Re: [PATCH] fs: btrfs: Add an assertion to warn incorrct case in insert_inline_extent()

2019-07-30 Thread David Sterba
On Sat, Jul 27, 2019 at 04:51:13PM +0800, Jia-Ju Bai wrote: > In insert_inline_extent(), the case that compressed_size > 0 > and compressed_pages = NULL cannot occur, otherwise a null-pointer > dereference may occur on line 215: > cpage = compressed_pages[i]; > > To warn this incorrect case,

Re: [PATCH 00/14 RFC] Btrfs: Add journal for raid5/6 writes

2019-07-30 Thread Goffredo Baroncelli
On 30/07/2019 16.48, Torstein Eide wrote: > Hi > Is there any news to implementing journal for raid5/6 writes? > I think that you should ask to ML. I am (was) occasional contributor than a active btrfs developers. BR G.Baroncelli -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint

Re: [PATCH] Btrfs: fix deadlock between fiemap and transaction commits

2019-07-30 Thread David Sterba
On Mon, Jul 29, 2019 at 09:37:10AM +0100, fdman...@kernel.org wrote: > From: Filipe Manana > > The fiemap handler locks a file range that can have unflushed delalloc, > and after locking the range, it tries to attach to a running transaction. > If the running transaction started its commit, that

Re: [PATCH v2 2/5] btrfs: extent-tree: Kill BUG_ON() in __btrfs_free_extent() and do better comment

2019-07-30 Thread kbuild test robot
Hi Qu, Thank you for the patch! Perhaps something to improve: [auto build test WARNING on linus/master] [cannot apply to v5.3-rc2 next-20190730] [if your patch is applied to the wrong git tree, please drop us a note to help improve the system] url: https://github.com/0day-ci/linux/commits

Re: [BUG] BTRFS critical corrupt leaf - bisected to 496245cac57e

2019-07-30 Thread Qu Wenruo
On 2019/7/30 下午9:57, Swâmi Petaramesh wrote: > Hello, > > Le 30/07/2019 à 09:21, Qu Wenruo a écrit : >> Unfortunately, transid error here helps nothing. > > Now, each and everytime I try to mount this disk on the original > machine, or another one, I get : When the message shows, it means the d

Re: [BUG] BTRFS critical corrupt leaf - bisected to 496245cac57e

2019-07-30 Thread Swâmi Petaramesh
Hello, Le 30/07/2019 à 09:21, Qu Wenruo a écrit : > Unfortunately, transid error here helps nothing. Now, each and everytime I try to mount this disk on the original machine, or another one, I get : systemd[1]: run-media-.mount: Succeeded. kernel: BTRFS info (device dm-2): disk s

Re: [PATCH v2 0/2] Refactor snapshot vs nocow writers locking

2019-07-30 Thread Valentin Schneider
And with a more accurate spec (appended at the end of this email): Model checking completed. No error has been found. Estimates of the probability that TLC did not check all reachable states because two distinct states had the same fingerprint: calculated (optimistic): val =

Re: Btrfs progs release 5.2.1

2019-07-30 Thread David Sterba
On Tue, Jul 30, 2019 at 02:17:16PM +0300, Baruch Siach wrote: > Hi David, > > On Fri, Jul 26 2019, David Sterba wrote: > > btrfs-progs version 5.2.1 have been released. This is a bugfix release. > > > > Changes: > > * scrub status: fix ETA calculation after resume > > * check: fix crash when u

Re: [PATCH v2 0/2] Refactor snapshot vs nocow writers locking

2019-07-30 Thread Nikolay Borisov
On 30.07.19 г. 14:03 ч., Valentin Schneider wrote: > On 29/07/2019 17:32, Valentin Schneider wrote: >> On 29/07/2019 16:33, Catalin Marinas wrote: > [...] >>> I'd say that's one of the pitfalls of PlusCal. The above is executed >>> atomically, so you'd have the lock_state read and updated in the

Re: Btrfs progs release 5.2.1

2019-07-30 Thread Holger Hoffstätte
On 7/30/19 1:17 PM, Baruch Siach wrote: Hi David, On Fri, Jul 26 2019, David Sterba wrote: btrfs-progs version 5.2.1 have been released. This is a bugfix release. Changes: * scrub status: fix ETA calculation after resume * check: fix crash when using -Q * restore: fix symlink owner re

Re: Btrfs progs release 5.2.1

2019-07-30 Thread Baruch Siach
Hi David, On Fri, Jul 26 2019, David Sterba wrote: > btrfs-progs version 5.2.1 have been released. This is a bugfix release. > > Changes: > * scrub status: fix ETA calculation after resume > * check: fix crash when using -Q > * restore: fix symlink owner restoration > * mkfs: fix regressio

Re: [PATCH v2 0/2] Refactor snapshot vs nocow writers locking

2019-07-30 Thread Valentin Schneider
On 29/07/2019 17:32, Valentin Schneider wrote: > On 29/07/2019 16:33, Catalin Marinas wrote: [...] >> I'd say that's one of the pitfalls of PlusCal. The above is executed >> atomically, so you'd have the lock_state read and updated in the same >> action. Looking at the C patches, there is an >> ato

Re: how to recover data from formatted btrfs partition

2019-07-30 Thread Konstantin V. Gavrilenko
>>On Sun, Jul 28, 2019 at 10:36 AM Konstantin V. Gavrilenko >> wrote: >> >> Hi list, >> >> I accidentally formatted the existing btrfs partition today with mkfs.btrfs >> Partition obviously table remained intact, while all three superblock 0,1,2 >> correspond to the new btrfs UUID. >> The origin

Re: Massive filesystem corruption since kernel 5.2 (ARCH)

2019-07-30 Thread Swâmi Petaramesh
Hi, On 7/30/19 10:04 AM, Henk Slager wrote: > Maybe you could zoom-in a bit more on the kernel (and btrfs-progs) binary. > Does Arch do any changes to the kernel.org version 5.2.0 ? I don't precisely know. Each and every distro out there applies some set of patches to the kernel that they package

Re: Massive filesystem corruption since kernel 5.2 (ARCH)

2019-07-30 Thread Swâmi Petaramesh
On 7/29/19 9:10 PM, Chris Murphy wrote: > We've discussed many times how both file system repair, and file > system restore from backup, simply are not scalable for big file > systems. It takes too long. So what would be the solution ? IMHO yes, having to full backup then reformat then full resto

Re: [BUG] BTRFS critical corrupt leaf - bisected to 496245cac57e

2019-07-30 Thread Swâmi Petaramesh
On 7/30/19 9:21 AM, Qu Wenruo wrote: > >> I have emergency downgraded my system to 5.1 not to take any risk of >> crashing my SSD again (and if it crashes again anyway, then I will know >> it is not kernel 5.2's fault and let you know...) > That kernel message is to *prevent* any further damage, by

Re: Massive filesystem corruption since kernel 5.2 (ARCH)

2019-07-30 Thread Henk Slager
On Mon, Jul 29, 2019 at 4:17 PM Swâmi Petaramesh wrote: > > On 7/29/19 3:29 PM, Lionel Bouton wrote: > > For another reference point, my personal laptop reports 17 days of > > uptime on 5.2.0-arch2-1-ARCH. > > I use BTRFS both over LUKS over LVM and directly over LVM. The system > > is suspended d

Re: [BUG] BTRFS critical corrupt leaf - bisected to 496245cac57e

2019-07-30 Thread Qu Wenruo
On 2019/7/30 下午2:44, Swâmi Petaramesh wrote: > Le 30/07/2019 à 06:56, Qu Wenruo a écrit : I think I'me gonna emergency downgrade all my BTRFS machines to kernel 5.1 before they break ,-( >>> >>> Full kernel message please. >>> >>> That commit is designed to corrupted inode item, w