Re: Can't mount array with super_total_bytes mismatch with fs_devices total_rw_bytes

2017-10-03 Thread Qu Wenruo
On 2017年10月04日 12:00, Asif Youssuff wrote: Thanks for the advice. On 10/03/2017 09:38 PM, Qu Wenruo wrote: [210017.281912] BTRFS info (device sdb): disk space caching is enabled [210017.281915] BTRFS info (device sdb): has skinny extents [210017.402084] BTRFS error (device sdb):

Re: Btrfs "failed to repair damaged filesystem" - RAID10 going RO when any write attempts are made

2017-10-03 Thread Timothy White
Any suggestions on this? Or do I just blow it away and hope the bug is fixed in a newer version? Regards Tim On Mon, Oct 2, 2017 at 8:44 PM, Timothy White wrote: > I have a BTRFS RAID 10 filesystem that was crashing and going into RO > mode. A did a kernel upgrade,

Re: Can't mount array with super_total_bytes mismatch with fs_devices total_rw_bytes

2017-10-03 Thread Asif Youssuff
Thanks for the advice. On 10/03/2017 09:38 PM, Qu Wenruo wrote: [210017.281912] BTRFS info (device sdb): disk space caching is enabled [210017.281915] BTRFS info (device sdb): has skinny extents [210017.402084] BTRFS error (device sdb): super_total_bytes 92017859088384 mismatch with

[PATCH v2] Btrfs: fix overlap of fs_info->flags values

2017-10-03 Thread Tsutomu Itoh
Because the values of BTRFS_FS_EXCL_OP and BTRFS_FS_QUOTA_OVERRIDE overlap, we should change the value. First, BTRFS_FS_EXCL_OP was set to 14. commit 171938e52807 ("btrfs: track exclusive filesystem operation in flags") Next, the value of BTRFS_FS_QUOTA_OVERRIDE was set to 14. commit

Re: Can't mount array with super_total_bytes mismatch with fs_devices total_rw_bytes

2017-10-03 Thread Qu Wenruo
On 2017年10月04日 07:32, Asif Youssuff wrote: Hi, My power went out at my home, and I'm now having trouble mounting my array. I'm mounting with the 'recovery' option in fstab. When mounting, dmesg output shows: [210017.281912] BTRFS info (device sdb): disk space caching is enabled

[RFC] Is it advisable to use btrfs check --repair flag to fix/find errors?

2017-10-03 Thread Soujanya Ponnapalli
Hi, We are researchers from UT Austin, working on building CrashMonkey[1], a simple, flexible, file-system agnostic test framework to systematically check file-systems for inconsistencies if a failure occurs during a file operation. Here is a brief description of what we are trying to do:

Can't mount array with super_total_bytes mismatch with fs_devices total_rw_bytes

2017-10-03 Thread Asif Youssuff
Hi, My power went out at my home, and I'm now having trouble mounting my array. I'm mounting with the 'recovery' option in fstab. When mounting, dmesg output shows: [210017.281912] BTRFS info (device sdb): disk space caching is enabled [210017.281915] BTRFS info (device sdb): has skinny

Re: Seeking Help on Corruption Issues

2017-10-03 Thread Stephen Nesbitt
On 10/3/2017 2:11 PM, Hugo Mills wrote: Hi, Stephen, On Tue, Oct 03, 2017 at 08:52:04PM +, Stephen Nesbitt wrote: Here it i. There are a couple of out-of-order entries beginning at 117. And yes I did uncover a bad stick of RAM: btrfs-progs v4.9.1 leaf 2589782867968 items 134 free

Re: Something like ZFS Channel Programs for BTRFS & probably XFS or even VFS?

2017-10-03 Thread Dave Chinner
On Tue, Oct 03, 2017 at 01:40:51PM -0700, Matthew Wilcox wrote: > On Wed, Oct 04, 2017 at 07:10:35AM +1100, Dave Chinner wrote: > > On Tue, Oct 03, 2017 at 03:19:18PM +0200, Martin Steigerwald wrote: > > > [repost. I didn´t notice autocompletion gave me wrong address for > > > fsdevel, > > >

Re: Seeking Help on Corruption Issues

2017-10-03 Thread Hugo Mills
Hi, Stephen, On Tue, Oct 03, 2017 at 08:52:04PM +, Stephen Nesbitt wrote: > Here it i. There are a couple of out-of-order entries beginning at 117. And > yes I did uncover a bad stick of RAM: > > btrfs-progs v4.9.1 > leaf 2589782867968 items 134 free space 6753 generation 3351574 owner 2

Re: Something like ZFS Channel Programs for BTRFS & probably XFS or even VFS?

2017-10-03 Thread Matthew Wilcox
On Wed, Oct 04, 2017 at 07:10:35AM +1100, Dave Chinner wrote: > On Tue, Oct 03, 2017 at 03:19:18PM +0200, Martin Steigerwald wrote: > > [repost. I didn´t notice autocompletion gave me wrong address for fsdevel, > > blacklisted now] > > > > Hello. > > > > What do you think of > > > >

Re: Something like ZFS Channel Programs for BTRFS & probably XFS or even VFS?

2017-10-03 Thread Randy Dunlap
On 10/03/17 13:10, Dave Chinner wrote: > On Tue, Oct 03, 2017 at 03:19:18PM +0200, Martin Steigerwald wrote: >> [repost. I didn´t notice autocompletion gave me wrong address for fsdevel, >> blacklisted now] >> >> Hello. >> >> What do you think of >> >>

Re: Something like ZFS Channel Programs for BTRFS & probably XFS or even VFS?

2017-10-03 Thread Dave Chinner
On Tue, Oct 03, 2017 at 03:19:18PM +0200, Martin Steigerwald wrote: > [repost. I didn´t notice autocompletion gave me wrong address for fsdevel, > blacklisted now] > > Hello. > > What do you think of > > http://open-zfs.org/wiki/Projects/ZFS_Channel_Programs Domain not found. -Dave. -- Dave

Re: Seeking Help on Corruption Issues

2017-10-03 Thread Hugo Mills
On Tue, Oct 03, 2017 at 01:06:50PM -0700, Stephen Nesbitt wrote: > All: > > I came back to my computer yesterday to find my filesystem in read > only mode. Running a btrfs scrub start -dB aborts as follows: > > btrfs scrub start -dB /mnt > ERROR: scrubbing /mnt failed for device id 4: ret=-1,

Seeking Help on Corruption Issues

2017-10-03 Thread Stephen Nesbitt
All: I came back to my computer yesterday to find my filesystem in read only mode. Running a btrfs scrub start -dB aborts as follows: btrfs scrub start -dB /mnt ERROR: scrubbing /mnt failed for device id 4: ret=-1, errno=5 (Input/output error) ERROR: scrubbing /mnt failed for device id 5:

[PATCH] btrfs: avoid overflow when sector_t is 32 bit

2017-10-03 Thread Goffredo Baroncelli
From: Goffredo Baroncelli Jean-Denis Girard noticed commit c821e7f3 "pass bytes to btrfs_bio_alloc" (https://patchwork.kernel.org/patch/9763081/) introduces a regression on 32 bit machines. When CONFIG_LBDAF is _not_ defined (CONFIG_LBDAF == Support for large (2TB+) block

Re: [PATCH] btrfs-progs: misc-test: use raid1 for data to enable mount with -o degraded

2017-10-03 Thread David Sterba
On Tue, Oct 03, 2017 at 03:47:26PM +0900, Misono, Tomohiro wrote: > kernel 4.14 introduces new function for checking if all chunks is ok for > mount with -o degraded option. > > commit 21634a19f646 ("btrfs: Introduce a function to check if all > chunks a OK for degraded rw mount") > > As a

Re: [PATCH] Btrfs: fix fs_info->flags value

2017-10-03 Thread David Sterba
On Mon, Oct 02, 2017 at 05:34:12PM +0900, Tsutomu Itoh wrote: > Because the values of BTRFS_FS_QUOTA_OVERRIDE and BTRFS_FS_EXCL_OP overlap, > we should change the value. > > Signed-off-by: Tsutomu Itoh Please write a more descriptive subject and changelog. > --- >

[PATCH v8 2/2] btrfs: check device for critical errors and mark failed

2017-10-03 Thread Anand Jain
From: Anand Jain Write and flush errors are critical errors, upon which the device fd must be closed and marked as failed. There are two type of device close in btrfs, one, close as part of clean up where we shall release the struct btrfs_device and or btrfs_fs_devices as

[PATCH v8 1/2] btrfs: introduce device dynamic state transition to failed

2017-10-03 Thread Anand Jain
From: Anand Jain This patch provides helper functions to force a device to failed, and we need it for the following reasons, 1) a. It can be reported that device has failed when it does and b. Close the device when it goes offline so that blocklayer can cleanup 2)

[PATCH v8 0/2] [RFC] Introduce device state 'failed'

2017-10-03 Thread Anand Jain
When one device fails it has to be closed and marked as failed. Further it needs sysfs (or some) interface to provide complete information about the device and the volume status to the user land from the kernel. Next when the disappeared device reappears we need to resilver/insync depending on the

Re: Lost about 3TB

2017-10-03 Thread Hugo Mills
On Tue, Oct 03, 2017 at 05:45:54PM +0200, fred.lar...@free.fr wrote: > Hi, > > > > What does "btrfs sub list -a /RAID01/" say? > Nothing (no lines displayed) > > > Also "grep /RAID01/ /proc/self/mountinfo"? > Nothing (no lines displayed) > > > Also server has been rebooted many times and

Re: Lost about 3TB

2017-10-03 Thread fred . larive
Hi, > What does "btrfs sub list -a /RAID01/" say? Nothing (no lines displayed) > Also "grep /RAID01/ /proc/self/mountinfo"? Nothing (no lines displayed) Also server has been rebooted many times and no process has left "deleted open files" on the volume (lsof...). Fred. - Mail

[PATCH 1/4] Btrfs: compress_file_range() remove dead variable num_bytes

2017-10-03 Thread Timofey Titovets
Remove dead assigment of num_bytes Also as num_bytes only used in will_compress block as copy of total_in just replace that with total_in and drop num_bytes entire Signed-off-by: Timofey Titovets Reviewed-by: Nikolay Borisov --- fs/btrfs/inode.c | 10

[PATCH 4/4] Btrfs: btrfs_dedupe_file_range() ioctl, remove 16MiB restriction

2017-10-03 Thread Timofey Titovets
At now btrfs_dedupe_file_range() restricted to 16MiB range for limit locking time and memory requirement for dedup ioctl() For too big input rage code silently set range to 16MiB Let's remove that restriction by do iterating over dedup range. That's backward compatible and will not change

[PATCH 3/4] Btrfs: handle unaligned tail of data ranges more efficient

2017-10-03 Thread Timofey Titovets
At now while switch page bits in data ranges we always hande +1 page, for cover case where end of data range is not page aligned Let's handle that case more obvious and efficient Check end aligment directly and touch +1 page only then needed Signed-off-by: Timofey Titovets

[PATCH 2/4] Btrfs: clear_dirty only on pages only in compression range

2017-10-03 Thread Timofey Titovets
We need to call extent_range_clear_dirty_for_io() on compression range to prevent application from changing page content, while pages compressing. but "(end - start)" can be much (up to 1024 times) bigger then compression range (BTRFS_MAX_UNCOMPRESSED), so optimize that by calculating compression

[PATCH 0/4] Just bunch of btrfs patches

2017-10-03 Thread Timofey Titovets
Some patches has review, some not, all compile tested and hand tested. (i.e. boot into patched system and do some small tests). All based on kDave for-next branch Patches: 1. Just remove useless u64 num_bytes from compress_file_range() No functional changes 2. For make compression on on

Kickstarting snapshot-aware defrag?

2017-10-03 Thread Niccolò Belli
Hi, It seems to me that the proposal[1] for a snapshot-aware defrag has long been abandoned. Since most peoples badly need this feature I tought about how to possibly speed up the achievement of this goal. I know of several bounty-based kickstarting platforms, among them the best ones are

Something like ZFS Channel Programs for BTRFS & probably XFS or even VFS?

2017-10-03 Thread Martin Steigerwald
[repost. I didn´t notice autocompletion gave me wrong address for fsdevel, blacklisted now] Hello. What do you think of http://open-zfs.org/wiki/Projects/ZFS_Channel_Programs ? There are quite some BTRFS maintenance programs like the deduplication stuff. Also regular scrubs… and in certain

Re: Lost about 3TB

2017-10-03 Thread Roman Mamedov
On Tue, 3 Oct 2017 10:54:05 + Hugo Mills wrote: >There are other possibilities for missing space, but let's cover > the obvious ones first. One more obvious thing would be files that are deleted, but still kept open by some app (possibly even from network, via NFS or

Re: Lost about 3TB

2017-10-03 Thread Timofey Titovets
2017-10-03 13:54 GMT+03:00 Hugo Mills : > On Tue, Oct 03, 2017 at 12:44:29PM +0200, btrfs.fr...@xoxy.net wrote: >> Hi, >> >> I can't figure out were 3TB on a 36 TB BTRFS volume (on LVM) are gone ! >> >> I know BTRFS can be tricky when speaking about space usage when using many

Re: Lost about 3TB

2017-10-03 Thread Hugo Mills
On Tue, Oct 03, 2017 at 12:44:29PM +0200, btrfs.fr...@xoxy.net wrote: > Hi, > > I can't figure out were 3TB on a 36 TB BTRFS volume (on LVM) are gone ! > > I know BTRFS can be tricky when speaking about space usage when using many > physical drives in a RAID setup, but my conf is a very simple

Lost about 3TB

2017-10-03 Thread btrfs . fredo
Hi, I can't figure out were 3TB on a 36 TB BTRFS volume (on LVM) are gone ! I know BTRFS can be tricky when speaking about space usage when using many physical drives in a RAID setup, but my conf is a very simple BTRFS volume without RAID(single Data type) using the whole disk (perhaps did I

Something like ZFS Channel Programs for BTRFS & probably XFS or even VFS?

2017-10-03 Thread Martin Steigerwald
Hello. What do you think of http://open-zfs.org/wiki/Projects/ZFS_Channel_Programs ? There are quite some BTRFS maintenance programs like the deduplication stuff. Also regular scrubs… and in certain circumstances probably balances can make sense. In addition to this XFS got scrub

[PATCH] btrfs-progs: doc: update help/document of btrfs device remove

2017-10-03 Thread Misono, Tomohiro
This patch updates help/document of "btrfs device remove" in two points: 1. Add explanation of 'missing' for 'device remove'. This is only written in wikipage currently. (https://btrfs.wiki.kernel.org/index.php/Using_Btrfs_with_Multiple_Devices) 2. Add example of device removal in the man

[PATCH] btrfs-progs: misc-test: use raid1 for data to enable mount with -o degraded

2017-10-03 Thread Misono, Tomohiro
kernel 4.14 introduces new function for checking if all chunks is ok for mount with -o degraded option. commit 21634a19f646 ("btrfs: Introduce a function to check if all chunks a OK for degraded rw mount") As a result, raid0 profile cannot be mounted with -o degraded on 4.14. This causes