Re: state of btrfs snapshot limitations?

2018-09-14 Thread James A. Robinson
Wow, thank to everyone for all that information, I'm going to have to take some time to digest everything. :) I just wanted to quickly say one thing: As Duncan surmised, I'm not treating this as my primary backup, but more of an experimental add-on feature. The primary backup goes to an ext4 part

Re: state of btrfs snapshot limitations?

2018-09-14 Thread Duncan
James A. Robinson posted on Fri, 14 Sep 2018 14:05:29 -0700 as excerpted: > The mail archive seems to indicate this list is appropriate for not only > the technical coding issues, but also for user questions, so I wanted to > pose a question here. If I'm wrong about that, I apologize in advance.

Re: state of btrfs snapshot limitations?

2018-09-14 Thread Chris Murphy
On Fri, Sep 14, 2018 at 3:05 PM, James A. Robinson wrote: > https://btrfs.wiki.kernel.org/index.php/Incremental_Backup > > talks about the basic snapshot capabilities of btrfs and led > me to look up what, if any, limits might apply. I find some > threads from a few years ago that talk about lim

Re: state of btrfs snapshot limitations?

2018-09-14 Thread James A. Robinson
Thanks very much for the useful information. I'll give the simple scheme a try, after I adjust mount preferences. Jim

Re: state of btrfs snapshot limitations?

2018-09-14 Thread Hans van Kranenburg
Hi, On 09/14/2018 11:05 PM, James A. Robinson wrote: > The mail archive seems to indicate this list is appropriate > for not only the technical coding issues, but also for user > questions, so I wanted to pose a question here. If I'm > wrong about that, I apologize in advance. It's fine. Your ob

state of btrfs snapshot limitations?

2018-09-14 Thread James A. Robinson
The mail archive seems to indicate this list is appropriate for not only the technical coding issues, but also for user questions, so I wanted to pose a question here. If I'm wrong about that, I apologize in advance. The page https://btrfs.wiki.kernel.org/index.php/Incremental_Backup talks abou

[PATCH v2] Btrfs: remove wait_ordered_rane in btrfs_evict_inode

2018-09-14 Thread Liu Bo
When we delete an inode, btrfs_evict_inode() { truncate_inode_pages_final() truncate_inode_pages_range() lock_page() truncate_cleanup_page() btrfs_invalidatepage() wait_on_page_writeback btrfs_loo

Re: Problem with BTRFS

2018-09-14 Thread Nicholas D Steeves
Hi, On Fri, Sep 14, 2018 at 10:13:06PM +0200, Rafael Jesús Alcántara Pérez wrote: > Hi, > > It seems that btrfs-progs_4.17-1 from Sid, includes that feature (at > least, it says so in the manual page). I don't know if I can install it > on Stretch but I'll try. > > Greets and thank you very much

Re: Problem with BTRFS

2018-09-14 Thread Rafael Jesús Alcántara Pérez
Hi, It seems that btrfs-progs_4.17-1 from Sid, includes that feature (at least, it says so in the manual page). I don't know if I can install it on Stretch but I'll try. Greets and thank you very much to both of you ;) Rafael J. Alcántara Pérez. El 14/09/18 a las 20:18, Nicholas D Steeves escrib

Re: [PATCH] Btrfs: remove wait_ordered_range in btrfs_evict_inode

2018-09-14 Thread Liu Bo
On Fri, Sep 14, 2018 at 03:14:33PM +0200, David Sterba wrote: > On Wed, Sep 12, 2018 at 06:06:22AM +0800, Liu Bo wrote: > > As VFS has called ->invalidatepage() to get all ordered extents done > > and truncated all page cache pages, wait_ordered_range() is just a > > noop. > > Agreed, though looki

Re: [PATCH 0/5] rb_first to rb_first_cached conversion

2018-09-14 Thread Liu Bo
On Fri, Sep 14, 2018 at 05:11:02PM +0200, David Sterba wrote: > On Tue, Sep 11, 2018 at 11:31:49AM -0700, Liu Bo wrote: > > On Tue, Sep 11, 2018 at 05:34:03PM +0200, David Sterba wrote: > > > On Thu, Aug 23, 2018 at 03:51:48AM +0800, Liu Bo wrote: > > > > Several structs in btrfs are using rb_first

Re: Problem with BTRFS

2018-09-14 Thread Nicholas D Steeves
Hi, On Fri, Sep 14, 2018 at 10:46:12PM +0500, Roman Mamedov wrote: > On Fri, 14 Sep 2018 19:27:04 +0200 > Rafael Jesús Alcántara Pérez wrote: > > > BTRFS info (device sdc1): use lzo compression, level 0 > > BTRFS warning (device sdc1): 'recovery' is deprecated, use > > 'usebackuproot' in

Re: Problem with BTRFS

2018-09-14 Thread Roman Mamedov
On Fri, 14 Sep 2018 19:27:04 +0200 Rafael Jesús Alcántara Pérez wrote: > BTRFS info (device sdc1): use lzo compression, level 0 > BTRFS warning (device sdc1): 'recovery' is deprecated, use > 'usebackuproot' instead > BTRFS info (device sdc1): trying to use backup root at mount time > BTRF

Problem with BTRFS

2018-09-14 Thread Rafael Jesús Alcántara Pérez
Hi: After a power outage, my server refuses to mount a BTRFS file system. This is the result of trying to mount with options recovery,ro: BTRFS info (device sdc1): use lzo compression, level 0 BTRFS warning (device sdc1): 'recovery' is deprecated, use 'usebackuproot' instead BTRFS info (device s

Re: Problem with BTRFS

2018-09-14 Thread Rafael Jesús Alcántara Pérez
Hi again: Sorry, I missed some details :) $ uname -a Linux gemini 4.17.0-0.bpo.3-amd64 #1 SMP Debian 4.17.17-1~bpo9+1 (2018-08-27) x86_64 GNU/Linux $ btrfs --version btrfs-progs v4.13.3 $ sudo btrfs fi show Label: '/srv/dedalo' uuid: d1071744-ac3b-4926-bb43-27091ea73c05 Total devices 4

Re: [PATCH 05/11] btrfs: dev-replace: avoid useless lock on error handling path

2018-09-14 Thread David Sterba
On Fri, Sep 07, 2018 at 04:55:11PM +0200, David Sterba wrote: > The exit sequence in btrfs_dev_replace_start does not allow to simply > add a label to the right place so the error handling after starting > transaction failure jumps there. Currently there's a lock that pairs > with the unlock in the

[PATCH 4/4 v2] btrfs: tests: polish ifdefs around testing helper

2018-09-14 Thread David Sterba
Avoid the inline ifdefs and use two sections for self-tests enabled and disabled. Though there could be no ifdef and unconditional test_bit of BTRFS_FS_STATE_DUMMY_FS_INFO, the static inline can help to optimize out any code that would depend on conditions using btrfs_is_testing. As this is only

[PATCH 1/4 v2] btrfs: tests: add separate stub for find_lock_delalloc_range

2018-09-14 Thread David Sterba
The helper find_lock_delalloc_range is now conditionally built static, dpending on whether the self-tests are enabled or not. There's a macro that is supposed to hide the export, used only once. To discourage further use, drop it an add a public wrapper for the helper needed by tests. Signed-off-b

Re: [PATCH 0/3] btrfs: trim latency improvements

2018-09-14 Thread David Sterba
On Thu, Sep 06, 2018 at 05:18:13PM -0400, je...@suse.com wrote: > From: Jeff Mahoney > > This patch set fixes a few issues with trim. > > 1) Fix device list iteration. We're iterating the ->alloc_list while >holding the device_list_mutex. The ->alloc_list is protected by >the chunk mut

Re: [PATCH 3/3] btrfs: keep trim from interfering with transaction commits

2018-09-14 Thread David Sterba
On Thu, Sep 06, 2018 at 05:18:16PM -0400, je...@suse.com wrote: > From: Jeff Mahoney > > Commit 499f377f49f08 (btrfs: iterate over unused chunk space in FITRIM) > fixed free space trimming, but introduced latency when it was running. > This is due to it pinning the transaction using both a increm

Re: [PATCH 2/3] btrfs: don't attempt to trim devices that don't support it

2018-09-14 Thread David Sterba
On Thu, Sep 06, 2018 at 05:18:15PM -0400, je...@suse.com wrote: > From: Jeff Mahoney > > We check whether any device the file system is using supports discard > in the ioctl call, but then we attempt to trim free extents on every > device regardless of whether discard is supported. Due to the wa

Re: [PATCH 1/3] btrfs: use ->devices list instead of ->alloc_list in btrfs_trim_fs

2018-09-14 Thread David Sterba
On Thu, Sep 06, 2018 at 05:18:14PM -0400, je...@suse.com wrote: > From: Jeff Mahoney > > btrfs_trim_fs iterates over the fs_devices->alloc_list while holding > the device_list_mutex. The problem is that ->alloc_list is protected > by the chunk mutex. We don't want to hold the chunk mutex over >

Re: [PATCH v3.1 2/2] btrfs: Ensure btrfs_trim_fs can trim the whole fs

2018-09-14 Thread David Sterba
On Fri, Sep 07, 2018 at 02:16:24PM +0800, Qu Wenruo wrote: > [BUG] > fstrim on some btrfs only trims the unallocated space, not trimming any > space in existing block groups. > > [CAUSE] > Before fstrim_range passed to btrfs_trim_fs(), it get truncated to > range [0, super->total_bytes). > So late

Re: [PATCH v3.1 1/2] btrfs: Enhance btrfs_trim_fs function to handle error better

2018-09-14 Thread David Sterba
On Fri, Sep 07, 2018 at 02:16:23PM +0800, Qu Wenruo wrote: > + if (bg_failed) > + btrfs_warn(fs_info, "failed to trim %llu block group(s)", > +bg_failed); > + if (dev_failed) > + btrfs_warn(fs_info, "failed to trim %llu device(s)", > +

Re: [PATCH v3.1 1/2] btrfs: Enhance btrfs_trim_fs function to handle error better

2018-09-14 Thread David Sterba
On Fri, Sep 07, 2018 at 02:16:23PM +0800, Qu Wenruo wrote: > Function btrfs_trim_fs() doesn't handle errors in a consistent way, if > error happens when trimming existing block groups, it will skip the > remaining blocks and continue to trim unallocated space for each device. > > And the return va

Re: [PATCH] btrfs: fix error handling in btrfs_dev_replace_start

2018-09-14 Thread David Sterba
On Mon, Sep 10, 2018 at 07:47:29PM +0200, David Sterba wrote: > On Thu, Sep 06, 2018 at 03:52:17PM -0400, je...@suse.com wrote: > > From: Jeff Mahoney > > > > When we fail to start a transaction in btrfs_dev_replace_start, > > we leave dev_replace->replace_start set to STARTED but clear > > ->src

Re: [PATCH 0/8 V2] Add delayed-refs support to btrfs-progs

2018-09-14 Thread David Sterba
On Thu, Aug 16, 2018 at 04:10:27PM +0300, Nikolay Borisov wrote: > Hello, > > Here is the second version of the delayed refs for progs support. The first > version can be found here [1]. I've taken into account all the feedback from > Misono and have verified the code is working and rebased it

Re: [PATCH 0/5] rb_first to rb_first_cached conversion

2018-09-14 Thread David Sterba
On Tue, Sep 11, 2018 at 11:31:49AM -0700, Liu Bo wrote: > On Tue, Sep 11, 2018 at 05:34:03PM +0200, David Sterba wrote: > > On Thu, Aug 23, 2018 at 03:51:48AM +0800, Liu Bo wrote: > > > Several structs in btrfs are using rb_first() in a while loop, it'd be > > > more efficient to do this with rb_fi

Re: [PATCH] btrfs-progs: change filename limit to 255 when creating subvolume

2018-09-14 Thread David Sterba
On Wed, Sep 12, 2018 at 03:39:03PM +0800, Su Yanjun wrote: > Modify the file name length limit to meet the Linux naming convention. > In addition, the file name length is always bigger than 0, no need to > compare with 0 again. > > Issue: #145 > Signed-off-by: Su Yanjun Looks good, please send

Re: [PATCH] btrfs-progs: Continue checking even we found something wrong in free space cache

2018-09-14 Thread David Sterba
On Wed, Sep 12, 2018 at 07:43:27AM +0800, Qu Wenruo wrote: > > > On 2018/9/11 下午10:48, David Sterba wrote: > > On Tue, Sep 04, 2018 at 08:41:28PM +0800, Qu Wenruo wrote: > >> No need to abort checking, especially for RO check free space cache is > >> meaningless, the errors in fs/extent tree is m

Re: [PATCH 0/6] btrfs-progs: Variant fixes for fuzz-tests

2018-09-14 Thread David Sterba
On Wed, Sep 12, 2018 at 07:59:34AM +0800, Qu Wenruo wrote: > > > On 2018/9/12 上午12:38, David Sterba wrote: > > On Wed, Aug 29, 2018 at 01:27:39PM +0800, Qu Wenruo wrote: > >> Gentle ping. > >> > >> These fixes are pretty small, I'd like to see them merged before I need > >> to rebase them again a

Re: [PATCH 4/4] btrfs: tests: polish ifdefs around testing helper

2018-09-14 Thread David Sterba
On Tue, Sep 11, 2018 at 12:14:47PM -0700, Omar Sandoval wrote: > > The unlikely can go away, sure. > > > > I would still like to remove the test_bit call when tests are compiled > > out. There are about 10 calls to btrfs_is_testing in various core > > functions, followed by further statements. Thi

Re: [PATCH] Btrfs: skip setting path to blocking mode if balance is not needed

2018-09-14 Thread David Sterba
On Wed, Sep 12, 2018 at 09:51:33AM +0300, Nikolay Borisov wrote: > > > On 12.09.2018 01:06, Liu Bo wrote: > > balance_level() may return early in some cases, but these checks don't > > have to be done with blocking write lock. > > > > This puts together these checks into a helper and the benefit

Re: [PATCH] btrfs: Remove logically dead code from btrfs_orphan_cleanup

2018-09-14 Thread David Sterba
On Thu, Sep 13, 2018 at 11:35:00AM +0300, Nikolay Borisov wrote: > In btrfs_orphan_cleanup the final 'if (ret) goto out' cannot ever be > executed. This is due to the last assignment to 'ret' depending on > the return value of btrfs_iget. If an error other than -ENOENT is > returned then the loop i

Re: [PATCH] Btrfs: remove wait_ordered_range in btrfs_evict_inode

2018-09-14 Thread David Sterba
On Wed, Sep 12, 2018 at 06:06:22AM +0800, Liu Bo wrote: > As VFS has called ->invalidatepage() to get all ordered extents done > and truncated all page cache pages, wait_ordered_range() is just a > noop. Agreed, though looking up the exact points when there are no pages to be waited for took me so

Re: [PATCH v2] Btrfs: skip set_page_dirty if eb is dirty

2018-09-14 Thread David Sterba
On Fri, Sep 14, 2018 at 01:44:42AM +0800, Liu Bo wrote: > As long as @eb is marked with EXTENT_BUFFER_DIRTY, all of its pages > are dirty, so no need to set pages dirty again. > > Ftrace showed that the loop took 10us on my dev box, so removing this > can save us at least 10us if eb is already dir

Re: [PATCH v2] Btrfs: assert page dirty bit

2018-09-14 Thread David Sterba
On Fri, Sep 14, 2018 at 01:46:08AM +0800, Liu Bo wrote: > Just in case that someone breaks the rule that pages are dirty as long > as eb is dirty. > > Signed-off-by: Liu Bo Reviewed-by: David Sterba I've reordered the patches so the assert comes first, before the actual conditional dirtying.

Re: [PATCH v2] Btrfs: remove level==0 check in balance_level

2018-09-14 Thread David Sterba
On Fri, Sep 14, 2018 at 01:55:59AM +0800, Liu Bo wrote: > btrfs_search_slot() >if (level != 0) > setup_nodes_for_search() > balance_level() > > It is just impossible to have level=0 in balance_level. > > Reviewed-by: Nikolay Borisov > Signed-off-by: Liu Bo Reviewed-by: Davi

Re: btrfs filesystem show takes a long time

2018-09-14 Thread David Sterba
Hi, thanks for the report, I've forwarded it to the issue tracker https://github.com/kdave/btrfs-progs/issues/148 The show command uses the information provided by blkid, that presumably caches that. The default behaviour of 'fi show' is to skip mount checks, so the delays are likely caused by bl

btrfs filesystem show takes a long time

2018-09-14 Thread Stefan K
Dear Maintainer, the command btrfs fi show takes too much time: time btrfs fi show Label: none uuid: 513dc574-e8bc-4336-b181-00d1e9782c1c Total devices 2 FS bytes used 2.34GiB devid1 size 927.79GiB used 4.03GiB path /dev/sdv2 devid2 size 927.79GiB used 4.03GiB path /dev/sda

[PATCH v1.1] btrfs-progs: Do metadata prealloc as long as we're not modifying extent tree

2018-09-14 Thread Qu Wenruo
In github issues, one user reports unexpected ENOSPC error if enabling datasum. After some investigation, it looks like that during ext2_saved/image creation, we could create large file extent whose size can be 128M (max data extent size). In that case, its csum will be at least 128K. Under certai

[PATCH] btrfs-progs: Do metadata prealloc as long as we're not modifying csum tree

2018-09-14 Thread Qu Wenruo
In github issues, one user reports unexpected ENOSPC error if enabling datasum. After some investigation, it looks like that during ext2_saved/image creation, we could create large file extent whose size can be 128M (max data extent size). In that case, its csum will be at least 128K. Under certai

Re: [PATCH 2/2] btrfs-progs: convert: Output meaningful error messages for create_image()

2018-09-14 Thread Nikolay Borisov
On 14.09.2018 10:25, Qu Wenruo wrote: > When convert failed, the error messsage would look like: > > create btrfs filesystem: > blocksize: 4096 > nodesize: 16384 > features: extref, skinny-metadata (default) > creating ext2 image file > ERROR: failed to create ext2_sav

Re: [PATCH 1/2] btrfs: convert: Make read_disk_extent() return more meaningful -EIO other -1

2018-09-14 Thread Nikolay Borisov
On 14.09.2018 10:25, Qu Wenruo wrote: > When pread64() returns value smaller than expected, it normally means > EIO, so just return -EIO to replace the intermediate number. > So when IO fails, we should be able to get more meaningful error number > of than EPERM. > > Signed-off-by: Qu Wenruo R

[PATCH 2/2] btrfs-progs: convert: Output meaningful error messages for create_image()

2018-09-14 Thread Qu Wenruo
When convert failed, the error messsage would look like: create btrfs filesystem: blocksize: 4096 nodesize: 16384 features: extref, skinny-metadata (default) creating ext2 image file ERROR: failed to create ext2_saved/image: -1 WARNING: an error occurred during conversi

[PATCH 1/2] btrfs: convert: Make read_disk_extent() return more meaningful -EIO other -1

2018-09-14 Thread Qu Wenruo
When pread64() returns value smaller than expected, it normally means EIO, so just return -EIO to replace the intermediate number. So when IO fails, we should be able to get more meaningful error number of than EPERM. Signed-off-by: Qu Wenruo --- convert/source-fs.c | 2 +- 1 file changed, 1 ins

[PATCH 0/2] Enhanced error messages for btrfs-convert

2018-09-14 Thread Qu Wenruo
This patchset can be fetched from github: https://github.com/adam900710/btrfs-progs/tree/convert_error_messages As usual, it's based on latest stable tag (v4.17.1). There is one error report of btrfs-convert, the error message looks pretty meaningless: create btrfs filesystem: blocksize:

Re: [PATCH v2 4/7] btrfs-progs: lowmem: search key of root again after check_fs_root() under repair

2018-09-14 Thread Su Yue
On 09/14/2018 02:27 PM, Nikolay Borisov wrote: On 14.09.2018 03:58, Su Yue wrote: On 09/14/2018 07:37 AM, Qu Wenruo wrote: On 2018/9/13 上午4:49, damenly...@gmail.com wrote: From: Su Yue In check_fs_roots_lowmem(), we do search and follow the resulted path to call check_fs_root(), the