Re: [survey] BTRFS_IOC_DEVICES_READY return status

2015-06-15 Thread Lennart Poettering
On Fri, 12.06.15 21:16, Anand Jain (anand.j...@oracle.com) wrote: BTRFS_IOC_DEVICES_READY is to check if all the required devices are known by the btrfs kernel, so that admin/system-application could mount the FS. It is checked against a device in the argument. However the actual

Re: @ prefix in subvolume paths

2015-06-15 Thread Gábor Nyers
Jeff, Thanks for elaborating on this. I'm wondering where in the SLES/openSUSE documentation these considerations are documented. Do you have a pointer? Thanks, Gábor On Wed, 10 Jun 2015 21:50:19 -0400 Jeff Mahoney je...@suse.com wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On

CoW with webserver databases: innodb_file_per_table and dedicated tables for blobs?

2015-06-15 Thread Ingvar Bogdahn
Hello there, I'm planing to use btrfs for a medium-sized webserver. It is commonly recommended to set nodatacow for database files to avoid performance degradation. However, apparently nodatacow disables some of my main motivations of using btrfs : checksumming and (probably) incremental

Re: CoW with webserver databases: innodb_file_per_table and dedicated tables for blobs?

2015-06-15 Thread Hugo Mills
On Mon, Jun 15, 2015 at 11:34:35AM +0200, Ingvar Bogdahn wrote: Hello there, I'm planing to use btrfs for a medium-sized webserver. It is commonly recommended to set nodatacow for database files to avoid performance degradation. However, apparently nodatacow disables some of my main

Re: [systemd-devel] [survey] BTRFS_IOC_DEVICES_READY return status

2015-06-15 Thread Lennart Poettering
On Sat, 13.06.15 17:35, Anand Jain (anand.j...@oracle.com) wrote: Are there any other users? - If the the device in the argument is already mounted, can it straightaway return 0 (ready) ? (as of now it would again independently read the SB determine total_devices and

Re: [systemd-devel] [survey] BTRFS_IOC_DEVICES_READY return status

2015-06-15 Thread Lennart Poettering
On Sat, 13.06.15 17:09, Goffredo Baroncelli (kreij...@libero.it) wrote: Further, the problem will be more intense in this eg. if you use dd and copy device A to device B. After you mount device A, by just providing device B in the above two commands you could let kernel update the device

Re: kernel crashes with btrfs and busy database IO - how to debug?

2015-06-15 Thread Tomasz Chmielewski
Hello, Do you want me to produce one more crash / hang? I had to restart the hanged server (via echo b /proc/sysrq-trigger). Tomasz On 2015-06-15 17:10, Qu Wenruo wrote: Now we can get the full backtrace. That's a step forward [45705.854778] BUG: unable to handle kernel NULL pointer

Re: Bad performance with near-full FS

2015-06-15 Thread David Sterba
On Mon, Jun 15, 2015 at 02:47:24PM +0200, Tovo Rabemanantsoa wrote: Hi all, By browsing this list's archive, I've found a thread initiated by Charles Cazabon entitled: Oddly slow read performance with near-full largish FS. Actually, I'm living the same experience but with a not so large FS

[PATCH v5] btrfs: fix automatic blockgroup remove + discard

2015-06-15 Thread Jeff Mahoney
Apologies for the out-of-order post. git send-email caught ^C at just the right time and send them without my confirmation. That's also why the map-lookup casting patch got mixed in here. The automatic block group removal patch introduced some regressions in how discards are handled. 1/ FITRIM

Re: [PATCH 1/2] Btrfs: use kmem_cache_free when freeing entry in inode cache

2015-06-15 Thread David Sterba
On Sat, Jun 13, 2015 at 06:52:56AM +0100, fdman...@kernel.org wrote: From: Filipe Manana fdman...@suse.com The free space entries are allocated using kmem_cache_zalloc(), through __btrfs_add_free_space(), therefore we should use kmem_cache_free() and not kfree() to avoid any confusion and

Bad performance with near-full FS

2015-06-15 Thread Tovo Rabemanantsoa
Hi all, By browsing this list's archive, I've found a thread initiated by Charles Cazabon entitled: Oddly slow read performance with near-full largish FS. Actually, I'm living the same experience but with a not so large FS (256GB on a SSD). Indeed, when I have less than 1GB of free space, the

[PATCH] Btrfs: fix typo in the error log

2015-06-15 Thread Anand Jain
--- fs/btrfs/disk-io.c | 2 +- fs/btrfs/super.c | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 57d48f8..5147cb7 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -2896,7 +2896,7 @@ retry_root_backup:

Re: [PATCH RFC] btrfs: csum: Introduce partial csum for tree block.

2015-06-15 Thread David Sterba
On Mon, Jun 15, 2015 at 04:02:49PM +0800, Qu Wenruo wrote: In the following case of corruption, RAID1 or DUP will fail to recover it(Use 16K as leafsize) 0 4K 8K 12K 16K Mirror 0: |-OK--|ERROR|-OK-|

[PATCH 2/7] btrfs: btrfs_issue_discard ensure offset/length are aligned to sector boundaries

2015-06-15 Thread jeffm
From: Jeff Mahoney je...@suse.com It's possible, though unexpected, to pass unaligned offsets and lengths to btrfs_issue_discard. We then shift the offset/length values to sector units. If an unaligned offset has been passed, it will result in the entire sector being discarded, possibly losing

[PATCH 7/7] btrfs: cleanup, stop casting for extent_map-lookup everywhere

2015-06-15 Thread jeffm
From: Jeff Mahoney je...@suse.com Overloading extent_map-bdev to struct map_lookup * might have started out as a means to an end, but it's a pattern that's used all over the place now. Let's get rid of the casting and just add a union instead. Signed-off-by: Jeff Mahoney je...@suse.com ---

[PATCH 5/7] btrfs: explictly delete unused block groups in close_ctree and ro-remount

2015-06-15 Thread jeffm
From: Jeff Mahoney je...@suse.com The cleaner thread may already be sleeping by the time we enter close_ctree. If that's the case, we'll skip removing any unused block groups queued for removal, even during a normal umount. They'll be cleaned up automatically at next mount, but users expect a

[PATCH 3/7] btrfs: skip superblocks during discard

2015-06-15 Thread jeffm
From: Jeff Mahoney je...@suse.com Btrfs doesn't track superblocks with extent records so there is nothing persistent on-disk to indicate that those blocks are in use. We track the superblocks in memory to ensure they don't get used by removing them from the free space cache when we load a block

[PATCH 1/7] btrfs: make btrfs_issue_discard return bytes discarded

2015-06-15 Thread jeffm
From: Jeff Mahoney je...@suse.com Initially this will just be the length argument passed to it, but the following patches will adjust that to reflect re-alignment and skipped blocks. Signed-off-by: Jeff Mahoney je...@suse.com --- fs/btrfs/extent-tree.c | 19 ++- 1 file changed,

[PATCH 6/7] btrfs: add missing discards when unpinning extents with -o discard

2015-06-15 Thread jeffm
From: Jeff Mahoney je...@suse.com When we clear the dirty bits in btrfs_delete_unused_bgs for extents in the empty block group, it results in btrfs_finish_extent_commit being unable to discard the freed extents. The block group removal patch added an alternate path to forget extents other than

[PATCH 4/7] btrfs: iterate over unused chunk space in FITRIM

2015-06-15 Thread jeffm
From: Jeff Mahoney je...@suse.com Since we now clean up block groups automatically as they become empty, iterating over block groups is no longer sufficient to discard unused space. This patch iterates over the unused chunk space and discards any regions that are unallocated, regardless of

Re: multiple devices with multiple partitions on one SSD

2015-06-15 Thread Calvin Walton
On Fri, 2015-06-12 at 18:11 +0800, Tom Yan wrote: Thank you for you reply. I thought of the same thing as you've told, but because I don't really know how SSD/SATA works internally, so I was also thinking if raid0 on the same drive might works as something like Hyper-Threading in CPU. Modern

Re: Bad performance with near-full FS

2015-06-15 Thread Clemens Eisserer
Hi, Thanks for you reply, If I really understand, it's always a good idea to keep more than 1% of free space. Right ? Usually the rule of thumb is 10-20%. Br, Clemens -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to

Re: Bad performance with near-full FS

2015-06-15 Thread Austin S Hemmelgarn
On 2015-06-15 10:44, Tovo Rabemanantsoa wrote: On 06/15/2015 03:29 PM, David Sterba wrote: On Mon, Jun 15, 2015 at 02:47:24PM +0200, Tovo Rabemanantsoa wrote: Hi all, By browsing this list's archive, I've found a thread initiated by Charles Cazabon entitled: Oddly slow read performance with

Re: [survey] BTRFS_IOC_DEVICES_READY return status

2015-06-15 Thread David Sterba
On Fri, Jun 12, 2015 at 09:16:30PM +0800, Anand Jain wrote: BTRFS_IOC_DEVICES_READY is to check if all the required devices are known by the btrfs kernel, so that admin/system-application could mount the FS. It is checked against a device in the argument. However the actual implementation is

Re: Bad performance with near-full FS

2015-06-15 Thread Tovo Rabemanantsoa
On 06/15/2015 03:29 PM, David Sterba wrote: On Mon, Jun 15, 2015 at 02:47:24PM +0200, Tovo Rabemanantsoa wrote: Hi all, By browsing this list's archive, I've found a thread initiated by Charles Cazabon entitled: Oddly slow read performance with near-full largish FS. Actually, I'm living the

Re: [PATCH RFC] btrfs: csum: Introduce partial csum for tree block.

2015-06-15 Thread Qu Wenruo
David Sterba wrote on 2015/06/15 15:15 +0200: On Mon, Jun 15, 2015 at 04:02:49PM +0800, Qu Wenruo wrote: In the following case of corruption, RAID1 or DUP will fail to recover it(Use 16K as leafsize) 0 4K 8K 12K 16K Mirror 0:

Re: Uncorrectable errors on RAID6

2015-06-15 Thread Tobias Holst
Hi Qu, hi all, RO snapshot, I remember there is a RO snapshot bug, but seems fixed in 4.x? Yes, that bug has already been fixed. For recovery, first just try cp -r mnt/* to grab what's still completely OK. Maybe recovery mount option can do some help in the process? That's what I did now. I

Re: Uncorrectable errors on RAID6

2015-06-15 Thread Qu Wenruo
Tobias Holst wrote on 2015/06/16 03:31 +0200: Hi Qu, hi all, RO snapshot, I remember there is a RO snapshot bug, but seems fixed in 4.x? Yes, that bug has already been fixed. For recovery, first just try cp -r mnt/* to grab what's still completely OK. Maybe recovery mount option can do

Re: Can't mount btrfs volume on rbd

2015-06-15 Thread Steve Dainard
Hi Qu, # btrfs --version btrfs-progs v4.0.1 # btrfs check /dev/rbd30 Checking filesystem on /dev/rbd30 UUID: 1bb22a03-bc25-466f-b078-c66c6f6a6d28 checking extents cmds-check.c:3735: check_owner_ref: Assertion `rec-is_root` failed. btrfs[0x41aee6] btrfs[0x423f5d] btrfs[0x424c99] btrfs[0x4258f6]

Re: [systemd-devel] [survey] BTRFS_IOC_DEVICES_READY return status

2015-06-15 Thread Lennart Poettering
On Mon, 15.06.15 19:23, Goffredo Baroncelli (kreij...@inwind.it) wrote: On 2015-06-15 12:46, Lennart Poettering wrote: On Sat, 13.06.15 17:09, Goffredo Baroncelli (kreij...@libero.it) wrote: Further, the problem will be more intense in this eg. if you use dd and copy device A to device

Re: [systemd-devel] [survey] BTRFS_IOC_DEVICES_READY return status

2015-06-15 Thread Goffredo Baroncelli
On 2015-06-15 12:46, Lennart Poettering wrote: On Sat, 13.06.15 17:09, Goffredo Baroncelli (kreij...@libero.it) wrote: Further, the problem will be more intense in this eg. if you use dd and copy device A to device B. After you mount device A, by just providing device B in the above two

Re: Can't mount btrfs volume on rbd

2015-06-15 Thread Qu Wenruo
Steve Dainard wrote on 2015/06/15 09:19 -0700: Hi Qu, # btrfs --version btrfs-progs v4.0.1 # btrfs check /dev/rbd30 Checking filesystem on /dev/rbd30 UUID: 1bb22a03-bc25-466f-b078-c66c6f6a6d28 checking extents cmds-check.c:3735: check_owner_ref: Assertion `rec-is_root` failed. btrfs[0x41aee6]

Re: [PATCH RFC] btrfs: csum: Introduce partial csum for tree block.

2015-06-15 Thread Qu Wenruo
Qu Wenruo wrote on 2015/06/16 09:22 +0800: David Sterba wrote on 2015/06/15 15:15 +0200: On Mon, Jun 15, 2015 at 04:02:49PM +0800, Qu Wenruo wrote: In the following case of corruption, RAID1 or DUP will fail to recover it(Use 16K as leafsize) 04K8K12K16K

Re: [PATCH RFC] btrfs: csum: Introduce partial csum for tree block.

2015-06-15 Thread Qu Wenruo
On 06/11/2015 11:00 PM, Qu Wenruo wrote: Introduce the new partial csum mechanism for tree block. [Old tree block csum] 0 4 8121620242832 - |csum | unused, all 0 |

Re: Can't mount btrfs volume on rbd

2015-06-15 Thread Qu Wenruo
The debug result seems valid. So I'm afraid the problem is not in btrfs. Would your please try the following 2 things to eliminate btrfs problems? 1) btrfsck from 4.0.1 on the rbd If assert still happens, please update the image of the volume(dd image), to help us improve btrfs-progs. 2)

Re: kernel crashes with btrfs and busy database IO - how to debug?

2015-06-15 Thread Qu Wenruo
Now we can get the full backtrace. That's a step forward [45705.854778] BUG: unable to handle kernel NULL pointer dereference at 0008 [45705.854824] IP: [c0158b8e] btrfs_wait_pending_ordered+0x5e/0x110 [btrfs] [45705.855615] Call Trace: [45705.855637] [c015addb]