Re: btrfs: relocation: Fix leaking qgroups numbers on data extents

2016-07-29 Thread Qu Wenruo
Hi, Goldwyn, This patch is replaced by the following patchset: https://patchwork.kernel.org/patch/9213915/ https://patchwork.kernel.org/patch/9213913/ Would you mind testing the new patch? Thanks, Qu On 07/30/2016 02:42 AM, Goldwyn Rodrigues wrote: On Tue, Jun 14, 2016 at 05:26:22PM +0800, Q

Re: [PATCH 2/5] New btrfs command: "btrfs inspect physical-find"

2016-07-29 Thread Qu Wenruo
On 07/30/2016 01:14 AM, Goffredo Baroncelli wrote: On 2016-07-29 08:44, Qu Wenruo wrote: On 07/29/2016 01:08 PM, Goffredo Baroncelli wrote: On 2016-07-29 03:34, Qu Wenruo wrote: I am not against about your proposal; however I have to point out that the goal of these command was not to *tra

Re: memory overflow or undeflow in free space tree / space_info?

2016-07-29 Thread Holger Hoffstätte
On Fri, 29 Jul 2016 22:57:36 +, Holger Hoffstätte wrote: > The only other patch I just found missing and which looks like it > could/should (I think?) work on top of the 4.4.x pagesize-based > calculations in file.c is: > > a2af23b7 "__btrfs_buffered_write: Pass valid file offset when > rele

Re: memory overflow or undeflow in free space tree / space_info?

2016-07-29 Thread Holger Hoffstätte
On Fri, 29 Jul 2016 17:03:43 -0400, Josef Bacik wrote: > On 07/29/2016 03:14 PM, Omar Sandoval wrote: >> On Fri, Jul 29, 2016 at 12:11:53PM -0700, Omar Sandoval wrote: >>> On Fri, Jul 29, 2016 at 08:40:26PM +0200, Stefan Priebe - Profihost AG >>> wrote: Dear list, i'm seeing btrfs

Re: Send-recieve performance

2016-07-29 Thread Libor Klepáč
On pátek 22. července 2016 13:27:15 CEST Libor Klepáč wrote: > Hello, > > Dne pátek 22. července 2016 14:59:43 CEST, Henk Slager napsal(a): > > > On Wed, Jul 20, 2016 at 11:15 AM, Libor Klepáč > > wrote: > > > > Hello, > > > we use backuppc to backup our hosting machines. > > > > > > I have re

Re: btrfs on sparc64 results in kernel stack trace in 1 minute test

2016-07-29 Thread Jeff Mahoney
> On Jul 29, 2016, at 5:11 PM, Anatoly Pugachev wrote: > >> On Thu, Jul 14, 2016 at 1:29 PM, Filipe Manana wrote: >>> On Thu, Jul 14, 2016 at 11:08 AM, Anatoly Pugachev >>> wrote: >>> Hi! >>> >>> I'm using git (describe, v4.7-rc7-16-gcf875cc) kernel, >>> with patch "fix extent buffer bitmap

Re: btrfs on sparc64 results in kernel stack trace in 1 minute test

2016-07-29 Thread Anatoly Pugachev
On Thu, Jul 14, 2016 at 1:29 PM, Filipe Manana wrote: > On Thu, Jul 14, 2016 at 11:08 AM, Anatoly Pugachev wrote: >> Hi! >> >> I'm using git (describe, v4.7-rc7-16-gcf875cc) kernel, >> with patch "fix extent buffer bitmap tests on big-endian systems", see >> [1] (to be able to load/use btrfs modu

Re: memory overflow or undeflow in free space tree / space_info?

2016-07-29 Thread Josef Bacik
On 07/29/2016 03:14 PM, Omar Sandoval wrote: On Fri, Jul 29, 2016 at 12:11:53PM -0700, Omar Sandoval wrote: On Fri, Jul 29, 2016 at 08:40:26PM +0200, Stefan Priebe - Profihost AG wrote: Dear list, i'm seeing btrfs no space messages frequently on big filesystems (> 30TB). In all cases i'm gett

Re: memory overflow or undeflow in free space tree / space_info?

2016-07-29 Thread Stefan Priebe - Profihost AG
Am 29.07.2016 um 21:14 schrieb Omar Sandoval: > On Fri, Jul 29, 2016 at 12:11:53PM -0700, Omar Sandoval wrote: >> On Fri, Jul 29, 2016 at 08:40:26PM +0200, Stefan Priebe - Profihost AG wrote: >>> Dear list, >>> >>> i'm seeing btrfs no space messages frequently on big filesystems (> 30TB). >>> >>>

Re: memory overflow or undeflow in free space tree / space_info?

2016-07-29 Thread Stefan Priebe - Profihost AG
Am 29.07.2016 um 21:11 schrieb Omar Sandoval: > On Fri, Jul 29, 2016 at 08:40:26PM +0200, Stefan Priebe - Profihost AG wrote: >> Dear list, >> >> i'm seeing btrfs no space messages frequently on big filesystems (> 30TB). >> >> In all cases i'm getting a trace like this one a space_info warning. >>

Re: memory overflow or undeflow in free space tree / space_info?

2016-07-29 Thread Omar Sandoval
On Fri, Jul 29, 2016 at 12:11:53PM -0700, Omar Sandoval wrote: > On Fri, Jul 29, 2016 at 08:40:26PM +0200, Stefan Priebe - Profihost AG wrote: > > Dear list, > > > > i'm seeing btrfs no space messages frequently on big filesystems (> 30TB). > > > > In all cases i'm getting a trace like this one a

Re: memory overflow or undeflow in free space tree / space_info?

2016-07-29 Thread Omar Sandoval
On Fri, Jul 29, 2016 at 08:40:26PM +0200, Stefan Priebe - Profihost AG wrote: > Dear list, > > i'm seeing btrfs no space messages frequently on big filesystems (> 30TB). > > In all cases i'm getting a trace like this one a space_info warning. > (since commit [1]). Could someone please be so kind

Re: btrfs: relocation: Fix leaking qgroups numbers on data extents

2016-07-29 Thread Goldwyn Rodrigues
On Tue, Jun 14, 2016 at 05:26:22PM +0800, Qu Wenruo wrote: > When balancing data extents, qgroup will leak all its numbers for > balanced data extents. > > The root cause is the non-standard extent reference update used in > balance code. > > The problem happens in the following steps: > (Use 4M

memory overflow or undeflow in free space tree / space_info?

2016-07-29 Thread Stefan Priebe - Profihost AG
Dear list, i'm seeing btrfs no space messages frequently on big filesystems (> 30TB). In all cases i'm getting a trace like this one a space_info warning. (since commit [1]). Could someone please be so kind and help me debugging / fixing this bug? I'm using space_cache=v2 on all those systems. -

[PATCH v2] Btrfs: clarify do_chunk_alloc()'s return value

2016-07-29 Thread Liu Bo
Function start_transaction() can return ERR_PTR(1) when flush is BTRFS_RESERVE_FLUSH_LIMIT, so the call graph is start_transaction (return ERR_PTR(1)) -> btrfs_block_rsv_add (return 1) -> reserve_metadata_bytes (return 1) -> flush_space (return 1) -> do_chunk_alloc (retu

[PATCH v4] Btrfs: remove BUG() in raid56

2016-07-29 Thread Liu Bo
This BUG() has been triggered by a fuzz testing image, which contains an invalid chunk type, ie. a single stripe chunk has the raid6 type. Btrfs can handle this gracefully by returning -EIO, so besides using btrfs_warn to give us more debugging information rather than a single BUG(), we can return

Re: [PATCH] Btrfs: deal with unexpected return value in flush_space

2016-07-29 Thread Liu Bo
On Fri, Jul 29, 2016 at 07:01:50PM +0200, David Sterba wrote: > On Thu, Jul 28, 2016 at 11:49:14AM -0700, Liu Bo wrote: > > > For reviewers - this came up before here: > > > https://patchwork.kernel.org/patch/7778651/ David, this patch made a mistake in commit log. > > > > > > Same fix basically

Re: BTRFS: Transaction aborted (error -28)

2016-07-29 Thread Duncan
Mordechay Kaganer posted on Fri, 29 Jul 2016 16:37:05 +0300 as excerpted: > While saving a backup (using rsync) the FS went read only and i've got > BTRFS: Transaction aborted (error -28) error. While writing a new backup > i was also doing btrfs send of another subvolume on the same FS from > thi

Re: [PATCH 2/5] New btrfs command: "btrfs inspect physical-find"

2016-07-29 Thread Goffredo Baroncelli
On 2016-07-29 08:44, Qu Wenruo wrote: > > > On 07/29/2016 01:08 PM, Goffredo Baroncelli wrote: >> On 2016-07-29 03:34, Qu Wenruo wrote: I am not against about your proposal; however I have to point out that the goal of these command was not to *traverse* the file, but only to found

Re: [PATCH] Btrfs: deal with unexpected return value in flush_space

2016-07-29 Thread David Sterba
On Thu, Jul 28, 2016 at 11:49:14AM -0700, Liu Bo wrote: > > For reviewers - this came up before here: > > https://patchwork.kernel.org/patch/7778651/ > > > > Same fix basically. > > Aha, I've given it my Reviewed-by. > > Taking either one works for me, I can make the clarifying comment into a >

Re: [PATCH v3] Btrfs: remove BUG() in raid56

2016-07-29 Thread David Sterba
On Wed, Jul 27, 2016 at 11:56:47AM -0700, Liu Bo wrote: > This BUG() has been triggered by a fuzz testing image, which contains > an invalid chunk type, ie. a single stripe chunk has the raid6 type. > > Btrfs can handle this gracefully by returning -EIO, so besides using > btrfs_warn to give us mo

Re: Transaction aborted in btrfs_rename2

2016-07-29 Thread Jeff Mahoney
On 7/29/16 12:13 PM, Adam Borowski wrote: > On Fri, Jul 29, 2016 at 11:43:29AM -0400, Jeff Mahoney wrote: >> On 6/6/16 10:13 AM, Jeff Mahoney wrote: >>> On 6/6/16 7:47 AM, Adam Borowski wrote: Hi! I just got this thrice, in 4.7-rc1 and 4.7-rc2: [ 1836.672368] [ cut h

Re: Transaction aborted in btrfs_rename2

2016-07-29 Thread Adam Borowski
On Fri, Jul 29, 2016 at 11:43:29AM -0400, Jeff Mahoney wrote: > On 6/6/16 10:13 AM, Jeff Mahoney wrote: > > On 6/6/16 7:47 AM, Adam Borowski wrote: > >> Hi! > >> I just got this thrice, in 4.7-rc1 and 4.7-rc2: > >> > >> [ 1836.672368] [ cut here ] > >> [ 1836.672382] WARNING

Re: Transaction aborted in btrfs_rename2

2016-07-29 Thread Jeff Mahoney
On 6/6/16 10:13 AM, Jeff Mahoney wrote: > On 6/6/16 7:47 AM, Adam Borowski wrote: >> Hi! >> I just got this thrice, in 4.7-rc1 and 4.7-rc2: >> >> [ 1836.672368] [ cut here ] >> [ 1836.672382] WARNING: CPU: 1 PID: 16348 at fs/btrfs/inode.c:9820 >> btrfs_rename2+0xcd2/0x2a50

Re: [sparc64] mkfs.btrfs bus error / align issue?

2016-07-29 Thread Anatoly Pugachev
On Fri, Jul 29, 2016 at 3:41 PM, David Sterba wrote: > On Thu, Jul 28, 2016 at 11:34:58PM +0300, Anatoly Pugachev wrote: >> well, I think mkfs.btrfs is fixed, since I just tested it with : > > Good news, thanks. > > quick stats of the TPC messages: > > 23 __btrfs_map_block+0x36c/0x1180 >

Re: [PATCH] btrfs-progs: fix memory leak with missing device

2016-07-29 Thread David Sterba
On Thu, Jul 28, 2016 at 10:44:11AM -0700, Justin Maggard wrote: > In read_one_chunk(), we may add an empty entry for a missing device. > However, this entry wasn't being added to the dev_list, and so it never > got freed. > > Signed-off-by: Justin Maggard Applied (to 4.7), thanks. -- To unsubscr

Re: Btrfs send to send out metadata and data separately

2016-07-29 Thread Libor Klepáč
Hello, just a little question on receiver point 0), see bellow Dne pátek 29. července 2016 20:40:38 CEST, Qu Wenruo napsal(a): > Hi Filipe, and maintainers, > > Receive will do the following thing first before recovering the > subvolume/snapshot: > > 0) Create temporary dir for data extents >

BTRFS: Transaction aborted (error -28)

2016-07-29 Thread Mordechay Kaganer
B.H. Hello! While saving a backup (using rsync) the FS went read only and i've got BTRFS: Transaction aborted (error -28) error. While writing a new backup i was also doing btrfs send of another subvolume on the same FS from this server out to another server. After reboot, the device mounts OK.

Btrfs progs release 4.7

2016-07-29 Thread David Sterba
Hi, btrfs-progs 4.7 have been released. There are several new enhancements, a handful of bugfixes and other cleanups. The changes since rc1 are more fixes for unaligned access on sparc64. Changes: * convert: fix creating discontig extents * check: speed up traversing heavily reflinked extent

Re: [sparc64] mkfs.btrfs bus error / align issue?

2016-07-29 Thread David Sterba
On Thu, Jul 28, 2016 at 11:34:58PM +0300, Anatoly Pugachev wrote: > well, I think mkfs.btrfs is fixed, since I just tested it with : Good news, thanks. > Not sure what we need to do with sparc64 btrfs module TPC messages. > Probably fill kernel bugzilla report? Yes please. > [1] http://u163.eas

Btrfs send to send out metadata and data separately

2016-07-29 Thread Qu Wenruo
Hi Filipe, and maintainers, I'm recently working on the root fix to free send from calling backref walk. My current idea is to send data and metadata separately, and only do clone detection inside the send subvolume. This method needs two new send commands: (And new send attribute, A_DATA_BYT

Re: [sparc64] mkfs.btrfs bus error / align issue?

2016-07-29 Thread John Paul Adrian Glaubitz
On 07/28/2016 08:04 PM, David Sterba wrote: > And I've fixed it that way, now pushed to devel ("btrfs-progs: fix > unaligned access in raid6 calculations" [1]). Would be great if you or > Anatoly can test it so I can add it to the 4.7 release (ETA tomorrow). Awesome, thank you very much! Much appr

Re: [sparc64] mkfs.btrfs bus error / align issue?

2016-07-29 Thread Anatoly Pugachev
On Thu, Jul 28, 2016 at 11:34 PM, Anatoly Pugachev wrote: > On Thu, Jul 28, 2016 at 9:04 PM, David Sterba wrote: >> On Thu, Jul 28, 2016 at 04:28:41PM +0200, John Paul Adrian Glaubitz wrote: >>> On 07/28/2016 04:25 PM, John Paul Adrian Glaubitz wrote: >>> > On 07/28/2016 04:01 PM, Anatoly Pugache