Re: FYI: Kernel crash info

2016-05-11 Thread g6094199
Hi Henk! Thanks for clarification! Its indeed an Seagate Archive 8TB drive. So it is vital info to let the drive settle down a bit when using it at least in an HotPlug or USB szenario sash Am 11.05.2016 um 03:02 schrieb Henk Slager: > On Tue, May 10, 2016 at 9:35 PM, wrote: >> He guys!

Re: [PATCH] Trivial fix for typos in comments.

2016-05-11 Thread David Sterba
On Tue, May 10, 2016 at 07:42:50PM -0400, Nicholas D Steeves wrote: > On 10 May 2016 at 09:33, David Sterba wrote: > > On Mon, May 09, 2016 at 08:13:29PM -0400, Nicholas D Steeves wrote: > >> Trivial fix for typos in comments; I hope this patch isn't a nuisance! > > > > No, but I don't see the typ

Device replace issues and disabling it until they are solved

2016-05-11 Thread Filipe Manana
Hi, I've noticed some time ago that our device replace implementation is unreliable. Basically under several situations it ends up not copying extents (both data and metadata) from the old device to the new device (I briefly talked about some of the problems with Josef at LSF). Essentially it ope

Re: About in-band dedupe for v4.7

2016-05-11 Thread Qu Wenruo
Solve the hard problems first (dedupe with a disk backend, make in-memory perform), then come asking for inclusion of your feature. Again, this I would say about the patches regardless of whose name is on them. --Mark David has already agreed on the idea to merge in-memory backend f

Re: Device replace issues and disabling it until they are solved

2016-05-11 Thread Qu Wenruo
Filipe Manana wrote on 2016/05/11 10:09 +0100: Hi, I've noticed some time ago that our device replace implementation is unreliable. Basically under several situations it ends up not copying extents (both data and metadata) from the old device to the new device (I briefly talked about some of t

Re: Device replace issues and disabling it until they are solved

2016-05-11 Thread Filipe Manana
On Wed, May 11, 2016 at 10:29 AM, Qu Wenruo wrote: > > > Filipe Manana wrote on 2016/05/11 10:09 +0100: >> >> Hi, >> >> I've noticed some time ago that our device replace implementation is >> unreliable. Basically under several situations it ends up not copying >> extents (both data and metadata)

Re: [PATCH] btrfs: switch to common message helpers in open_ctree, adjust messages

2016-05-11 Thread David Sterba
On Tue, May 10, 2016 at 11:00:01PM +0800, Anand Jain wrote: > >>If there is something that works simple, better. I am fine. > >> > >>Generally servers may have more than one fs mounted. So filter > >>by fsid comes handy. Without worrying about when it was labeled, > >>and troublesho

Re: [PATCH] btrfs: switch to common message helpers in open_ctree, adjust messages

2016-05-11 Thread Anand Jain
On 05/11/2016 06:56 PM, David Sterba wrote: On Tue, May 10, 2016 at 11:00:01PM +0800, Anand Jain wrote: If there is something that works simple, better. I am fine. Generally servers may have more than one fs mounted. So filter by fsid comes handy. Without worrying about when it w

Re: [PATCH] btrfs-progs: fsck: Fix found bytes accounting error

2016-05-11 Thread David Sterba
On Tue, Apr 26, 2016 at 10:49:49AM +0800, Qu Wenruo wrote: > In the new add_extent_rec_nolookup() function, we add bytes_used to > update found bytes accounting. > > However there is a typo that we used tmpl->nr, which should be rec->nr. > This will make us to add 1 for data backref, instead the c

Re: [PATCH 2/3] Btrfs-progs: add three more valid checks for superblock

2016-05-11 Thread David Sterba
On Mon, May 02, 2016 at 11:18:54AM -0700, Liu Bo wrote: > This adds valid checks for super_total_bytes, super_bytes_used and > super_stripesize. > > Since these checks are made after superblock finishes checksum > checking, this also adds a notice of "superblock checksum match but..". > > This a

Btrfs progs release 4.5.3

2016-05-11 Thread David Sterba
Hi, btrfs-progs 4.5.2 have been released. A bugfix release. Changes: * ioctl: fix unaligned access in buffer from TREE_SEARCH; might cause SIGBUS on architectures that do not support unaligned access and do not performa any fixups * improved validation checks of superblock and chunk-r

Re: Device replace issues and disabling it until they are solved

2016-05-11 Thread Filipe Manana
On Wed, May 11, 2016 at 10:09 AM, Filipe Manana wrote: > Hi, > > I've noticed some time ago that our device replace implementation is > unreliable. Basically under several situations it ends up not copying > extents (both data and metadata) from the old device to the new device > (I briefly talked

Re: [PATCH 1/2] Btrfs: fix race between fsync and direct IO writes for prealloc extents

2016-05-11 Thread Josef Bacik
On 05/09/2016 07:01 AM, fdman...@kernel.org wrote: From: Filipe Manana When we do a direct IO write against a preallocated extent (fallocate) that does not go beyond the i_size of the inode, we do the write operation without holding the inode's i_mutex (an optimization that landed in commit 388

Re: [PATCH 2/2] Btrfs: fix race between block group relocation and nocow writes

2016-05-11 Thread Josef Bacik
On 05/09/2016 07:02 AM, fdman...@kernel.org wrote: From: Filipe Manana Relocation of a block group waits for all existing tasks flushing dellaloc, starting direct IO writes and any ordered extents before starting the relocation process. However for direct IO writes that end up doing nocow (inod

Re: [PATCH 1/2] Btrfs: fix race between fsync and direct IO writes for prealloc extents

2016-05-11 Thread Filipe Manana
On Wed, May 11, 2016 at 4:41 PM, Josef Bacik wrote: > On 05/09/2016 07:01 AM, fdman...@kernel.org wrote: >> >> From: Filipe Manana >> >> When we do a direct IO write against a preallocated extent (fallocate) >> that does not go beyond the i_size of the inode, we do the write operation >> without

Re: [PATCH 1/2] Btrfs: fix race between fsync and direct IO writes for prealloc extents

2016-05-11 Thread Josef Bacik
On 05/11/2016 08:56 AM, Filipe Manana wrote: On Wed, May 11, 2016 at 4:41 PM, Josef Bacik wrote: On 05/09/2016 07:01 AM, fdman...@kernel.org wrote: From: Filipe Manana When we do a direct IO write against a preallocated extent (fallocate) that does not go beyond the i_size of the inode, we

Re: [PATCH 1/2] Btrfs: fix race between fsync and direct IO writes for prealloc extents

2016-05-11 Thread Chris Mason
On Wed, May 11, 2016 at 04:56:56PM +0100, Filipe Manana wrote: > On Wed, May 11, 2016 at 4:41 PM, Josef Bacik wrote: > > On 05/09/2016 07:01 AM, fdman...@kernel.org wrote: > >> > >> From: Filipe Manana > >> > >> When we do a direct IO write against a preallocated extent (fallocate) > >> that does

Re: [PATCH 1/2] Btrfs: fix race between fsync and direct IO writes for prealloc extents

2016-05-11 Thread Josef Bacik
On 05/11/2016 09:05 AM, Chris Mason wrote: On Wed, May 11, 2016 at 04:56:56PM +0100, Filipe Manana wrote: On Wed, May 11, 2016 at 4:41 PM, Josef Bacik wrote: On 05/09/2016 07:01 AM, fdman...@kernel.org wrote: From: Filipe Manana When we do a direct IO write against a preallocated extent (f

Re: [PATCH 1/2] Btrfs: fix race between fsync and direct IO writes for prealloc extents

2016-05-11 Thread Chris Mason
On Wed, May 11, 2016 at 09:10:12AM -0700, Josef Bacik wrote: > On 05/11/2016 09:05 AM, Chris Mason wrote: > >On Wed, May 11, 2016 at 04:56:56PM +0100, Filipe Manana wrote: > >>On Wed, May 11, 2016 at 4:41 PM, Josef Bacik wrote: > >>>On 05/09/2016 07:01 AM, fdman...@kernel.org wrote: > > F

Re: [PATCH 1/2] Btrfs: fix race between fsync and direct IO writes for prealloc extents

2016-05-11 Thread Josef Bacik
On 05/11/2016 09:12 AM, Chris Mason wrote: On Wed, May 11, 2016 at 09:10:12AM -0700, Josef Bacik wrote: On 05/11/2016 09:05 AM, Chris Mason wrote: On Wed, May 11, 2016 at 04:56:56PM +0100, Filipe Manana wrote: On Wed, May 11, 2016 at 4:41 PM, Josef Bacik wrote: On 05/09/2016 07:01 AM, fdman.

Re: About in-band dedupe for v4.7

2016-05-11 Thread David Sterba
On Tue, May 10, 2016 at 03:19:52PM +0800, Qu Wenruo wrote: > As merge window for v4.7 is coming, it would be good to hear your ideas > about the inband dedupe. For me it's still in the process of review. > We are addressing the ENOSPC problem which Josef pointed out, and we > believe the final

Re: About in-band dedupe for v4.7

2016-05-11 Thread David Sterba
On Tue, May 10, 2016 at 03:11:19PM -0700, Mark Fasheh wrote: > On Tue, May 10, 2016 at 03:19:52PM +0800, Qu Wenruo wrote: > > Hi, Chris, Josef and David, > > > > As merge window for v4.7 is coming, it would be good to hear your > > ideas about the inband dedupe. > > > > We are addressing the ENOS

Re: [PATCH v4] btrfs: qgroup: Fix qgroup accounting when creating snapshot

2016-05-11 Thread Mark Fasheh
Hi Josef, On Fri, Apr 22, 2016 at 02:12:11PM -0400, Josef Bacik wrote: > On 04/15/2016 05:08 AM, Qu Wenruo wrote: > >Current btrfs qgroup design implies a requirement that after calling > >btrfs_qgroup_account_extents() there must be a commit root switch. > > > >Normally this is OK, as btrfs_qgro

Re: [PATCH v4] btrfs: qgroup: Fix qgroup accounting when creating snapshot

2016-05-11 Thread Josef Bacik
On 05/11/2016 09:57 AM, Mark Fasheh wrote: Hi Josef, On Fri, Apr 22, 2016 at 02:12:11PM -0400, Josef Bacik wrote: On 04/15/2016 05:08 AM, Qu Wenruo wrote: Current btrfs qgroup design implies a requirement that after calling btrfs_qgroup_account_extents() there must be a commit root switch. No

Re: About in-band dedupe for v4.7

2016-05-11 Thread David Sterba
On Tue, May 10, 2016 at 07:52:11PM -0700, Mark Fasheh wrote: > Taking your history with qgroups out of this btw, my opinion does not > change. > > With respect to in-memory only dedupe, it is my honest opinion that such a > limited feature is not worth the extra maintenance work. In particular > t

Re: WARNING at fs/btrfs/inode.c:9261 btrfs_destroy_inode()

2016-05-11 Thread Adam Borowski
Chris Mason wrote: > On Tue, May 10, 2016 at 06:19:20PM -0500, Eric Biggers wrote: > > The following warning has been triggering for me since about v4.6-rc3: > > > > WARN_ON(BTRFS_I(inode)->csum_bytes); > > > > On one machine the warning has occurred 657 times since v4.6-rc5. On > > anoth

[PATCH for 4.6] btrfs: disable a spurious WARN_ON in btrfs_destroy_inode.

2016-05-11 Thread Adam Borowski
This happens a lot on real-world loads. The issue is apparently benign, as unsaved pending checksums are moot when the ship^Winode is going down anyway. Thus, no need to cause panic in users. I've retained the warning in CONFIG_BTRFS_ASSERT builds, as this shouldn't happen. I've replaced the no

BTRFS Data at Rest File Corruption

2016-05-11 Thread Richard Lochner
Hello, I have encountered a data corruption error with BTRFS which may or may not be of interest to your developers. The problem is that an unmodified file on a RAID-1 volume that had been scrubbed successfully is now corrupt. The details follow. The volume was formatted as btrfs with raid1 dat

Re: BTRFS Data at Rest File Corruption

2016-05-11 Thread Roman Mamedov
On Wed, 11 May 2016 13:36:23 -0500 Richard Lochner wrote: > Recently, a scrub returned an unrecoverable error on that file. > Again, the file has not been modified since it was originally copied > and has the time stamp from December. Furthermore, SMART tests (long) > for both drives do not indi

Re: BTRFS Data at Rest File Corruption

2016-05-11 Thread Austin S. Hemmelgarn
On 2016-05-11 14:36, Richard Lochner wrote: Hello, I have encountered a data corruption error with BTRFS which may or may not be of interest to your developers. The problem is that an unmodified file on a RAID-1 volume that had been scrubbed successfully is now corrupt. The details follow. Th

Amount of scrubbed data goes from 15.90GiB to 26.66GiB after defragment -r -v -clzo on a fs always mounted with compress=lzo

2016-05-11 Thread Niccolò Belli
Hi, Before doing the daily backup I did a btrfs check and btrfs scrub as usual. After that this time I also decided to run btrfs filesystem defragment -r -v -clzo on all subvolumes (from a live distro) and just to be sure I runned check and scrub once again. Before defragment: total bytes scr

Re: [PATCH v4] btrfs: qgroup: Fix qgroup accounting when creating snapshot

2016-05-11 Thread Mark Fasheh
On Wed, May 11, 2016 at 09:59:52AM -0700, Josef Bacik wrote: > On 05/11/2016 09:57 AM, Mark Fasheh wrote: > >Hi Josef, > > > >On Fri, Apr 22, 2016 at 02:12:11PM -0400, Josef Bacik wrote: > >>On 04/15/2016 05:08 AM, Qu Wenruo wrote: > >>>Current btrfs qgroup design implies a requirement that after c

Re: Amount of scrubbed data goes from 15.90GiB to 26.66GiB after defragment -r -v -clzo on a fs always mounted with compress=lzo

2016-05-11 Thread Christoph Anton Mitterer
On Wed, 2016-05-11 at 21:50 +0200, Niccolò Belli wrote: > What did happen? Perhaps because defrag unfortunately breaks up any reflinks? Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature

Re: [PATCH for 4.6] btrfs: disable a spurious WARN_ON in btrfs_destroy_inode.

2016-05-11 Thread Josef Bacik
On 05/11/2016 11:29 AM, Adam Borowski wrote: This happens a lot on real-world loads. The issue is apparently benign, as unsaved pending checksums are moot when the ship^Winode is going down anyway. Thus, no need to cause panic in users. I've retained the warning in CONFIG_BTRFS_ASSERT builds,

Re: [PATCH v4] btrfs: qgroup: Fix qgroup accounting when creating snapshot

2016-05-11 Thread Josef Bacik
On 05/11/2016 12:53 PM, Mark Fasheh wrote: On Wed, May 11, 2016 at 09:59:52AM -0700, Josef Bacik wrote: On 05/11/2016 09:57 AM, Mark Fasheh wrote: Hi Josef, On Fri, Apr 22, 2016 at 02:12:11PM -0400, Josef Bacik wrote: On 04/15/2016 05:08 AM, Qu Wenruo wrote: Current btrfs qgroup design impli

fsck: to repair or not to repair

2016-05-11 Thread Nikolaus Rath
Hello, I recently ran btrfsck on one of my file systems, and got the following messages: checking extents checking free space cache checking fs roots root 5 inode 3149867 errors 400, nbytes wrong root 5 inode 3150237 errors 400, nbytes wrong root 5 inode 3150238 errors 400, nbytes wrong root 5 in

Re: [PATCH 2/3] Btrfs-progs: add three more valid checks for superblock

2016-05-11 Thread Liu Bo
On Wed, May 11, 2016 at 03:36:29PM +0200, David Sterba wrote: > On Mon, May 02, 2016 at 11:18:54AM -0700, Liu Bo wrote: > > This adds valid checks for super_total_bytes, super_bytes_used and > > super_stripesize. > > > > Since these checks are made after superblock finishes checksum > > checking,

Re: [PATCH v4] btrfs: qgroup: Fix qgroup accounting when creating snapshot

2016-05-11 Thread Qu Wenruo
On 05/12/2016 04:30 AM, Josef Bacik wrote: On 05/11/2016 12:53 PM, Mark Fasheh wrote: On Wed, May 11, 2016 at 09:59:52AM -0700, Josef Bacik wrote: On 05/11/2016 09:57 AM, Mark Fasheh wrote: Hi Josef, On Fri, Apr 22, 2016 at 02:12:11PM -0400, Josef Bacik wrote: On 04/15/2016 05:08 AM, Qu We

[PATCH 0/1] btrfs-progs: Typo review of strings and comments

2016-05-11 Thread Nicholas D Steeves
Thank you David Sterba for the btrfs-typos.txt which gave me a head start. Unfortunately I wasn't able finish before btrfs-progs-4.5.3 was released, because I decided to use emacs' ispell-comments-and-strings to do a full review. I had to rebase to kdave's 4.5.2 branch on github, and that is w

[PATCH 1/1] Typo review of strings and comments.

2016-05-11 Thread Nicholas D Steeves
Signed-off-by: Nicholas D Steeves --- CHANGES| 12 - backref.c | 4 +-- btrfs-convert.c| 4 +-- btrfs-corrupt-block.c | 4 +-- btrfs-deb

Re: [PATCH] fstests: btrfs: Test fiemap ioctl on completely deduped file

2016-05-11 Thread Darrick J. Wong
On Wed, May 11, 2016 at 10:14:42AM +0800, Qu Wenruo wrote: > > > Filipe Manana wrote on 2016/05/10 11:01 +0100: > >On Tue, May 10, 2016 at 9:39 AM, Qu Wenruo wrote: > >>For a completely deduped file, which means all its file extent are > >>pointing to one bytenr, if calling fiemap on it, btrfs w

Re: [PATCH] fstests: btrfs: Test fiemap ioctl on completely deduped file

2016-05-11 Thread Qu Wenruo
Darrick J. Wong wrote on 2016/05/11 17:23 -0700: On Wed, May 11, 2016 at 10:14:42AM +0800, Qu Wenruo wrote: Filipe Manana wrote on 2016/05/10 11:01 +0100: On Tue, May 10, 2016 at 9:39 AM, Qu Wenruo wrote: For a completely deduped file, which means all its file extent are pointing to one b

Re: Idea on compatibility for old distributions

2016-05-11 Thread Qu Wenruo
Thanks for the info. I also read the phoronix news yesterday. So for RHEL6 that's meaningless. But I'm not sure whether it's still meaningless for OpenSUSE, maybe David has some plan on it? Thanks, Qu Eric Sandeen wrote on 2016/05/10 15:23 -0500: On 5/9/16 8:16 PM, Qu Wenruo wrote: Hi Davi

Re: [PATCH] fstests: btrfs: Test fiemap ioctl on completely deduped file

2016-05-11 Thread Dave Chinner
On Thu, May 12, 2016 at 08:46:41AM +0800, Qu Wenruo wrote: > >Filesystem type is: 58465342 > >File size of a is 262144 (64 blocks of 4096 bytes) > > ext: logical_offset:physical_offset: length: expected: flags: > > 0:0.. 31: 24..55: 32: s

Re: [PATCH] fstests: btrfs: Test fiemap ioctl on completely deduped file

2016-05-11 Thread Qu Wenruo
Dave Chinner wrote on 2016/05/12 11:19 +1000: On Thu, May 12, 2016 at 08:46:41AM +0800, Qu Wenruo wrote: Filesystem type is: 58465342 File size of a is 262144 (64 blocks of 4096 bytes) ext: logical_offset:physical_offset: length: expected: flags: 0:0.. 31:

[PATCH] fstests: generic: Test reserved extent map search routine on deduped file

2016-05-11 Thread Qu Wenruo
For fully dedupe file, which means all its file exntents are pointing to the same bytenr, btrfs can cause soft lockup when calling fiemap ioctl on that file, like the following output: -- CPU: 1 PID: 7500 Comm: xfs_io Not tainted 4.5.0-rc6+ #2 Hardware name: XXX task: ff

Re: [PATCH] fstests: generic: Test reserved extent map search routine on deduped file

2016-05-11 Thread Darrick J. Wong
On Thu, May 12, 2016 at 12:51:35PM +0800, Qu Wenruo wrote: > For fully dedupe file, which means all its file exntents are pointing to > the same bytenr, btrfs can cause soft lockup when calling fiemap ioctl > on that file, like the following output: > -- > CPU: 1 PID: 7500 Comm: xfs_io Not tain

Re: [PATCH] fstests: generic: Test reserved extent map search routine on deduped file

2016-05-11 Thread Darrick J. Wong
On Wed, May 11, 2016 at 10:01:18PM -0700, Darrick J. Wong wrote: > On Thu, May 12, 2016 at 12:51:35PM +0800, Qu Wenruo wrote: > > For fully dedupe file, which means all its file exntents are pointing to > > the same bytenr, btrfs can cause soft lockup when calling fiemap ioctl > > on that file, lik

Re: [PATCH] fstests: generic: Test reserved extent map search routine on deduped file

2016-05-11 Thread Qu Wenruo
Darrick J. Wong wrote on 2016/05/11 22:01 -0700: On Thu, May 12, 2016 at 12:51:35PM +0800, Qu Wenruo wrote: For fully dedupe file, which means all its file exntents are pointing to the same bytenr, btrfs can cause soft lockup when calling fiemap ioctl on that file, like the following output: -

Re: [PATCH] fstests: generic: Test reserved extent map search routine on deduped file

2016-05-11 Thread Darrick J. Wong
On Thu, May 12, 2016 at 01:44:01PM +0800, Qu Wenruo wrote: > > > Darrick J. Wong wrote on 2016/05/11 22:01 -0700: > >On Thu, May 12, 2016 at 12:51:35PM +0800, Qu Wenruo wrote: > >>For fully dedupe file, which means all its file exntents are pointing to > >>the same bytenr, btrfs can cause soft lo

Re: 4.4.0 - no space left with >1.7 TB free space left

2016-05-11 Thread Tomasz Chmielewski
On 2016-04-08 20:53, Roman Mamedov wrote: > Do you snapshot the parent subvolume which holds the databases? Can you > correlate that perhaps ENOSPC occurs at the time of snapshotting? If > yes, then > you should try the patch https://patchwork.kernel.org/patch/7967161/ > > (Too bad this was not

Re: 4.4.0 - no space left with >1.7 TB free space left

2016-05-11 Thread Tomasz Chmielewski
On 2016-05-12 15:03, Tomasz Chmielewski wrote: FYI, I'm still getting this with 4.5.3, which probably means the fix was not yet included ("No space left" at snapshot time): /var/log/postgresql/postgresql-9.3-main.log:2016-05-11 06:06:10 UTC LOG: could not close temporary statistics file "pg_st

Re: BTRFS Data at Rest File Corruption

2016-05-11 Thread Chris Murphy
What are the mount options for this filesystem? Maybe filter with grep/egrep the journalctl output or /var/log/messages for -i btrfs, and also for libata/scsi messages. Anything over previous days might reveal some clue. I've got multiple Btrfs raid1's and several times I've had *correctable* err