Re: [DRBD-user] BUG: Uncatchable DRBD out-of-sync issue

2014-03-18 Thread Stanislav German-Evtushenko
Hello Lars, Usually I need to wait for a week to get out of sync so investigation is going slow. Could you suggest a reliable way to simulate "Digest mismatch, *buffer modified* by upper layers during write"? That would help a lot. Best regards, Stanislav _

Re: [DRBD-user] BUG: Uncatchable DRBD out-of-sync issue

2014-03-10 Thread Stanislav German-Evtushenko
Hello Lars, > Upper layer submits write to DRBD. > DRBD calculates checksum over data buffer. > DRBD sends that checksum. > DRBD submits data buffer to "local" backend block device. > Meanwhile, upper layer changes data buffer. > DRBD sends data buffer to peer. > DRBD receives lo

Re: [DRBD-user] BUG: Uncatchable DRBD out-of-sync issue

2014-02-24 Thread Stanislav German-Evtushenko
>> Can LVM cause these OOS? > Very unlikely. I have another idea here. I'll try to switch drive options for KVM from cache=none to cache=directsync or cache=writethrough. In that case KVM has to ensure that data is on disk. I suppose this means (with disk-flushes and disk-barrier disabled) that KV

Re: [DRBD-user] BUG: Uncatchable DRBD out-of-sync issue

2014-02-24 Thread lars . ellenberg
On Mon, Feb 24, 2014 at 01:28:58PM +0400, Stanislav German-Evtushenko wrote: > > > Most of the time (99%) I see ERR for the swap space of virtual machines. > > > > If you enable "integrity-alg", do you still see those "buffer modified > > by upper layers during write"? > > > > Well, then that is yo

Re: [DRBD-user] BUG: Uncatchable DRBD out-of-sync issue

2014-02-24 Thread Stanislav German-Evtushenko
> > Most of the time (99%) I see ERR for the swap space of virtual machines. > > If you enable "integrity-alg", do you still see those "buffer modified > by upper layers during write"? > > Well, then that is your problem, > and that problem can *NOT* be fixed with DRBD "config tuning". > > What doe

Re: [DRBD-user] BUG: Uncatchable DRBD out-of-sync issue

2014-02-04 Thread Lars Ellenberg
On Thu, Jan 30, 2014 at 11:26:43AM +0400, Stanislav German-Evtushenko wrote: > Just to make things clearer. These results are not false-positive, they are > real. False-positive also happen but rarely. Since you re-opened this after about one year, allow me to paste my answer from back then as wel

Re: [DRBD-user] BUG: Uncatchable DRBD out-of-sync issue

2014-01-29 Thread Stanislav German-Evtushenko
Just to make things clearer. These results are not false-positive, they are real. False-positive also happen but rarely. I do check for false-positive using the following script: -- #!/bin/bash # Usage: cat /var/log/kern.log | drbd_out_of_syn

Re: [DRBD-user] BUG: Uncatchable DRBD out-of-sync issue

2014-01-27 Thread Stanislav German-Evtushenko
> > > Have you figured out on which one of the servers the data is correct? And > is > it always the same server? This assumes a primary/secondary setup. > If you know on which server the data is correct then you know - IF it's a > hardware problem - which server is at fault. If it's a software pro

Re: [DRBD-user] BUG: Uncatchable DRBD out-of-sync issue

2014-01-27 Thread Pascal BERTON
-user-boun...@lists.linbit.com [mailto:drbd-user-boun...@lists.linbit.com] De la part de Stanislav German-Evtushenko Envoyé : lundi 27 janvier 2014 13:51 À : Bram Matthys Cc : drbd-user Objet : Re: [DRBD-user] BUG: Uncatchable DRBD out-of-sync issue On Mon, Jan 27, 2014 at 4:18 PM, Bram Matthys

Re: [DRBD-user] BUG: Uncatchable DRBD out-of-sync issue

2014-01-27 Thread Stanislav German-Evtushenko
51 > *À :* Bram Matthys > *Cc :* drbd-user > *Objet :* Re: [DRBD-user] BUG: Uncatchable DRBD out-of-sync issue > > > > > I don't use iSCSI, I use LVM on top of DRBD. Stanislav ___ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user

Re: [DRBD-user] BUG: Uncatchable DRBD out-of-sync issue

2014-01-27 Thread Stanislav German-Evtushenko
On Mon, Jan 27, 2014 at 4:18 PM, Bram Matthys wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > Hi, > > Just jumping in, unaware of the history of this thread... > > Stanislav German-Evtushenko wrote, on 27-1-2014 7:08: > > > > On Thu, Apr 18, 2013 at 4:21 PM, Stanislav German-Evtush

Re: [DRBD-user] BUG: Uncatchable DRBD out-of-sync issue

2014-01-27 Thread Bram Matthys
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Hi, Just jumping in, unaware of the history of this thread... Stanislav German-Evtushenko wrote, on 27-1-2014 7:08: > > On Thu, Apr 18, 2013 at 4:21 PM, Stanislav German-Evtushenko > mailto:ginerm...@gmail.com>> wrote: > > No choice so far :)

Re: [DRBD-user] BUG: Uncatchable DRBD out-of-sync issue

2014-01-26 Thread Stanislav German-Evtushenko
On Thu, Apr 18, 2013 at 4:21 PM, Stanislav German-Evtushenko < ginerm...@gmail.com> wrote: > No choice so far :) > http://pve.proxmox.com/wiki/Roadmap#Proxmox_VE_2.3 > > I don't think this is a kernel bug. Anyway would be nice if sombody > can investigate and fix or at least find work around. IDE

Re: [DRBD-user] BUG: Uncatchable DRBD out-of-sync issue

2013-04-18 Thread Stanislav German-Evtushenko
No choice so far :) http://pve.proxmox.com/wiki/Roadmap#Proxmox_VE_2.3 I don't think this is a kernel bug. Anyway would be nice if sombody can investigate and fix or at least find work around. IDE is slow in compare to VIRTIO. On Thu, Apr 18, 2013 at 2:31 PM, Felix Frank wrote: > On 04/18/2013 1

Re: [DRBD-user] BUG: Uncatchable DRBD out-of-sync issue

2013-04-18 Thread Felix Frank
On 04/18/2013 12:20 PM, Stanislav German-Evtushenko wrote: >> Note that your kernel (and hence kvm/virtio) can be considered rather old by >> now. > This is a stable RHEL 6 kernel at the moment. Exactly ;-) Same for Debian 6, which I no longer consider fit for KVM setups (without backports and s

Re: [DRBD-user] BUG: Uncatchable DRBD out-of-sync issue

2013-04-18 Thread Stanislav German-Evtushenko
> Note that your kernel (and hence kvm/virtio) can be considered rather old by > now. This is a stable RHEL 6 kernel at the moment. On Thu, Apr 18, 2013 at 1:16 PM, Felix Frank wrote: > On 04/18/2013 08:26 AM, Stanislav German-Evtushenko wrote: >> If I change VIRTIO to IDE then no issues. > > F

Re: [DRBD-user] BUG: Uncatchable DRBD out-of-sync issue

2013-04-18 Thread Felix Frank
On 04/18/2013 08:26 AM, Stanislav German-Evtushenko wrote: > If I change VIRTIO to IDE then no issues. Fascinating. Thanks for sharing! Note that your kernel (and hence kvm/virtio) can be considered rather old by now. You may see better mileage with the more recent longterm kernels such as 3.4 or

[DRBD-user] BUG: Uncatchable DRBD out-of-sync issue

2013-04-18 Thread Stanislav German-Evtushenko
Beginning is here: http://www.gossamer-threads.com/lists/drbd/users/25146 Hello everybody, Finally I think I can reproduce the issue. When it happens: Linux kernel: 2.6.32-19-pve (based on vzkernel-2.6.32-042stab075.2.src.rpm) DRBD Version: 8.3.13 DRBD Mode: dual primary + LVM on top of DRBD Vir