Re: [DRBD-user] 8 Zettabytes out-of-sync?

2018-11-02 Thread Roberto Resoli
Il 2 novembre 2018 11:16:54 CET, Yannis Milios ha scritto: >On Fri, 2 Nov 2018 at 09:25, Jarno Elonen wrote: > >> >> This is getting quite worrisome. Is anyone else experiencing this >with >> DRBD 9? Is it something really wrong in my setup, or are there >perhaps some >> known instabilities in

Re: [DRBD-user] 8 Zettabytes out-of-sync?

2018-11-02 Thread Eddie Chapman
On 02/11/18 08:45, Jarno Elonen wrote: More clues: Just witnessed a resync (after invalidate) to steadily go from 100% out-of-sync to 0% (after several automatic disconnects and reconnects). Immediately after reaching 0%, it went to negative -% ! After that, drbdtop started showing 8.0ZiB

Re: [DRBD-user] 8 Zettabytes out-of-sync?

2018-11-02 Thread Yannis Milios
On Fri, 2 Nov 2018 at 09:25, Jarno Elonen wrote: > > This is getting quite worrisome. Is anyone else experiencing this with > DRBD 9? Is it something really wrong in my setup, or are there perhaps some > known instabilities in DRBD 9.0.15-1? > Yes, I have been facing this as well on all

[DRBD-user] 8 Zettabytes out-of-sync?

2018-11-02 Thread Jarno Elonen
I've got several DRBD 9 resource that constantly show *UpToDate* with 9223372036854774304 bytes (exactly 8ZiB) of OutOfDate data. Any idea what might cause this and how to fix it? Example: # drbdsetup status --verbose --statistics vm-106-disk-1 vm-106-disk-1 node-id:0 role:Primary suspended:no

Re: [DRBD-user] 8 Zettabytes out-of-sync?

2018-11-02 Thread Jarno Elonen
Here's some more info. Dmesg shows some suspicious looking log message, such as: 1) FIXME drbd_s_vm-117-s[2830] op clear, bitmap locked for 'receive bitmap' by drbd_r_vm-117-s[5038] 2) Wrong magic value 0x0007 in protocol version 114 3) peer request with dagtag 399201392 not found

Re: [DRBD-user] 8 Zettabytes out-of-sync?

2018-11-02 Thread Jarno Elonen
More clues: Just witnessed a resync (after invalidate) to steadily go from 100% out-of-sync to 0% (after several automatic disconnects and reconnects). Immediately after reaching 0%, it went to negative -% ! After that, drbdtop started showing 8.0ZiB out-of-sync. Looks like a severe wrap-around

Re: [DRBD-user] 8 Zettabytes out-of-sync?

2018-11-02 Thread Jarno Elonen
Okay, today one of these resources got a sudden, severe filesystem corruption on the primary. On the other hand, the secondaries (that showed 8ZiB out-of-sync) were still mountable after I disconnected the corrupted primary. No idea how current data the secondaries had, but drbdtop still showed