Re: [DRBD-user] (DRBD 9) promote secondary to primary with primary crashed

2018-11-02 Thread Yannis Milios
Try adding --force parameter in drbdadm command. On Friday, November 2, 2018, Daniel Hertanu wrote: > Hi, > > I'm running two nodes with DRBD 9 and I want to simulate a primary node > crash followed by restoring the access to the data on the secondary node > left. > > So, having the sync done

[DRBD-user] linstor-proxmox controller toggle tests

2018-11-02 Thread opengreb
Hello, I did some tests Proxmox / Linstor with 2 nodes; I have problems in my controller toggle tests. In preparation, I stop the controller service on my first node (vpx3-1) and copy the contents of the database folder to the second node (vpx3-2). In my test, I abruptly shutdown the controll

[DRBD-user] (DRBD 9) promote secondary to primary with primary crashed

2018-11-02 Thread Daniel Hertanu
Hi, I'm running two nodes with DRBD 9 and I want to simulate a primary node crash followed by restoring the access to the data on the secondary node left. So, having the sync done between the hosts I'm disconnecting the network to the primary to simulate the crash. All my attempts to promote the

Re: [DRBD-user] 8 Zettabytes out-of-sync?

2018-11-02 Thread Roberto Resoli
Il 2 novembre 2018 11:16:54 CET, Yannis Milios ha scritto: >On Fri, 2 Nov 2018 at 09:25, Jarno Elonen wrote: > >> >> This is getting quite worrisome. Is anyone else experiencing this >with >> DRBD 9? Is it something really wrong in my setup, or are there >perhaps some >> known instabilities in D

Re: [DRBD-user] 8 Zettabytes out-of-sync?

2018-11-02 Thread Eddie Chapman
On 02/11/18 08:45, Jarno Elonen wrote: More clues: Just witnessed a resync (after invalidate) to steadily go from 100% out-of-sync to 0% (after several automatic disconnects and reconnects). Immediately after reaching 0%, it went to negative -% ! After that, drbdtop started showing 8.0ZiB out

Re: [DRBD-user] 8 Zettabytes out-of-sync?

2018-11-02 Thread Yannis Milios
On Fri, 2 Nov 2018 at 09:25, Jarno Elonen wrote: > > This is getting quite worrisome. Is anyone else experiencing this with > DRBD 9? Is it something really wrong in my setup, or are there perhaps some > known instabilities in DRBD 9.0.15-1? > Yes, I have been facing this as well on all "recent"

[DRBD-user] 8 Zettabytes out-of-sync?

2018-11-02 Thread Jarno Elonen
I've got several DRBD 9 resource that constantly show *UpToDate* with 9223372036854774304 bytes (exactly 8ZiB) of OutOfDate data. Any idea what might cause this and how to fix it? Example: # drbdsetup status --verbose --statistics vm-106-disk-1 vm-106-disk-1 node-id:0 role:Primary suspended:no

Re: [DRBD-user] 8 Zettabytes out-of-sync?

2018-11-02 Thread Jarno Elonen
Here's some more info. Dmesg shows some suspicious looking log message, such as: 1) FIXME drbd_s_vm-117-s[2830] op clear, bitmap locked for 'receive bitmap' by drbd_r_vm-117-s[5038] 2) Wrong magic value 0x0007 in protocol version 114 3) peer request with dagtag 399201392 not found got_peer_a

Re: [DRBD-user] 8 Zettabytes out-of-sync?

2018-11-02 Thread Jarno Elonen
More clues: Just witnessed a resync (after invalidate) to steadily go from 100% out-of-sync to 0% (after several automatic disconnects and reconnects). Immediately after reaching 0%, it went to negative -% ! After that, drbdtop started showing 8.0ZiB out-of-sync. Looks like a severe wrap-around b

Re: [DRBD-user] 8 Zettabytes out-of-sync?

2018-11-02 Thread Jarno Elonen
Okay, today one of these resources got a sudden, severe filesystem corruption on the primary. On the other hand, the secondaries (that showed 8ZiB out-of-sync) were still mountable after I disconnected the corrupted primary. No idea how current data the secondaries had, but drbdtop still showed th