Re: [DRBD-user] local WRITE IO error sector 21776+1016 on dm-2
On Wed, Jul 24, 2019 at 10:29:26AM +0200, Roland JARRY wrote:
> Hello,
>
> I answer to myself (and others if it can help !).
>
> I seems to be the same issue described by Lars
> (http://lists.linbit.com/pipermail/drbd-user/2017-February/023024.html)
> : WRITE SAME not supported by my hardware (HPE Smart Array P816i-a SR
> Gen10 on HP ProLiant DL380 Gen10).
>
> I saw a lot of posts about this issue but didn't see messages about
> WRITE SAME on my log (neither by DRBD, nor by device manager LVM).
>
> Work around proposed by Lars works.
No need to use a workaround anymore, we added that as config option,
too much hardware (actually, mostly "hardware" in virtualization
environments) did pretend to support it but then failed when someone
would try to actually use the feature.
So now you can just tell DRBD to "disable-write-same".
> Maybe this one can also be used :
> https://chris.hofstaedtler.name/blog/2016/10/kernel319plus-3par-incompat.html
> finding before ATTRS{rev} property of disks.
For your specific hardware, probably yes.
--
: Lars Ellenberg
: LINBIT | Keeping the Digital World Running
: DRBD -- Heartbeat -- Corosync -- Pacemaker
DRBD® and LINBIT® are registered trademarks of LINBIT
__
please don't Cc me, but send to list -- I'm subscribed
___
Star us on GITHUB: https://github.com/LINBIT
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user
Re: [DRBD-user] local WRITE IO error sector 21776+1016 on dm-2
Hello,
I answer to myself (and others if it can help !).
I seems to be the same issue described by Lars
(http://lists.linbit.com/pipermail/drbd-user/2017-February/023024.html)
: WRITE SAME not supported by my hardware (HPE Smart Array P816i-a SR
Gen10 on HP ProLiant DL380 Gen10).
I saw a lot of posts about this issue but didn't see messages about
WRITE SAME on my log (neither by DRBD, nor by device manager LVM).
Work around proposed by Lars works.
Maybe this one can also be used :
https://chris.hofstaedtler.name/blog/2016/10/kernel319plus-3par-incompat.html
finding before ATTRS{rev} property of disks.
Roland.
On 23/07/2019 10:53, Roland JARRY wrote:
> Hello,
>
> I have an issue mounting drbd 8.4.11-1 resources on a kernel
> 4.9.0-9-amd64 (debian 9.9). I have this error message : block drbd3:
> local WRITE IO error sector 21776+1016 on dm-2
>
> Then, the resource becomes diskless.
>
> Here are the settings of the resource :
>
> root@srv-pg-sav-p:~# cat /etc/drbd.d/vgbackup-lv-back3.res
> resource vgbackup-lv-back3 {
> net {
> allow-two-primaries;
> }
> startup {
> wfc-timeout 120;
> degr-wfc-timeout 120;
> }
>
> volume 0 {
> device /dev/drbd3;
> #meta-disk internal;
> meta-disk /dev/vgbackup/lv-md-back3;
> disk /dev/vgbackup/lv-back3;
> }
>
> on srv-pg-sav-p {
> address 192.168.8.221:7803;
>
> }
> on srv-pg-sav-s {
> address 192.168.8.222:7803;
> }
> }
>
> I've changed meta-disk internal to external lv device to have more space
> (1GB), but I have the same issue :
>
> root@srv-pg-sav-p:~# lvs
> LV VG Attr LSize Pool Origin Data% Meta% Move
> Log Cpy%Sync Convert
> lv-back1 vgbackup -wi-ao
> 21.00t
> lv-back2 vgbackup -wi-ao
> 21.00t
> lv-back3 vgbackup -wi-a-
> 21.00t
> lv-md-back3 vgbackup -wi-a- 1.00g
>
> I have 3 resources of same size. 2 works right now and not the 3rd. And
> I had the same issue before with 2 first resources.
>
> I notice that the error is on the same sector on each resource and at
> each time. Is there a limitation somewhere ?
>
> Here is more log :
>
> Jul 23 10:23:52 srv-pg-sav-p kernel: [1532462.342531] EXT4-fs (drbd3):
> mounted filesystem with ordered data mode. Opts: (null)
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.860138] block drbd3: local
> WRITE IO error sector 21776+1016 on dm-2
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.860146] block drbd3: disk(
> UpToDate -> Failed )
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.860177] block drbd3: Local
> IO failed in __req_mod. Detaching...
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.868306] block drbd3:
> helper command: /sbin/drbdadm pri-on-incon-degr minor-3
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.868356] block drbd3: IO
> ERROR: neither local nor remote data, sector 21776+8
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.876611] block drbd3: IO
> ERROR: neither local nor remote data, sector 21784+8
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.881934] block drbd3:
> helper command: /sbin/drbdadm pri-on-incon-degr minor-3 exit code 0 (0x0)
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.885020] block drbd3: IO
> ERROR: neither local nor remote data, sector 21792+8
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.894204] block drbd3: 21 TB
> (5637144528 bits) marked out-of-sync by on disk bit-map.
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.894207] block drbd3: disk(
> Failed -> Diskless )
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.894220] block drbd3: IO
> ERROR: neither local nor remote data, sector 21800+8
> Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.902118] block drbd3: IO
> ERROR: neither local nor remote data, sector 21808+8
> Jul 23 10:23:59 srv-pg-sav-p kernel: [1532469.637796] block drbd3: 122
> messages suppressed in /usr/src/modules/drbd/drbd/drbd_req.c:1446.
> Jul 23 10:23:59 srv-pg-sav-p kernel: [1532469.637802] block drbd3: IO
> ERROR: neither local nor remote data, sector 22548840448+8
> Jul 23 10:23:59 srv-pg-sav-p kernel: [1532469.648265] Buffer I/O error
> on dev drbd3, logical block 2818605056, lost sync page write
> Jul 23 10:23:59 srv-pg-sav-p kernel: [1532469.658056] JBD2: Error -5
> detected when updating journal superblock for drbd3-8.
> Jul 23 10:23:59 srv-pg-sav-p kernel: [1532469.668068] Aborting journal
> on device drbd3-8.
> Jul 23 10:23:59 srv-pg-sav-p kernel: [1532469.678189] Buffer I/O error
> on dev drbd3, logical block 2818605056, lost sync page write
> Jul 23 10:23:59 srv-pg-sav-p kernel: [1532469.688556] JBD2: Error -5
> detected when updating journal superblock for drbd3-8.
> Jul 23 10:45:28 srv-pg-sav-p kernel: [1533756.213355] block drbd3: 1
> messages suppressed in /usr/src/modules/drbd/drbd/drbd_req.c:1446.
> Jul 23 10:45:28 srv-pg-sav-p kernel: [1533756.213361] block
[DRBD-user] local WRITE IO error sector 21776+1016 on dm-2
Hello,
I have an issue mounting drbd 8.4.11-1 resources on a kernel
4.9.0-9-amd64 (debian 9.9). I have this error message : block drbd3:
local WRITE IO error sector 21776+1016 on dm-2
Then, the resource becomes diskless.
Here are the settings of the resource :
root@srv-pg-sav-p:~# cat /etc/drbd.d/vgbackup-lv-back3.res
resource vgbackup-lv-back3 {
net {
allow-two-primaries;
}
startup {
wfc-timeout 120;
degr-wfc-timeout 120;
}
volume 0 {
device /dev/drbd3;
#meta-disk internal;
meta-disk /dev/vgbackup/lv-md-back3;
disk /dev/vgbackup/lv-back3;
}
on srv-pg-sav-p {
address 192.168.8.221:7803;
}
on srv-pg-sav-s {
address 192.168.8.222:7803;
}
}
I've changed meta-disk internal to external lv device to have more space
(1GB), but I have the same issue :
root@srv-pg-sav-p:~# lvs
LV VG Attr LSize Pool Origin Data% Meta% Move
Log Cpy%Sync Convert
lv-back1 vgbackup -wi-ao
21.00t
lv-back2 vgbackup -wi-ao
21.00t
lv-back3 vgbackup -wi-a-
21.00t
lv-md-back3 vgbackup -wi-a- 1.00g
I have 3 resources of same size. 2 works right now and not the 3rd. And
I had the same issue before with 2 first resources.
I notice that the error is on the same sector on each resource and at
each time. Is there a limitation somewhere ?
Here is more log :
Jul 23 10:23:52 srv-pg-sav-p kernel: [1532462.342531] EXT4-fs (drbd3):
mounted filesystem with ordered data mode. Opts: (null)
Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.860138] block drbd3: local
WRITE IO error sector 21776+1016 on dm-2
Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.860146] block drbd3: disk(
UpToDate -> Failed )
Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.860177] block drbd3: Local
IO failed in __req_mod. Detaching...
Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.868306] block drbd3:
helper command: /sbin/drbdadm pri-on-incon-degr minor-3
Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.868356] block drbd3: IO
ERROR: neither local nor remote data, sector 21776+8
Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.876611] block drbd3: IO
ERROR: neither local nor remote data, sector 21784+8
Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.881934] block drbd3:
helper command: /sbin/drbdadm pri-on-incon-degr minor-3 exit code 0 (0x0)
Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.885020] block drbd3: IO
ERROR: neither local nor remote data, sector 21792+8
Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.894204] block drbd3: 21 TB
(5637144528 bits) marked out-of-sync by on disk bit-map.
Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.894207] block drbd3: disk(
Failed -> Diskless )
Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.894220] block drbd3: IO
ERROR: neither local nor remote data, sector 21800+8
Jul 23 10:23:53 srv-pg-sav-p kernel: [1532463.902118] block drbd3: IO
ERROR: neither local nor remote data, sector 21808+8
Jul 23 10:23:59 srv-pg-sav-p kernel: [1532469.637796] block drbd3: 122
messages suppressed in /usr/src/modules/drbd/drbd/drbd_req.c:1446.
Jul 23 10:23:59 srv-pg-sav-p kernel: [1532469.637802] block drbd3: IO
ERROR: neither local nor remote data, sector 22548840448+8
Jul 23 10:23:59 srv-pg-sav-p kernel: [1532469.648265] Buffer I/O error
on dev drbd3, logical block 2818605056, lost sync page write
Jul 23 10:23:59 srv-pg-sav-p kernel: [1532469.658056] JBD2: Error -5
detected when updating journal superblock for drbd3-8.
Jul 23 10:23:59 srv-pg-sav-p kernel: [1532469.668068] Aborting journal
on device drbd3-8.
Jul 23 10:23:59 srv-pg-sav-p kernel: [1532469.678189] Buffer I/O error
on dev drbd3, logical block 2818605056, lost sync page write
Jul 23 10:23:59 srv-pg-sav-p kernel: [1532469.688556] JBD2: Error -5
detected when updating journal superblock for drbd3-8.
Jul 23 10:45:28 srv-pg-sav-p kernel: [1533756.213355] block drbd3: 1
messages suppressed in /usr/src/modules/drbd/drbd/drbd_req.c:1446.
Jul 23 10:45:28 srv-pg-sav-p kernel: [1533756.213361] block drbd3: IO
ERROR: neither local nor remote data, sector 45097156480+8
Jul 23 10:45:28 srv-pg-sav-p kernel: [1533756.222815] block drbd3: IO
ERROR: neither local nor remote data, sector 45097156592+8
Jul 23 10:45:28 srv-pg-sav-p kernel: [1533756.232250] block drbd3: IO
ERROR: neither local nor remote data, sector 0+8
Jul 23 10:45:28 srv-pg-sav-p kernel: [1533756.241269] block drbd3: IO
ERROR: neither local nor remote data, sector 8+8
Kind regards.
___
Star us on GITHUB: https://github.com/LINBIT
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user
