Hi all, With drbd-8.4.3, I noticed that performing `drbdadm attach r0' with a newly-initialized empty disk causes initial full synchronization.
In the following document, it seems it is necessary to perform `drbdadm primary --force resource' to start the initial full synchronization. http://www.drbd.org/users-guide-emb/s-initial-full-sync.html However, it starts without performing `drbdadm primary --force resource'. I'd like to know whether this is an expected behavior or not. * enviroment OS: Scientific Linux release 6.2 (Carbon) DRBD: drbd-8.4.3 * configuration # /etc/drbd.conf common { } # resource r0 on nodea: not ignored, not stacked # defined at /etc/drbd.d/r0.res:1 resource r0 { on nodea { volume 0 { device /dev/drbd0 minor 0; disk /dev/sdc; meta-disk internal; } address ipv4 192.168.0.1:7789; } on nodeb { volume 0 { device /dev/drbd0 minor 0; disk /dev/sdc; meta-disk internal; } address ipv4 192.168.0.2:7789; } net { protocol C; } disk { resync-rate 2g; c-plan-ahead 0; } } * How To reproduce (1) cleanup metadata in /dev/sdc [root@nodea ~]# MegaCli ldinit start l2 a0 Start Initialization on Virtual Drive 2 (target id: 2) Success. Exit Code: 0x00 [root@nodeb ~]# MegaCli ldinit start l2 a0 Start Initialization on Virtual Drive 2 (target id: 2) Success. Exit Code: 0x00 (2) drbd start [root@nodea ~]# service drbd start Starting DRBD resources: [ create res: r0 prepare disk: r0 adjust disk: r0:failed(apply-al:255) adjust net: r0 ] .... [root@nodeb ~]# service drbd start Starting DRBD resources: [ create res: r0 prepare disk: r0 adjust disk: r0:failed(apply-al:255) adjust net: r0 ] . [root@nodea ~]# cat /proc/drbd version: 8.4.3 (api:1/proto:86-101) GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@nodea, 2013-07-09 17:08:55 0: cs:Connected ro:Secondary/Secondary ds:Diskless/Diskless C r----- ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0 [root@nodeb ~]# cat /proc/drbd version: 8.4.3 (api:1/proto:86-101) GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@nodeb, 2013-07-09 17:10:11 0: cs:Connected ro:Secondary/Secondary ds:Diskless/Diskless C r----- ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0 => Disk states is Diskless (3) create metadata [root@nodea ~]# drbdadm create-md r0 Writing meta data... initializing activity log NOT initializing bitmap New drbd meta data block successfully created. success [root@nodeb ~]# drbdadm create-md r0 Writing meta data... initializing activity log NOT initializing bitmap New drbd meta data block successfully created. success (4) attach backing device [root@nodea ~]# cat /proc/drbd version: 8.4.3 (api:1/proto:86-101) GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@nodea, 2013-07-09 17:08:55 0: cs:Connected ro:Secondary/Secondary ds:Diskless/Diskless C r----- ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0 [root@nodea ~]# drbdadm attach r0 [root@nodea ~]# cat /proc/drbd version: 8.4.3 (api:1/proto:86-101) GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@nodea, 2013-07-09 17:08:55 0: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Diskless C r----- ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:2147417308 [root@nodeb ~]# cat /proc/drbd version: 8.4.3 (api:1/proto:86-101) GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@nodeb, 2013-07-09 17:10:11 0: cs:Connected ro:Secondary/Secondary ds:Diskless/Inconsistent C r----- ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0 [root@nodeb ~]# drbdadm attach r0 [root@nodeb ~]# cat /proc/drbd version: 8.4.3 (api:1/proto:86-101) GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@nodeb, 2013-07-09 17:10:11 0: cs:SyncSource ro:Secondary/Secondary ds:UpToDate/Inconsistent C r---n- ns:460616 nr:0 dw:0 dr:468248 al:0 bm:28 lo:6 pe:11 ua:80 ap:0 ep:1 wo:f oos:2146957764 [>....................] sync'ed: 0.1% (2096636/2097084)M finish: 1:17:51 speed: 459,544 (459,544) K/sec (5) log [nodea] Jul 10 10:22:03 nodea kernel: events: mcg drbd: 2 Jul 10 10:22:03 nodea kernel: drbd: initialized. Version: 8.4.3 (api:1/proto:86-101) Jul 10 10:22:03 nodea kernel: drbd: GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@nodea, 2013-07-09 17:08:55 Jul 10 10:22:03 nodea kernel: drbd: registered as block device major 147 Jul 10 10:22:03 nodea kernel: d-con r0: Starting worker thread (from drbdsetup [32273]) Jul 10 10:22:03 nodea kernel: d-con r0: conn( StandAlone -> Unconnected ) Jul 10 10:22:03 nodea kernel: d-con r0: Starting receiver thread (from drbd_w_r0 [32275]) Jul 10 10:22:03 nodea kernel: d-con r0: receiver (re)started Jul 10 10:22:03 nodea kernel: d-con r0: conn( Unconnected -> WFConnection ) Jul 10 10:22:05 nodea kernel: d-con r0: Handshake successful: Agreed network protocol version 101 Jul 10 10:22:05 nodea kernel: d-con r0: conn( WFConnection -> WFReportParams ) Jul 10 10:22:05 nodea kernel: d-con r0: Starting asender thread (from drbd_r_r0 [32276]) Jul 10 10:22:05 nodea kernel: block drbd0: max BIO size = 4096 Jul 10 10:22:05 nodea kernel: block drbd0: peer( Unknown -> Secondary ) conn( WFReportParams -> Connected ) pdsk( DUnknown -> Diskless ) Jul 10 10:22:43 nodea kernel: block drbd0: disk( Diskless -> Attaching ) Jul 10 10:22:43 nodea kernel: d-con r0: Method to ensure write ordering: flush Jul 10 10:22:43 nodea kernel: block drbd0: max BIO size = 286720 Jul 10 10:22:43 nodea kernel: block drbd0: drbd_bm_resize called with capacity == 4294834616 Jul 10 10:22:43 nodea kernel: block drbd0: resync bitmap: bits=536854327 words=8388349 pages=16384 Jul 10 10:22:43 nodea kernel: block drbd0: size = 2048 GB (2147417308 KB) Jul 10 10:22:43 nodea kernel: block drbd0: Writing the whole bitmap, size changed Jul 10 10:22:43 nodea kernel: block drbd0: bitmap WRITE of 16384 pages took 20 jiffies Jul 10 10:22:43 nodea kernel: block drbd0: 2048 GB (536854327 bits) marked out-of-sync by on disk bit-map. Jul 10 10:22:43 nodea kernel: block drbd0: bitmap READ of 16384 pages took 18 jiffies Jul 10 10:22:43 nodea kernel: block drbd0: recounting of set bits took additional 28 jiffies Jul 10 10:22:43 nodea kernel: block drbd0: 2048 GB (536854327 bits) marked out-of-sync by on disk bit-map. Jul 10 10:22:43 nodea kernel: block drbd0: disk( Attaching -> Negotiating ) Jul 10 10:22:43 nodea kernel: block drbd0: attached to UUIDs 0000000000000004:0000000000000000:0000000000000000:0000000000000000 Jul 10 10:22:43 nodea kernel: block drbd0: disk( Negotiating -> Inconsistent ) Jul 10 10:22:59 nodea kernel: block drbd0: real peer disk state = Inconsistent Jul 10 10:22:59 nodea kernel: block drbd0: drbd_sync_handshake: Jul 10 10:22:59 nodea kernel: block drbd0: self 0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:536854327 flags:0 Jul 10 10:22:59 nodea kernel: block drbd0: peer 0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:536854327 flags:4 Jul 10 10:22:59 nodea kernel: block drbd0: uuid_compare()=0 by rule 10 Jul 10 10:22:59 nodea kernel: block drbd0: No resync, but 536854327 bits in bitmap! Jul 10 10:22:59 nodea kernel: block drbd0: pdsk( Diskless -> Inconsistent ) Jul 10 10:22:59 nodea kernel: block drbd0: Resync of new storage after online grow Jul 10 10:22:59 nodea kernel: block drbd0: conn( Connected -> WFSyncUUID ) pdsk( Inconsistent -> UpToDate ) Jul 10 10:22:59 nodea kernel: block drbd0: updated sync uuid 087F262FA3DE1BDE:0000000000000000:0000000000000000:0000000000000000 Jul 10 10:22:59 nodea kernel: block drbd0: helper command: /sbin/drbdadm before-resync-target minor-0 Jul 10 10:22:59 nodea kernel: block drbd0: helper command: /sbin/drbdadm before-resync-target minor-0 exit code 0 (0x0) Jul 10 10:22:59 nodea kernel: block drbd0: conn( WFSyncUUID -> SyncTarget ) Jul 10 10:22:59 nodea kernel: block drbd0: Began resync as SyncTarget (will sync 2147417308 KB [536854327 bits set]). [nodeb] Jul 10 10:22:05 nodeb kernel: events: mcg drbd: 2 Jul 10 10:22:05 nodeb kernel: drbd: initialized. Version: 8.4.3 (api:1/proto:86-101) Jul 10 10:22:05 nodeb kernel: drbd: GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@nodeb, 2013-07-09 17:10:11 Jul 10 10:22:05 nodeb kernel: drbd: registered as block device major 147 Jul 10 10:22:05 nodeb kernel: d-con r0: Starting worker thread (from drbdsetup [28581]) Jul 10 10:22:05 nodeb kernel: d-con r0: conn( StandAlone -> Unconnected ) Jul 10 10:22:05 nodeb kernel: d-con r0: Starting receiver thread (from drbd_w_r0 [28583]) Jul 10 10:22:05 nodeb kernel: d-con r0: receiver (re)started Jul 10 10:22:05 nodeb kernel: d-con r0: conn( Unconnected -> WFConnection ) Jul 10 10:22:05 nodeb kernel: d-con r0: Handshake successful: Agreed network protocol version 101 Jul 10 10:22:05 nodeb kernel: d-con r0: conn( WFConnection -> WFReportParams ) Jul 10 10:22:05 nodeb kernel: d-con r0: Starting asender thread (from drbd_r_r0 [28584]) Jul 10 10:22:05 nodeb kernel: block drbd0: max BIO size = 4096 Jul 10 10:22:05 nodeb kernel: block drbd0: peer( Unknown -> Secondary ) conn( WFReportParams -> Connected ) pdsk( DUnknown -> Diskless ) Jul 10 10:22:43 nodeb kernel: block drbd0: receiver updated UUIDs to effective data uuid: 0000000000000004 Jul 10 10:22:43 nodeb kernel: block drbd0: real peer disk state = Inconsistent Jul 10 10:22:43 nodeb kernel: block drbd0: pdsk( Diskless -> Inconsistent ) Jul 10 10:22:59 nodeb kernel: block drbd0: disk( Diskless -> Attaching ) Jul 10 10:22:59 nodeb kernel: d-con r0: Method to ensure write ordering: flush Jul 10 10:22:59 nodeb kernel: block drbd0: max BIO size = 286720 Jul 10 10:22:59 nodeb kernel: block drbd0: drbd_bm_resize called with capacity == 4294834616 Jul 10 10:22:59 nodeb kernel: block drbd0: resync bitmap: bits=536854327 words=8388349 pages=16384 Jul 10 10:22:59 nodeb kernel: block drbd0: size = 2048 GB (2147417308 KB) Jul 10 10:22:59 nodeb kernel: block drbd0: Writing the whole bitmap, size changed Jul 10 10:22:59 nodeb kernel: block drbd0: bitmap WRITE of 16384 pages took 20 jiffies Jul 10 10:22:59 nodeb kernel: block drbd0: 2048 GB (536854327 bits) marked out-of-sync by on disk bit-map. Jul 10 10:22:59 nodeb kernel: block drbd0: bitmap READ of 16384 pages took 19 jiffies Jul 10 10:22:59 nodeb kernel: block drbd0: recounting of set bits took additional 29 jiffies Jul 10 10:22:59 nodeb kernel: block drbd0: 2048 GB (536854327 bits) marked out-of-sync by on disk bit-map. Jul 10 10:22:59 nodeb kernel: block drbd0: disk( Attaching -> Negotiating ) Jul 10 10:22:59 nodeb kernel: block drbd0: attached to UUIDs 0000000000000004:0000000000000000:0000000000000000:0000000000000000 Jul 10 10:22:59 nodeb kernel: block drbd0: drbd_sync_handshake: Jul 10 10:22:59 nodeb kernel: block drbd0: self 0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:536854327 flags:0 Jul 10 10:22:59 nodeb kernel: block drbd0: peer 0000000000000004:0000000000000000:0000000000000000:0000000000000000 bits:536854327 flags:4 Jul 10 10:22:59 nodeb kernel: block drbd0: uuid_compare()=0 by rule 10 Jul 10 10:22:59 nodeb kernel: block drbd0: No resync, but 536854327 bits in bitmap! Jul 10 10:22:59 nodeb kernel: block drbd0: disk( Negotiating -> Inconsistent ) Jul 10 10:22:59 nodeb kernel: block drbd0: Resync of new storage after online grow Jul 10 10:22:59 nodeb kernel: block drbd0: helper command: /sbin/drbdadm before-resync-source minor-0 Jul 10 10:22:59 nodeb kernel: block drbd0: helper command: /sbin/drbdadm before-resync-source minor-0 exit code 0 (0x0) Jul 10 10:22:59 nodeb kernel: block drbd0: Implicitly upgraded disk Jul 10 10:22:59 nodeb kernel: block drbd0: conn( Connected -> SyncSource ) disk( Inconsistent -> UpToDate ) Jul 10 10:22:59 nodeb kernel: block drbd0: Began resync as SyncSource (will sync 2147417308 KB [536854327 bits set]). Jul 10 10:22:59 nodeb kernel: block drbd0: updated sync UUID 0000000000000004:087F262FA3DE1BDF:0000000000000000:0000000000000000 Regards, -- Hiroshi Fujishima _______________________________________________ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
