On Thu, Jul 11, 2013 at 05:32:00PM +0900, Hiroshi Fujishima wrote: > Hi all, > > With drbd-8.4.3, I noticed that performing `drbdadm attach r0' with a > newly-initialized empty disk causes initial full synchronization. > > In the following document, it seems it is necessary to perform `drbdadm > primary --force resource' to start the initial full synchronization.
If you follow the proper sequence (*create-md* first, then attach, then connect). > > http://www.drbd.org/users-guide-emb/s-initial-full-sync.html > > However, it starts without performing `drbdadm primary --force > resource'. > > I'd like to know whether this is an expected behavior or not. Actually yes, given your sequence of events, this is expected. To DRBD, this lokks like "online grow" (because you started with device size of 0): > Jul 10 10:22:59 nodea kernel: block drbd0: Resync of new storage after online > grow A bit ugly, because apparently that way you you may end up with a current uuid of "just created", while being "uptodate". This later detail should be fixed. Anyways: don't do that. > > * enviroment > > OS: Scientific Linux release 6.2 (Carbon) > DRBD: drbd-8.4.3 > > * configuration > > # /etc/drbd.conf > common { > } > > # resource r0 on nodea: not ignored, not stacked > # defined at /etc/drbd.d/r0.res:1 > resource r0 { > on nodea { > volume 0 { > device /dev/drbd0 minor 0; > disk /dev/sdc; > meta-disk internal; > } > address ipv4 192.168.0.1:7789; > } > on nodeb { > volume 0 { > device /dev/drbd0 minor 0; > disk /dev/sdc; > meta-disk internal; > } > address ipv4 192.168.0.2:7789; > } > net { > protocol C; > } > disk { > resync-rate 2g; > c-plan-ahead 0; > } > } > > * How To reproduce > > (1) cleanup metadata in /dev/sdc > > [root@nodea ~]# MegaCli ldinit start l2 a0 > > Start Initialization on Virtual Drive 2 (target id: 2) Success. > > Exit Code: 0x00 > [root@nodeb ~]# MegaCli ldinit start l2 a0 > > Start Initialization on Virtual Drive 2 (target id: 2) Success. > > Exit Code: 0x00 > > (2) drbd start > > [root@nodea ~]# service drbd start > Starting DRBD resources: [ > create res: r0 > prepare disk: r0 > adjust disk: r0:failed(apply-al:255) > adjust net: r0 > ] > .... > [root@nodeb ~]# service drbd start > Starting DRBD resources: [ > create res: r0 > prepare disk: r0 > adjust disk: r0:failed(apply-al:255) > adjust net: r0 > ] > . > [root@nodea ~]# cat /proc/drbd > version: 8.4.3 (api:1/proto:86-101) > GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@nodea, > 2013-07-09 17:08:55 > 0: cs:Connected ro:Secondary/Secondary ds:Diskless/Diskless C r----- > ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0 > [root@nodeb ~]# cat /proc/drbd > version: 8.4.3 (api:1/proto:86-101) > GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@nodeb, > 2013-07-09 17:10:11 > 0: cs:Connected ro:Secondary/Secondary ds:Diskless/Diskless C r----- > ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0 > > => Disk states is Diskless > > (3) create metadata > > [root@nodea ~]# drbdadm create-md r0 > Writing meta data... > initializing activity log > NOT initializing bitmap > New drbd meta data block successfully created. > success > [root@nodeb ~]# drbdadm create-md r0 > Writing meta data... > initializing activity log > NOT initializing bitmap > New drbd meta data block successfully created. > success > > (4) attach backing device > > [root@nodea ~]# cat /proc/drbd > version: 8.4.3 (api:1/proto:86-101) > GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@nodea, > 2013-07-09 17:08:55 > 0: cs:Connected ro:Secondary/Secondary ds:Diskless/Diskless C r----- > ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0 > [root@nodea ~]# drbdadm attach r0 > [root@nodea ~]# cat /proc/drbd > version: 8.4.3 (api:1/proto:86-101) > GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@nodea, > 2013-07-09 17:08:55 > 0: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Diskless C r----- > ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:f oos:2147417308 > [root@nodeb ~]# cat /proc/drbd > version: 8.4.3 (api:1/proto:86-101) > GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@nodeb, > 2013-07-09 17:10:11 > 0: cs:Connected ro:Secondary/Secondary ds:Diskless/Inconsistent C r----- > ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0 > [root@nodeb ~]# drbdadm attach r0 > [root@nodeb ~]# cat /proc/drbd > version: 8.4.3 (api:1/proto:86-101) > GIT-hash: 89a294209144b68adb3ee85a73221f964d3ee515 build by root@nodeb, > 2013-07-09 17:10:11 > 0: cs:SyncSource ro:Secondary/Secondary ds:UpToDate/Inconsistent C r---n- > ns:460616 nr:0 dw:0 dr:468248 al:0 bm:28 lo:6 pe:11 ua:80 ap:0 ep:1 wo:f > oos:2146957764 > [>....................] sync'ed: 0.1% (2096636/2097084)M > finish: 1:17:51 speed: 459,544 (459,544) K/sec > > (5) log > > [nodea] > Jul 10 10:22:03 nodea kernel: events: mcg drbd: 2 > Jul 10 10:22:03 nodea kernel: drbd: initialized. Version: 8.4.3 > (api:1/proto:86-101) > Jul 10 10:22:03 nodea kernel: drbd: GIT-hash: > 89a294209144b68adb3ee85a73221f964d3ee515 build by root@nodea, 2013-07-09 > 17:08:55 > Jul 10 10:22:03 nodea kernel: drbd: registered as block device major 147 > Jul 10 10:22:03 nodea kernel: d-con r0: Starting worker thread (from > drbdsetup [32273]) > Jul 10 10:22:03 nodea kernel: d-con r0: conn( StandAlone -> Unconnected ) > Jul 10 10:22:03 nodea kernel: d-con r0: Starting receiver thread (from > drbd_w_r0 [32275]) > Jul 10 10:22:03 nodea kernel: d-con r0: receiver (re)started > Jul 10 10:22:03 nodea kernel: d-con r0: conn( Unconnected -> WFConnection ) > Jul 10 10:22:05 nodea kernel: d-con r0: Handshake successful: Agreed network > protocol version 101 > Jul 10 10:22:05 nodea kernel: d-con r0: conn( WFConnection -> WFReportParams ) > Jul 10 10:22:05 nodea kernel: d-con r0: Starting asender thread (from > drbd_r_r0 [32276]) > Jul 10 10:22:05 nodea kernel: block drbd0: max BIO size = 4096 > Jul 10 10:22:05 nodea kernel: block drbd0: peer( Unknown -> Secondary ) conn( > WFReportParams -> Connected ) pdsk( DUnknown -> Diskless ) > Jul 10 10:22:43 nodea kernel: block drbd0: disk( Diskless -> Attaching ) > Jul 10 10:22:43 nodea kernel: d-con r0: Method to ensure write ordering: flush > Jul 10 10:22:43 nodea kernel: block drbd0: max BIO size = 286720 > Jul 10 10:22:43 nodea kernel: block drbd0: drbd_bm_resize called with > capacity == 4294834616 > Jul 10 10:22:43 nodea kernel: block drbd0: resync bitmap: bits=536854327 > words=8388349 pages=16384 > Jul 10 10:22:43 nodea kernel: block drbd0: size = 2048 GB (2147417308 KB) > Jul 10 10:22:43 nodea kernel: block drbd0: Writing the whole bitmap, size > changed > Jul 10 10:22:43 nodea kernel: block drbd0: bitmap WRITE of 16384 pages took > 20 jiffies > Jul 10 10:22:43 nodea kernel: block drbd0: 2048 GB (536854327 bits) marked > out-of-sync by on disk bit-map. > Jul 10 10:22:43 nodea kernel: block drbd0: bitmap READ of 16384 pages took 18 > jiffies > Jul 10 10:22:43 nodea kernel: block drbd0: recounting of set bits took > additional 28 jiffies > Jul 10 10:22:43 nodea kernel: block drbd0: 2048 GB (536854327 bits) marked > out-of-sync by on disk bit-map. > Jul 10 10:22:43 nodea kernel: block drbd0: disk( Attaching -> Negotiating ) > Jul 10 10:22:43 nodea kernel: block drbd0: attached to UUIDs > 0000000000000004:0000000000000000:0000000000000000:0000000000000000 > Jul 10 10:22:43 nodea kernel: block drbd0: disk( Negotiating -> Inconsistent ) > Jul 10 10:22:59 nodea kernel: block drbd0: real peer disk state = Inconsistent > Jul 10 10:22:59 nodea kernel: block drbd0: drbd_sync_handshake: > Jul 10 10:22:59 nodea kernel: block drbd0: self > 0000000000000004:0000000000000000:0000000000000000:0000000000000000 > bits:536854327 flags:0 > Jul 10 10:22:59 nodea kernel: block drbd0: peer > 0000000000000004:0000000000000000:0000000000000000:0000000000000000 > bits:536854327 flags:4 > Jul 10 10:22:59 nodea kernel: block drbd0: uuid_compare()=0 by rule 10 > Jul 10 10:22:59 nodea kernel: block drbd0: No resync, but 536854327 bits in > bitmap! > Jul 10 10:22:59 nodea kernel: block drbd0: pdsk( Diskless -> Inconsistent ) > Jul 10 10:22:59 nodea kernel: block drbd0: Resync of new storage after online > grow > Jul 10 10:22:59 nodea kernel: block drbd0: conn( Connected -> WFSyncUUID ) > pdsk( Inconsistent -> UpToDate ) > Jul 10 10:22:59 nodea kernel: block drbd0: updated sync uuid > 087F262FA3DE1BDE:0000000000000000:0000000000000000:0000000000000000 > Jul 10 10:22:59 nodea kernel: block drbd0: helper command: /sbin/drbdadm > before-resync-target minor-0 > Jul 10 10:22:59 nodea kernel: block drbd0: helper command: /sbin/drbdadm > before-resync-target minor-0 exit code 0 (0x0) > Jul 10 10:22:59 nodea kernel: block drbd0: conn( WFSyncUUID -> SyncTarget ) > Jul 10 10:22:59 nodea kernel: block drbd0: Began resync as SyncTarget (will > sync 2147417308 KB [536854327 bits set]). > > [nodeb] > Jul 10 10:22:05 nodeb kernel: events: mcg drbd: 2 > Jul 10 10:22:05 nodeb kernel: drbd: initialized. Version: 8.4.3 > (api:1/proto:86-101) > Jul 10 10:22:05 nodeb kernel: drbd: GIT-hash: > 89a294209144b68adb3ee85a73221f964d3ee515 build by root@nodeb, 2013-07-09 > 17:10:11 > Jul 10 10:22:05 nodeb kernel: drbd: registered as block device major 147 > Jul 10 10:22:05 nodeb kernel: d-con r0: Starting worker thread (from > drbdsetup [28581]) > Jul 10 10:22:05 nodeb kernel: d-con r0: conn( StandAlone -> Unconnected ) > Jul 10 10:22:05 nodeb kernel: d-con r0: Starting receiver thread (from > drbd_w_r0 [28583]) > Jul 10 10:22:05 nodeb kernel: d-con r0: receiver (re)started > Jul 10 10:22:05 nodeb kernel: d-con r0: conn( Unconnected -> WFConnection ) > Jul 10 10:22:05 nodeb kernel: d-con r0: Handshake successful: Agreed network > protocol version 101 > Jul 10 10:22:05 nodeb kernel: d-con r0: conn( WFConnection -> WFReportParams ) > Jul 10 10:22:05 nodeb kernel: d-con r0: Starting asender thread (from > drbd_r_r0 [28584]) > Jul 10 10:22:05 nodeb kernel: block drbd0: max BIO size = 4096 > Jul 10 10:22:05 nodeb kernel: block drbd0: peer( Unknown -> Secondary ) conn( > WFReportParams -> Connected ) pdsk( DUnknown -> Diskless ) > Jul 10 10:22:43 nodeb kernel: block drbd0: receiver updated UUIDs to > effective data uuid: 0000000000000004 > Jul 10 10:22:43 nodeb kernel: block drbd0: real peer disk state = Inconsistent > Jul 10 10:22:43 nodeb kernel: block drbd0: pdsk( Diskless -> Inconsistent ) > Jul 10 10:22:59 nodeb kernel: block drbd0: disk( Diskless -> Attaching ) > Jul 10 10:22:59 nodeb kernel: d-con r0: Method to ensure write ordering: flush > Jul 10 10:22:59 nodeb kernel: block drbd0: max BIO size = 286720 > Jul 10 10:22:59 nodeb kernel: block drbd0: drbd_bm_resize called with > capacity == 4294834616 > Jul 10 10:22:59 nodeb kernel: block drbd0: resync bitmap: bits=536854327 > words=8388349 pages=16384 > Jul 10 10:22:59 nodeb kernel: block drbd0: size = 2048 GB (2147417308 KB) > Jul 10 10:22:59 nodeb kernel: block drbd0: Writing the whole bitmap, size > changed > Jul 10 10:22:59 nodeb kernel: block drbd0: bitmap WRITE of 16384 pages took > 20 jiffies > Jul 10 10:22:59 nodeb kernel: block drbd0: 2048 GB (536854327 bits) marked > out-of-sync by on disk bit-map. > Jul 10 10:22:59 nodeb kernel: block drbd0: bitmap READ of 16384 pages took 19 > jiffies > Jul 10 10:22:59 nodeb kernel: block drbd0: recounting of set bits took > additional 29 jiffies > Jul 10 10:22:59 nodeb kernel: block drbd0: 2048 GB (536854327 bits) marked > out-of-sync by on disk bit-map. > Jul 10 10:22:59 nodeb kernel: block drbd0: disk( Attaching -> Negotiating ) > Jul 10 10:22:59 nodeb kernel: block drbd0: attached to UUIDs > 0000000000000004:0000000000000000:0000000000000000:0000000000000000 > Jul 10 10:22:59 nodeb kernel: block drbd0: drbd_sync_handshake: > Jul 10 10:22:59 nodeb kernel: block drbd0: self > 0000000000000004:0000000000000000:0000000000000000:0000000000000000 > bits:536854327 flags:0 > Jul 10 10:22:59 nodeb kernel: block drbd0: peer > 0000000000000004:0000000000000000:0000000000000000:0000000000000000 > bits:536854327 flags:4 > Jul 10 10:22:59 nodeb kernel: block drbd0: uuid_compare()=0 by rule 10 > Jul 10 10:22:59 nodeb kernel: block drbd0: No resync, but 536854327 bits in > bitmap! > Jul 10 10:22:59 nodeb kernel: block drbd0: disk( Negotiating -> Inconsistent ) > Jul 10 10:22:59 nodeb kernel: block drbd0: Resync of new storage after online > grow > Jul 10 10:22:59 nodeb kernel: block drbd0: helper command: /sbin/drbdadm > before-resync-source minor-0 > Jul 10 10:22:59 nodeb kernel: block drbd0: helper command: /sbin/drbdadm > before-resync-source minor-0 exit code 0 (0x0) > Jul 10 10:22:59 nodeb kernel: block drbd0: Implicitly upgraded disk > Jul 10 10:22:59 nodeb kernel: block drbd0: conn( Connected -> SyncSource ) > disk( Inconsistent -> UpToDate ) > Jul 10 10:22:59 nodeb kernel: block drbd0: Began resync as SyncSource (will > sync 2147417308 KB [536854327 bits set]). > Jul 10 10:22:59 nodeb kernel: block drbd0: updated sync UUID > 0000000000000004:087F262FA3DE1BDF:0000000000000000:0000000000000000 > > Regards, > > -- > Hiroshi Fujishima > _______________________________________________ > drbd-user mailing list > drbd-user@lists.linbit.com > http://lists.linbit.com/mailman/listinfo/drbd-user -- : Lars Ellenberg : LINBIT | Your Way to High Availability : DRBD/HA support and consulting http://www.linbit.com DRBD® and LINBIT® are registered trademarks of LINBIT, Austria. __ please don't Cc me, but send to list -- I'm subscribed _______________________________________________ drbd-user mailing list drbd-user@lists.linbit.com http://lists.linbit.com/mailman/listinfo/drbd-user
