Re: Dreadful gmirror performance - suggested changes to 'prefer'
I am just looking at this again, and am in a bit of a mood for writing some patches, so I wanted to run the following idea past people as regards the priority system in the 'prefer' balancing method. Just to recap, creating a gmirror creates the first device with priority zero. Adding extra devices lets you set their priorities, but you cant set negative values. The upshot is that the first device in the mirror always has the lowest priority - so you cannot (for example) create a mirror with a local disc, subsequently add a remote disc, but then set the mirror up to prefer the local drive. I am thinking of a couple of changes - the first being the patch prroposed from Andrew Snow which would create the mirror with the priority at something other than zero (128 would be my preference) so that extra devices can be inserted both above and below it. This solves the problem for newly created mirrors as the priority can then be set as appropiate. The other change I wanted to make was to add a second 'prefer' mode to gmirror though - one which would prefer the *lowest* priority instead of the highest priority. I would probably rename the existing mode to 'prefer-high' (keeping 'prefer' as a synonym for backward compatibility) and add a 'prefer-low' as well. As an existing gmirror can have it's load balance algorithm changed on the fly, this lets you change which of the drives is preferred in an existing installationg. This is precisely what you need when switching between two machines so that the local and remote drives become reversed. I havent looked at the code in detail, but I can't see that it would be too difficult. What do people think ? -pete. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Dreadful gmirror performance - suggested changes to 'prefer'
Pete French wrote: I am just looking at this again, and am in a bit of a mood for writing some patches, so I wanted to run the following idea past people as regards the priority system in the 'prefer' balancing method. Just to recap, creating a gmirror creates the first device with priority zero. Adding extra devices lets you set their priorities, but you cant set negative values. The upshot is that the first device in the mirror always has the lowest priority - so you cannot (for example) create a mirror with a local disc, subsequently add a remote disc, but then set the mirror up to prefer the local drive. Ok. I am thinking of a couple of changes - the first being the patch prroposed from Andrew Snow which would create the mirror with the priority at something other than zero (128 would be my preference) so that extra devices can be inserted both above and below it. This solves the problem for newly created mirrors as the priority can then be set as appropiate. The other change I wanted to make was to add a second 'prefer' mode to gmirror though - one which would prefer the *lowest* priority instead of the highest priority. I would probably rename the existing mode to 'prefer-high' (keeping 'prefer' as a synonym for backward compatibility) and add a 'prefer-low' as well. As an existing gmirror can have it's load balance algorithm changed on the fly, this lets you change which of the drives is preferred in an existing installationg. This is precisely what you need when switching between two machines so that the local and remote drives become reversed. I havent looked at the code in detail, but I can't see that it would be too difficult. What do people think ? Couple of ideas: - Don't use 128 as the default since it will lead people to think there's an 8-bit quantity behind the setting (and subsequently develop weird theories about how the setting works), when it isn't so. Use 100 or 1000. - Why not go all the way and make another argument or a switch that will specify exactly which drive to prefer, so you could say prefer N, where N is any drive / component, not only the one with lowest/highest priority? This is slightly more complex and will probably require an addition to the metadata (which isn't complicated but you have to be careful) and a workaround so the old semantics of the plain setting is supported. signature.asc Description: OpenPGP digital signature
Re: Dreadful gmirror performance - suggested changes to 'prefer'
Couple of ideas: - Don't use 128 as the default since it will lead people to think there's an 8-bit quantity behind the setting (and subsequently develop weird theories about how the setting works), when it isn't so. Use 100 or 1000. Are you sure it isn't an 8 bit value underneath ? I know it is defined as an int, but trying to set it to anything outside the range 0-255 results in it wrapping round (e.g. you try making the defalt 1000 and it comes out as 232). I havent looked in detail as *why* thats happening, but it certainly behaves like it is 8 bits, hence my choice of default. - Why not go all the way and make another argument or a switch that will specify exactly which drive to prefer, so you could say prefer N, where N is any drive / component, not only the one with lowest/highest priority? This is slightly more complex and will probably require an addition to the metadata (which isn't complicated but you have to be careful) and a workaround so the old semantics of the plain setting is supported. I thought about that, but the priority scheme doesnt just specify one drive, it specifies a whole set - effectively you find the highest usable drive. Just defining one isn't sufficient, you need to have a defined way of falling back if that one isn't active. Secondly I want to avoid having to search the whole list event time. Currently the list of drives is created in priority order (no matter what the balance algorithm) so the 'prefer' algorithm simply traverses the list looking for the first active drive. I wanted to keep that and thus simply convert the list to a tailq so I can traverse it both ways, rather than having to seek the whole list every time. It keeps the code as close to the current code as possible. I'll get an initial version out and then maybe take a look at doing something more complex though. cheers, -pete. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Dreadful gmirror performance - suggested changes to 'prefer'
Pete French wrote: Couple of ideas: - Don't use 128 as the default since it will lead people to think there's an 8-bit quantity behind the setting (and subsequently develop weird theories about how the setting works), when it isn't so. Use 100 or 1000. Are you sure it isn't an 8 bit value underneath ? I know it is defined as an int, but trying to set it to anything outside the range 0-255 results in it wrapping round (e.g. you try making the defalt 1000 and it comes out as 232). I havent looked in detail as *why* thats happening, but it certainly behaves like it is 8 bits, hence my choice of default. Ah, I see; It's defined as u_intd_priority;/* Disk priority. */ in struct g_mirror_disk but as uint8_t md_priority;/* Disk priority. */ in struct g_mirror_metadata. Ok, 128 looks reasonable now. I thought about that, but the priority scheme doesnt just specify one drive, it specifies a whole set - effectively you find the highest usable drive. Just defining one isn't sufficient, you need to have a defined way of falling back if that one isn't active. Secondly I want to avoid having to search the whole list event time. Currently the list of drives is created in priority order (no matter what the balance algorithm) so the 'prefer' algorithm simply traverses the list looking for the first active drive. I wanted to keep that and thus simply convert the list to a tailq so I can traverse it both ways, rather than having to seek the whole list every time. It keeps the code as close to the current code as possible. Hmm, it would seem you need N-and-upper and N-and-lower, but this is inconvenient. Your original idea is probably better. I'll get an initial version out and then maybe take a look at doing something more complex though. signature.asc Description: OpenPGP digital signature
Re: Dreadful gmirror performance - suggested changes to 'prefer'
Hmm, it would seem you need N-and-upper and N-and-lower, but this is inconvenient. Your original idea is probably better. Certainly simpler to implement. Ideally, of course, you could change the priority on the fly (which would solve all of this) but the fact that it is stored in priority order makes that a bit of a pain too I would imagine. Do you have any thoughts on dumping by the way ? I initially implemnted the code to chheck for 'prefer-low' and dump to the last disc instead of the first. But the manual page suggests ways of working with prefer which may well be broken doing it that way, so I took the change out in order to make the manpage advice still correct. I have a working implementation here now if anyone wants to test by the way - I ended up simply adding a 'prefer-low' and changing the default priority. Everything else behaves the same as it always did. -pete. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
Re: Dreadful gmirror performance - suggested changes to 'prefer'
O.K., heres an initial version of the patch - relative to 7.0 release.
Please test and let me know if there are any problems. Patch should
apply cleanly if you are in '/usr'.
cheers,
-pete.
begin 644 gmirror.patch.gz
M'XL(,/(T@`V=M:7)R;W(N[EMAIL PROTECTED](TC_+?\6$JRQV;($D/P!S
MX5L6G,1U8/:,DTV^U%J69)M%;+DE0DW'?WW[=TZ.GY=?[EMAIL PROTECTED]GJF
M_H]+N93)B\9%@'OF!,ST.QXYW/+7]^;'IF%X/'P`_XYU^/S(1B?`
MBBS+^Z6-$4YD96VK+:9IG:U=E?M'GQ#ZLKJJ)4ZO7ZSEM`A*(4#EE:@NP
M=9O-%83??\_DUDGCA-7Y_]]_7V$5%D9Y)C,G!D!XVQ[ICPS4\T_[\?^PU
[EMAIL PROTECTED],=-)CI[EMAIL PROTECTED]_;!8+%\:*Y\4V/!*;0=4P;P%K*62+(GXO
MJ^5N3C5\4([B/1%X``[HT`4\XK]U0JG9ZGB'HB^]8C-BCSPW'JX91L#0C
M-C4C5P_LG]DK^*_!EE[H3#W;8A/7F(:U\]+YAADY7XS(+L4!YQ6K/=4#56
M%T]DKO2$^Y.7U5DB;UB=P\-0.X'-IO;D6$9DJR]!FEA,^,\Y']L!,T(6
MDU0[JM1_P33F3R:A'%L!K./X?^)'[[EMAIL PROTECTED],8!KK_5ZC?T_C$M`
M%8QRHJ9VI!NAZ3A5SIT#`'UIP?$[R!E)FEM'[EMAIL PROTECTED]8K#P+']A4$Y
M'LRB,-D6!.H6#/$ZL5CSRR/POH?,/%),IW];MAP#W^`IT'9`V!VM3#A
M!4@,[EMAIL PROTECTED]:N5D!HUE]8N!=M/^!H5M2.KFJRI3#WM-I6N
MEE=490?-SR)+M+[=5=2NTBK5E5MH]K3`T7SJ!^QJSFI:85=!39(-S.$O(/H
MC8V\P,+1,[EMAIL PROTECTED]'T%D%#C6W30%$T4I##C0\-@(0;=\
MCSDP'3N3X7D6(S2R/F;!+X[EMAIL PROTECTED]/7.U#GFMA(@IQ]=:(9B^#5V)E.
M[3#*K;7$I:/.`X$`G6+P)[8084)B\4,=XK`LSGQI*DV.L`3)[EMAIL PROTECTED]
MH_)%7?\KKNGZAG64L#!9.OLXI:!Y!P*=!7ZEO1A'/#=?-XXJT$_M*SY,`'
MJ:FP]W`NF1I\SKS/;`WOMPL#L=(G(9R`'=*X[2_1FL0A#(6QX+'7#M
MT)2U,([=DRJTTFUKZ[WC1C19[6AW6UJI=FB=4]0.I!7E,)ONK`NWX6F
M?!:NNN;8(GO/^HW=U=_ZUVCA0K=]._'^EO[H:]RZMW533)#19/P+_!K-T
MS_X6H:FM2Z/[EMAIL PROTECTED]);LG__D9E^^L/3D-?^EAM25Z^?[?#
MJ3KE.D^/[51]?:R8CK0EX`_L(BP([EMAIL PROTECTED]CM926UJGPQ;^X*XU-,$Q)+SP
M)Y%YOC=]P][MW8V*_8X'E[EMAIL PROTECTED]'9E(L;[K]!91(-]'2N.G^^0D2AAD$
[EMAIL PROTECTED]'^-^-_N#=[?W%`HT5;/P35ZIG2X`0(CL!PH^MST%B_(#0[(Q
MZQ%X#\F4#+.VP%_`HXC6C\^X/[EMAIL PROTECTED]\B/'D0S\;)M%8I%=0(C(
M#BO4LRNLGR9$*18!'KZQ[UZS?[W5;_O#X=U0O^[?_TU_W/Y%GX;CCZ=YYBR
M7$`D9MQ2)4VK?@Y\=SM90P7Z()QX0;B6.#9/I2,$K:T$1[Q8Q$P;
M]#Z.H+'8/0%24?[EMAIL PROTECTED](H'.,GB1\D;UGJ*-3LACS;6SP7**DD3!'=2NI?2
[EMAIL PROTECTED]@=*.L6/[$[D7HYY^37J?^B141B#6WI(C4'[C.P]!C4,YZ
M3C,K5T).%`K%]5P,XEYV!U(+5LV9^GXFY2XD_89$D/8DXAO#1,QV+#'S*
ML-X/[]]650C4K^TOF%.]#+N,(-EX.5]`0`1_7R$)X@SYG8#P/XR:'F
M-N'Y[/X.IP_N#F?%YF#KK/M/@[EMAIL PROTECTED]1`()%'5BB(3TI_4V:
M?4)NYB3K9H3-]7;C/0?@_4_D_.+)ZQS)[EMAIL PROTECTED]
M5,Y=R;[EMAIL PROTECTED]['=5)F^!RG,TN;Z7R8J:)EAH`YEID:[EMAIL PROTECTED]/F
M:QL`PQI\G-'#V(.2;\Q5N*)E7DA^E(]'[EMAIL PROTECTED])D8
MJFC-AMJ$E([_THF#WN-7O!3?K+3$A2S0$4Q_0$=Q:[EMAIL PROTECTED])K:1C/(
M+)/W5,[EMAIL PROTECTED]B3X8TR9$QI.T0TKG#E?3SB.:WI^LR^UFSDI2:RZ#`#
M1U1$V+YGFY%MB7W8M!7/CW`[QAB$021Q'!W2R(Q)!([EMAIL PROTECTED]**:!
M$5XF%9W'_,[EMAIL PROTECTED]'X(A=1XN9_DL]%FFBU8SN[EMAIL PROTECTED]
M^)S#EDV(9GL1!!DJ1;+2*RI.;,-A\XYL,T7SY$(4#:#(M+!^2YF/US1'/8
M\L*8XEHNXS/^+3Q8/U)NBNDFJC#E;\:/!DF+#SV`(!'[EMAIL PROTECTED](B`VL9
M(:33CCF#C;FP'Q(7VLP1'#3AR,O0^E-=Z$9)]3S1\HS=_W5./J(4^TQAP
MC1U?CWSY`O,/L`4[EMAIL PROTECTED]EM^[EMAIL PROTECTED]/J#'[EMAIL PROTECTED]
MG/[`!/GM;TIB-A%;I-)L;`F3$.;QIX%H-F,%F,%;0'^^EEW/`2R)^7
M]M*N_1',)NR5TE-%P1,!Q*JXX7(552UU:3*7ZM3[EMAIL PROTECTED],H=,!60^
M9'O6R@K)R^(];\C%N7CD%5.):5PA4KA2:`VA'8/Z=E:H0GH$?:6.-\*S3
MT*BH_U2X8ZC(1)`L`3[K9-9J1:S$JXR[%5H8CT_E3TXJTK]/\;1P*L1=S
MVL9TE'*B=!38$R[G8$MF8OSJW8[W3EO8--RB]%DE'V55.!I4_40%]]
M%\=)ZH0:]:E)V:[X)6-Z`/Q]ZP_M^4:F\U]K2Q0QWBR^:[EMAIL PROTECTED];3QK
M'%$X6N=\-JFVCJ-BYOH#-XXH-@%5G(?^5*RZ++3R`L-YQ_2O#(M]%'VQ
[EMAIL PROTECTED]/.:Q%1%EA-YAE$8C[P_BV;@[S06VJ`]`(XT1%2R/G4;O'43VVW
MD]Q/F+LRP\AOX=*!V?]Z;Q'^S*7EHW0:JBKAVD`[Y(E?KB\N1Q]?2;
MN\OK+L^8LM0G*IHLCCB3$5'7H?LQV'O36_8Y89AK6F,$=8W8]'?]=^^ZW*=
MVHRJP511QTI5;R/BF[N?=L.KY,ER\/AW?O!-?S_0W^PAI7\1D?G-SHI`_A1
M=9ITQ==I)8I.82/W];W!W6WO%N-C#^-D#'[EMAIL PROTECTED]@A=(DE#U@G4X3WEF$)
MO2_(X'[EMAIL PROTECTED](3T`:Q37L3C:7DFKLAZEZY!E!;[EMAIL PROTECTED]/'_6E
MAS$U5I):4?)[EMAIL PROTECTED]``WFI/IR?QR6[EMAIL PROTECTED]/[EMAIL PROTECTED]
M*(Z?S=EH))`*AKZ3QH+%[3:,J)X/5IAM=^L`##F[U^R%U:-$`X/_;OQ5*
MZ$4KSG6.:/?,[O\A5A0S3(,.,[EMAIL PROTECTED])J7R`A/J1GZ,K/[EMAIL PROTECTED]']
M$J]$,B1F'),-X#H)H?'4W/W5UW.HVKA%H94-*+F,*M4(J\3(Q/ZQ
MTDKL:6K$GE8[:]1X^)*]4^UQ:P=BVN.RK/8KZRT,H7M42A60W.1ANX(Q
M:E$9[N9T,G+5DCMT)[[7ZB,0:+)14X^0T*-LWU^^Z6TBI,$BKLIE(\
M\MY\5?!B_B05\QL*BX'5TF5I%#S[EMAIL PROTECTED]/CPXRSQ44_P3)@7:S442H
MSI^5D0FX/F%I'`92%V)[EMAIL PROTECTED][EMAIL PROTECTED])@%+PKA
M_])-X2J.)S6FY92JZ090:ZI*:C5+BL?O8+55)-D)(?P[(=*+S8T0
MVH%/.%M%KJWPH;3%L8MBQ*0TG8`10Y6RMY:YZ:[EMAIL PROTECTED]('%WC5,DS
M-9MYS_3#X'!?YPGI!0J'CVXP=$];=9P88,*PQ,[EMAIL PROTECTED]
M0N_TCH]'B)E#)=BBT,$KB(0IM_'D#?`D%**[%PFH)V07*V:V0HT
M=SRQTTQ#1;%%)K0]2RS!05/52E2X]Z$W.G7=X/13Y?]4=QAV]+.L`I7UUJ0
MKM-PE,';U[G+PMG==%?5DJ1[9=B#A\E9U+O]?2IV%J2F-_-\;CO1W
MOOK+9U`=4)*CS'FHKL[FB!E+(JYOZC1]JK-2'M69_+*D]Q178;6F3?
MB)@@8,_]'`=ZC5985R1+^N!23-YGPDY,'J1)M;H5XLZ6M,VD;1CV*V^
MR?=KY!9Y2CW;(FVZDHM$7[X9]89K:Z13*!Y_A,8MR4)IW9UIVH^0I$S@
M_U!#\4Y(.+/U%BKGV+@QQK842C8ZJCI!P1;.J92(T6!BP_BWW:H=+LT\
M*C5,JA1,G6JM+96\)O^C?^5;3-,^OUBBRZ18)ZGWH7_5$3UKO?C2\^U3C
M[K8;T$M7^N-$F[EMAIL PROTECTED],6G::[EMAIL
Re: Dreadful gmirror performance - suggested changes to 'prefer'
I havent looked at the code in detail, but I can't see that it would be too difficult. What do people think ? If the first drive is always priority=0, then it is going to be stuck at the highest priority, or under your plan, the lower priority. My original idea OTOH (starting the counting at 100) solves the problem with aone-line patch. Call me biased but this is what I prefer :-) - Andrew ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to [EMAIL PROTECTED]
