Re: Processed: destruction of round-robin functionality is fucking up our mirrors and making Debian suck for many people, hence fixing this is a release-critical wish
On Sun, Dec 16, 2007 at 03:45:37AM +0100, Josip Rodin wrote: After around 11 hours, we've had: * villa 4.29 MB/s * lobos 3.91 MB/s * steffani 14.86 MB/s The rule9 prediction was: A: 000.000.000.000-127.255.255.255: steffani, villa, lobos B: 128.000.000.000-191.255.255.255: steffani C: 192.000.000.000-212.211.131.255: villa, lobos D: 212.211.132.000-212.211.132.127: villa E: 212.211.132.128-212.211.132.255: lobos F: 212.211.133.000-255.255.255.255: villa, lobos Class A is pure round-robin, so we can ignore rule9 and assign 1/3 of its traffic to each host. The difference between villa and lobos is aiui due not only to the difference between the D and E IP ranges, but also because the round-robin ordering of the hosts is [lobos, steffani, villa], which means that since rule9 happens after round-robin, you get orderings: lobos, steffani, villa - [lobos, villa], steffani steffani, villa, lobos - [villa, lobos], steffani villa, lobos, steffani - [villa, lobos], steffani A/3 + B = 14.86 MB/s (steffani) A/3 + 2C/3 + 2F/3 + D = 4.29 MB/s (villa) A/3 + C/3 + F/3 + E = 3.91 MB/s (lobos) If you assumethe 212.211.132.0/24 traffic is negligible, and thus D = E = 0, then subtracting lobos from villa gives: C/3 + F/3 = 4.29 MB/s - 3.91 MB/s = 0.38 MB/s And thus filling in for lobos, we get A/3 = 3.91 MB/s - 0.38 MB/s = 3.53 MB/s. Going back to steffani, that gives B = 14.86 MB/s - 3.53 MB/s = 11.33 MB/s, and we thus have: A = 10.59 MB/s B = 11.33 MB/s C + F = 1.14 MB/s D = 0MB/s (by assumption) E = 0MB/s (by assumption) Which gives us 23.06 MB/s which was the total of what we started with, yay. Note that 192.168.*.* addresses are in class C, so can only possibly make up just under 5% of our traffic, which seems pretty negligible. 10.*.*.* addresses are in class A, and 172.16... class addresses are in class B. I would've thought there wouldn't be significantly more of those than for the 192.168.*.* private addresses though. Anyway, hope that's of some use. Cheers, aj signature.asc Description: Digital signature
Re: Processed: destruction of round-robin functionality is fucking up our mirrors and making Debian suck for many people, hence fixing this is a release-critical wish
On Sat, Dec 15, 2007 at 03:43:22PM +0100, Josip Rodin wrote: On Sat, Dec 15, 2007 at 03:38:01PM +0100, Josip Rodin wrote: Steve pointed me to [...] BTW, if anyone reading has some time to do the math again (hi aj :) Hrm. How do you generalise it? ... Ah, like that. Attached. $ ./rule9-prediction.py ftp.us.debian.org 000.000.000.000-063.255.255.255: 035.009.037.225 064.000.000.000-127.255.255.255: 064.050.236.052 128.000.000.000-191.255.255.255: 128.030.002.036 192.000.000.000-255.255.255.255: 204.152.191.039 $ ./rule9-prediction.py security.debian.org 000.000.000.000-127.255.255.255: 128.031.000.036, 212.211.132.032, 212.211.132.250 128.000.000.000-191.255.255.255: 128.031.000.036 192.000.000.000-212.211.131.255: 212.211.132.032, 212.211.132.250 212.211.132.000-212.211.132.127: 212.211.132.032 212.211.132.128-212.211.132.255: 212.211.132.250 212.211.133.000-255.255.255.255: 212.211.132.032, 212.211.132.250 You can specify multiple addresses, either by name or IP. If you get non-IPv4 addresses it won't work though. Cheers, aj signature.asc Description: Digital signature
Re: Processed: destruction of round-robin functionality is fucking up our mirrors and making Debian suck for many people, hence fixing this is a release-critical wish
On Sun, Dec 16, 2007 at 06:07:24PM +1000, Anthony Towns wrote: Ah, like that. Attached. $ ./rule9-prediction.py ftp.us.debian.org No, really. Cheers, aj #!/usr/bin/env python import socket, sys def ip2bits(ip): return .join( [ .join( [ (%d % (int(x*(2**-y)) % 2)) for y in range(7,-1,-1) ] ) for x in [ int(x,10) for x in ip.split(.) ] ] ) def bits2ip(bits): return ..join([ %03d % (int(bits[x*8:x*8+8],2)) for x in range(4) ]) def iprange(prefix): lo = bits2ip( (prefix + 0*32)[:32] ) hi = bits2ip( (prefix + 1*32)[:32] ) if hi == 255.255.255.255: nx = else: nx = bits2ip( (prefix[:prefix.rfind(0)] + 1 + 0 * 31)[:32] ) return (lo, hi, nx) ips = [] for arg in sys.argv[1:]: ips.extend( [ (x[4][0], ip2bits(x[4][0])) for x in socket.getaddrinfo(arg, http) ] ) ips.sort( lambda x,y: cmp(x[1], y[1]) ) if ips == []: print Usage: %s host|address host|address... % sys.argv[0] sys.exit(1) working = [ (, [b for ip,b in ips]) ] result = [] while len(working) 0: newworking = [] for (prefix, ips) in working: if len(ips) == 1: result.append( (prefix, ips) ) continue prefixL = prefix + 0 prefixR = prefix + 1 ipsL = [] ipsR = [] for ip in ips: if ip.startswith(prefixL): ipsL.append(ip) else: ipsR.append(ip) if ipsL == []: result.append( (prefixL, ips) ) else: newworking.append( (prefixL, ipsL) ) if ipsR == []: result.append( (prefixR, ips) ) else: newworking.append( (prefixR, ipsR) ) working = newworking result.sort(lambda x,y: cmp(x[0], y[0])) prefix, ips = result[0] lo, hi, nx = iprange(prefix) for p2, ips2 in result[1:]: l2, h2, n2 = iprange(p2) if nx != l2: print OUT OF ORDER sys.exit(1) if ips != ips2: print %15s-%15s: %s % (lo, hi, , .join([bits2ip(b) for b in ips])) lo, hi, nx, ips = l2, h2, n2, ips2 else: hi, nx = h2, n2 print %15s-%15s: %s % (lo, hi, , .join([bits2ip(b) for b in ips])) signature.asc Description: Digital signature
Re: Processed: destruction of round-robin functionality is fucking up our mirrors and making Debian suck for many people, hence fixing this is a release-critical wish
On Sun, Dec 16, 2007 at 06:28:39PM +1000, Anthony Towns wrote: Going back to steffani, that gives B = 14.86 MB/s - 3.53 MB/s = 11.33 MB/s, and we thus have: A = 10.59 MB/s B = 11.33 MB/s C + F = 1.14 MB/s D = 0MB/s (by assumption) E = 0MB/s (by assumption) Hrm, presuming the change in glibc 2.7-4 worked as expected, all the users of unstable hitting security.debian.org should be included in the A count, above. I presume there's a good chance glibc 2.7-4 will hit testing tomorrow sometime, which will mean testing users in other categories will get moved into A too, and might result in a noticable change in the balance. Cheers, aj signature.asc Description: Digital signature
Re: Processed: destruction of round-robin functionality is fucking up our mirrors and making Debian suck for many people, hence fixing this is a release-critical wish
On Sat, Dec 15, 2007 at 07:15:53PM -0800, Steve Langasek wrote: At the same time, I should just get off my ass and go find the resolving code in the applications (apt-get, rsync, ...), and then just run it to see how it works... I edited apt's methods/connect.cc:Connect() function after the line saying struct addrinfo *CurHost = LastHostAddr; to do: char hbuf[NI_MAXHOST], sbuf[NI_MAXSERV]; getnameinfo( CurHost-ai_addr, CurHost-ai_addrlen, hbuf, sizeof(hbuf), sbuf, sizeof(sbuf), NI_NUMERICHOST ); char *msg; sprintf(msg, \nUsing IP address: %s, with service %s\n, hbuf, sbuf); return _error-Error( msg ); Then I ran it with a minimal sources.list file including only security.d.o: You haven't mentioned here what versions of apt and glibc you're using, and whether you have any settings in /etc/gai.conf. Oh, okay. That apt was apt-get source'd from sid, 0.7.9. The machine is running etch, so it has libc6 2.3.6.ds1-13etch2. /etc/gai.conf doesn't exist. -- 2. That which causes joy or happiness. -- To UNSUBSCRIBE, email to [EMAIL PROTECTED] with a subject of unsubscribe. Trouble? Contact [EMAIL PROTECTED]
