Re: [SLUG] Download speed
On 22/05/14 08:27, Edwin Humphries (text) wrote: > Can anyone suggest a way of testing the download speed of my NBN fibre > connection every hour and logging it? I have an ostensibly 100Mbps > connection, but the speed seems to vary enormously, so an automated > process would be good. The speed would be expected to vary enormously across the internet, only to nearby (and that pretty much means NSW, QLD or VIC for a user in NSW) servers that themselves have 100Mbit spare would you actually approach 100Mbit. Reasons you might not get 100Mbit, just off the top of my head, almost certainly widely incomplete. But first there's one important thing to know, the bandwidth/delay product, which in very simple terms means that as soon as there's any packet loss at all (and there's always some) the usable throughput is defined by a function of that loss and the latency of the connection. This is why achieving 100Mbit to Australian servers isn't too hard, even with a moderate amount of packet loss, but you can tolerate nearly no loss on a connection to a European server. http://en.wikipedia.org/wiki/Bandwidth_delay_product Your domain: 1. Client device incapable of 100Mbit transfers, still surprisingly common. OS, hardware, and client software all matter. 2. Client device to local gateway not capable of 100Mbit transfers. Wired gigabit ethernet (or *perhaps* 802.11n in an RF quiet environment) is needed for this. A bad ethernet cable may cause this too, always use pre-made (moulded strain relief) cables, humans are not reliable enough to make gigabit ethernet cables. Also, avoid a total cable length above 50m, many lower-end switches/routers can't actually drive ethernet to the full spec distance. 3. Local gateway (What's commonly called a router, industry call CPE), even fairly current gen stuff can struggle at 100Mbit transfer rate, especially if there's a high rate of session creation, or the router suffers from "bufferbloat". Nbnco's domain: 1. Media errors on the fibre, this should be monitored by NBNco. 2. Oversubscription on the fibre, the NBNco fibre is (currently) lit with GPON, which is a 2.5Gb signal (down), normally shared between 32 premises (64 or more is possible, but IIRC not used in .au). This should almost never occur if everyone's only at 100Mbit, even at higher speeds it works surprisingly well. Again, NBNco should know if this happens. 3. Oversubscription between the OLT (fibre head) and the ISP. Shouldn't happen, again monitoring should happen. (Your) ISP's domain: (Actually if you're not with one of the major providers there's a couple of extra interconnect points that can congest as well) 1. Oversubscription on the connection from NBNco to the ISP, this is known to cause some issues, sadly the best writeup of it I can't find. Actually monitoring this is a pain, the ISP can do some bits but are unlikely to do so, and NBNco probably don't either. 2. Congestion within the ISPs access network between the NBNco POI (120 nationally) and the ISP's core POP (usually 1 or 2 per city for all but the very largest). For large ISPs this is highly unlikely, and the smaller guys are covered by the wholesalers who don't save anything (significant) to get it wrong. 3. Congestion within an ISPs core. Unlikely, expect perhaps on inter-state links. 4. Congestion between an ISP and the next ISP in the path. Depending on who's involved this is either very unlikely or guaranteed. In the middle: 1. Did someone put in a stateful firewall which breaks TCP. This is pretty much all of them, this makes things a lot worse in high-latency or high-loss environments. 2. Is the server far away (see bandwidth/delay product above). The end server: 1. Does it have a path into a decent ISP network at > 100Mbit? 2. Can it actually serve at > 100Mbit? (plain static files are fairly easy, dynamic content can get expensive quick). The same caveats about OS, hardware and software apply as for the clients. -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] Download speed
ok, I patched those links in. Forgive my code posting, last time. I will answer offlist if I get any further requests. It has some command line options to specify the size to download, where to put the logfile, and the duration to wait before downloads. $> python x_download_test.py 1M -i 5 -Python- # /usr/bin/python import time, optparse, urllib2, csv if __name__ == "__main__": dlurls = {"1M" : "http://mirror.internode.on.net/pub/test/1meg.test";, "10M" : "http://mirror.internode.on.net/pub/test/10meg.test";, "50M" : "http://mirror.internode.on.net/pub/test/50meg.test";, "100M" : "http://mirror.internode.on.net/pub/test/100meg.test ", "1G" : "http://mirror.internode.on.net/pub/test/1000meg.test ", "5G" : "http://mirror.internode.on.net/pub/test/5000meg.test " } print("Network Download speed logger. Freeware Licence") usage = "usage: %prog [options] arg1 arg2" parser = optparse.OptionParser(usage=usage) parser.add_option("-i", "--interval", action="store", type="float", dest="interval", default=10, help="Interval in minutes between downloads") parser.add_option("-l", "--logfile", action="store", dest="logfilename", default="download_times.csv", help="Interval in minutes between downloads") (options, args) = parser.parse_args() download_size = "10M" if len(args) > 0: download_size = args[0] download_interval = options.interval * 60 download_url = dlurls[download_size] # setup a logfile f = open(options.logfilename, 'a') writer = csv.writer(f) while (1): print("Downloading %s" % download_size) # Initial Time reading start = time.clock() mp3file = urllib2.urlopen(download_url) mp3file.read() elapsed = time.clock() - start writer.writerow([time.strftime("%c"),download_size,elapsed,]) print("Pausing for %f minute(s)" % int(download_interval/60)) time.sleep(download_interval) # Time in seconds. On Thu, May 22, 2014 at 1:40 PM, Rick Welykochy wrote: > David wrote: > >> On 22/05/14 08:38, Rick Welykochy wrote: >> >>> Edwin Humphries (text) wrote: >>> Can anyone suggest a way of testing the download speed of my NBN fibre connection every hour and logging it? I have an ostensibly 100Mbps connection, but the speed seems to vary enormously, so an automated process would be good. >>> >>> Download a file of known length, say 1000 MB, from a server >>> whose speed you can trust every hour. Time and log each download. >>> Also verify the contents of the downloaded file with an md5 or sha >>> digest. >>> >>> This can be automated with an scp inside a simple (shell) script. >>> >>> >> Westnet used to have a file available for exactly this purpose - I dare >> say other ISP's do too. Perhaps you could ask your own ISP. >> > This looks promising: > > http://mirror.internode.on.net/pub/test/ > > I found this via a web search for "test download file residing on an isp > australia". > > > cheers > rickw > > > -- > > Rick Welykochy || Vitendo Consulting > > If consumers even know there's a DRM, what it is, and how it works, we've > already failed. > -- Peter Lee, Disney Executive > > > -- > SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ > Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html > -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] Download speed
David wrote: On 22/05/14 08:38, Rick Welykochy wrote: Edwin Humphries (text) wrote: Can anyone suggest a way of testing the download speed of my NBN fibre connection every hour and logging it? I have an ostensibly 100Mbps connection, but the speed seems to vary enormously, so an automated process would be good. Download a file of known length, say 1000 MB, from a server whose speed you can trust every hour. Time and log each download. Also verify the contents of the downloaded file with an md5 or sha digest. This can be automated with an scp inside a simple (shell) script. Westnet used to have a file available for exactly this purpose - I dare say other ISP's do too. Perhaps you could ask your own ISP. This looks promising: http://mirror.internode.on.net/pub/test/ I found this via a web search for "test download file residing on an isp australia". cheers rickw -- Rick Welykochy || Vitendo Consulting If consumers even know there's a DRM, what it is, and how it works, we've already failed. -- Peter Lee, Disney Executive -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] Download speed
Maybe something like this (python code): # /usr/bin/python import time import urllib2 if __name__ == "__main__": while (1): # Initial Time reading start = time.clock() mp3file = urllib2.urlopen("http://www.slug.org.au/event/91";) output = open('test.mp3','wb') output.write(mp3file.read()) output.close() print time.clock() - start time.sleep(3600) # Time in seconds. On Thu, May 22, 2014 at 9:34 AM, David wrote: > On 22/05/14 08:38, Rick Welykochy wrote: > >> Edwin Humphries (text) wrote: >> >>> Can anyone suggest a way of testing the download speed of my NBN fibre >>> connection every hour and logging it? I have an ostensibly 100Mbps >>> connection, but the speed seems to vary enormously, so an automated process >>> would be good. >>> >> >> Download a file of known length, say 1000 MB, from a server >> whose speed you can trust every hour. Time and log each download. >> Also verify the contents of the downloaded file with an md5 or sha >> digest. >> >> This can be automated with an scp inside a simple (shell) script. >> >> > Westnet used to have a file available for exactly this purpose - I dare > say other ISP's do too. Perhaps you could ask your own ISP. > > > > > > >> cheers >> rickw >> >> >> > -- > David McQuire > 0418 310312 > > > -- > SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ > Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html > -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] Download speed
On 22/05/14 08:38, Rick Welykochy wrote: Edwin Humphries (text) wrote: Can anyone suggest a way of testing the download speed of my NBN fibre connection every hour and logging it? I have an ostensibly 100Mbps connection, but the speed seems to vary enormously, so an automated process would be good. Download a file of known length, say 1000 MB, from a server whose speed you can trust every hour. Time and log each download. Also verify the contents of the downloaded file with an md5 or sha digest. This can be automated with an scp inside a simple (shell) script. Westnet used to have a file available for exactly this purpose - I dare say other ISP's do too. Perhaps you could ask your own ISP. cheers rickw -- David McQuire 0418 310312 -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] Download speed
Edwin Humphries (text) wrote: Can anyone suggest a way of testing the download speed of my NBN fibre connection every hour and logging it? I have an ostensibly 100Mbps connection, but the speed seems to vary enormously, so an automated process would be good. Download a file of known length, say 1000 MB, from a server whose speed you can trust every hour. Time and log each download. Also verify the contents of the downloaded file with an md5 or sha digest. This can be automated with an scp inside a simple (shell) script. cheers rickw -- Rick Welykochy || Vitendo Consulting If consumers even know there's a DRM, what it is, and how it works, we've already failed. -- Peter Lee, Disney Executive -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
[SLUG] Download speed
Can anyone suggest a way of testing the download speed of my NBN fibre connection every hour and logging it? I have an ostensibly 100Mbps connection, but the speed seems to vary enormously, so an automated process would be good. -- Thanks Edwin Humphries -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
[SLUG] download speed reduces when cpu active
Hi All The download speed drops as cpu increase activity. I see this from gkrellm. eg compiling kernel, opening OO, playing xine. What is happening here? is this normal? I don't think so. How can this be rectified? This is a debian box, with AMD 1.2thunderbird with Via chip set, and uses external modem. all help appreciated TIA Russell -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html