Agreed. Risks\rewards. It'd be great if we had dual infrastructures like INEX or LINX everywhere. Give some redundancy to mitigate *some* of that risk by having everything on an IX.
----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest Internet Exchange http://www.midwest-ix.com ----- Original Message ----- From: "Paul Stewart" <[email protected]> To: [email protected] Sent: Monday, June 1, 2015 2:00:06 PM Subject: Re: [AFMUG] Caching Boxes Ah ok … yeah thanks … I wasn’t sure myself J As long as people are realistic when buying things like transit across a peering fabric then my personal opinion is to go ahead…. From: Af [mailto:[email protected]] On Behalf Of Mike Hammett Sent: Monday, June 1, 2015 2:47 PM To: [email protected] Subject: Re: [AFMUG] Caching Boxes Also, I think what Justin was trying to convey is that we're not limiting the IX to purely SFI. If you want to purchase your transit or other services via someone on the IX, then go right ahead. Some IXes limit it to SFI, some don't. He wasn't implying that Midwest-IX sells caching or other type of services on the IX. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest Internet Exchange http://www.midwest-ix.com ----- Original Message ----- From: "Paul Stewart" < [email protected] > To: [email protected] Sent: Monday, June 1, 2015 1:41:59 PM Subject: Re: [AFMUG] Caching Boxes On a side note, I may look at dropping a port into your IX sometime next year (via 350 Cermak). That will depend at the time on the peers you have and traffic calculations – it’s purely a mechanical calculation to see if it’s worthwhile for us… currently connecting to Equinix Chicago at the moment (via remote fiber links) From: Af [ mailto:[email protected] ] On Behalf Of Justin Wilson - MTIN Sent: Monday, June 1, 2015 2:22 PM To: [email protected] Subject: Re: [AFMUG] Caching Boxes Yeah, we look as caching as a service available through member companies. If you want to buy a caching service then the exchange is the best place to buy it due to the low latency of an exchange. If it’s cached *and* a cross connect away it’s gonna be pretty darn fast comparatively speaking. Justin Justin Wilson [email protected] http://www.mtin.net Managed Services – xISP Solutions – Data Centers http://www.thebrotherswisp.com Podcast about xISP topics http://www.midwest-ix.com Peering – Transit – Internet Exchange On Jun 1, 2015, at 2:11 PM, Paul Stewart < [email protected] > wrote: Ok ok… so they are “just peers” on the exchange like anyone else? Thanks, Paul From: Af [ mailto:[email protected] ] On Behalf Of Mike Hammett Sent: Monday, June 1, 2015 1:26 PM To: [email protected] Subject: Re: [AFMUG] Caching Boxes I am not the one providing that service, they are. They are the ones facilitating their equipment. I just support them where asked by them. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest Internet Exchange http://www.midwest-ix.com From: "Paul Stewart" < [email protected] > To: [email protected] Sent: Monday, June 1, 2015 12:07:41 PM Subject: Re: [AFMUG] Caching Boxes Purely a personal opinion … one based on working with an IX for about 5 years, working with many ISP etc… IX’s should * not * be in the business of providing caching content to their “customers” – you are in some regards competing with your own peers (especially if they sell transit services). From: Af [ mailto:[email protected] ] On Behalf Of Mike Hammett Sent: Monday, June 1, 2015 11:33 AM To: [email protected] Subject: Re: [AFMUG] Caching Boxes It does help to an extent. You didn't know my intentions, but few people would have experience with my position anyway. I was inquiring as an IX, not as a WISP. Different companies have different models and so we're trying to gauge infrastructure requirements. For instance, we're already in contract with Akamai for some boxes. They send out a few boxes and gradually increase that count until they switch to a different model. Limelight only does "superPOPs", so they'll only utilize transport from their remote facility. Netflix has caching boxes available for people, so it's assumed that they would do something similar. Google does have caching boxes, but was asking about wavelength providers in the facility. Those that provide boxes still need Internet connectivity to pull whatever isn't on the box. They work with the IX to get that. Knowing how the different caching mechanisms work provides background as to the scale of support infrastructure they need. Of course they'll tell you when they sign what they require, but knowing ahead of time is nice. We offered some of them transport, but due to their infrastructure, they preferred transit over transport. Just looking to be as helpful as we can and make it as cost effective as we can for everybody. We are up to about 2.3M IPs either advertised today or are in process of getting hooked up on our Indy IX and looking to branch out soon. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest Internet Exchange http://www.midwest-ix.com From: "Paul Stewart" < [email protected] > To: [email protected] Sent: Monday, June 1, 2015 10:19:14 AM Subject: Re: [AFMUG] Caching Boxes So I’ll try to answer this for you … have all of them… Percent of bandwidth saved… really depends on your network, traffic levels, subscriber breakdown (residential vs business), connectivity etc… there’s no magic answer but if you had all three systems you listed then it should hit a minimum of 50%. Upstream amount used is less than you used before J Akamai and Google both “fill on demand” so they are generally speaking very linear in upstream vs content served out of them – over time that difference is quite noticeable. Netflix uses a “prefill” mechanism which is typically 1.5Gb/s for 8-10 hours per day off hours so if you’re not serving at least 2-3Gb/s of Netflix traffic it’s not remotely worthwhile. Also consider the space/power that some of these servers require – it can add up in a hurry. Generally speaking I wouldn’t deploy any of their solutions until you have 1Gb/s on Akamai minimum, 3 Gb/s minimum on Google/Youtube, and 5 Gb/s minimum on Netflix… assuming you have lots of space/power. These are just rough guidelines – everyone’s situation is different. Netflix servers are the heaviest density servers and depending on traffic levels then Akamai is next in density but you will only see those boxes for Akamai in very high traffic environments – their “stock” servers I think do about 700-800Mb/s per physical server (usually a cluster of 3 to start), while their more dense servers do 10Gb/s+ per box. Google servers typically go up to a max of 3.1-3.2 Gb/s per server (usually a cluster of 3 to start as well). Netflix servers (latest generation “C”) have 2x10G interfaces and can serve up to around 15Gb/s each – they usually will start you with 2 of these servers but not always. The ASN that fills the caches varies all over the place and is totally dependent on your network and upstreams. Peering links with these providers are heavily used when possible. Let me know if this helps… Paul From: Af [ mailto:[email protected] ] On Behalf Of Mike Hammett Sent: Sunday, May 31, 2015 10:10 AM To: Animal Farm Subject: [AFMUG] Caching Boxes For those of you with caching boxes from Akamai, Gogole, NetFlix, etc. could you let me know: * Percent of bandwidth saved (IE, 10 gigs of downstream traffic and 1 gig of upstream traffic would be 90% saved) * Upstream amount used * ASN the upstream comes from * Traceroute to the upstream (to see where it actually comes from) On or off list for any of the information above is fine. ----- Mike Hammett Intelligent Computing Solutions http://www.ics-il.com
