Build a CDN of CDNs? Serve some via Google, some via Akamai, some via Edgecast, some via Cloudfront, etc.
Will need a funky DNS server that can do round robin DNS (with cnames), but you probably already looking at fairly specialist DNS (your DNS traffic must spike too) On 20 May 2011 10:15, Brandon Wirtz <drak...@digerat.com> wrote: > I like you. The CDN Players want about 4x what they should . The problem > for anyone is that it screws with their Peak Valley balance. > > The Adult hosts do a lot of load balancing that doesn't have to be low > latency, they didn't think they could hit the 1.5 seconds. > > It’s the difference between serving big files at high latency and small > files at low latency. > > In theory... If I ran master slave the edge cache would just handle this and > I'd only pay for the first request. Do you think Google will let me do that > on the Free Quota? > > -----Original Message----- > From: google-appengine@googlegroups.com > [mailto:google-appengine@googlegroups.com] On Behalf Of Jeff Schnitzer > Sent: Friday, May 20, 2011 1:38 AM > To: google-appengine@googlegroups.com > Subject: Re: [google-appengine] Static Files: Here is a real Problem. Can I > solve it with GAE? > > 2M * 10K * 8 / 1.5 = you're trying to burst to ~100 Gigabits? > > I can't imagine you'll get that served out of a GAE datacenter, but it > surprises me that the big CDN players can't do it. Even the smallish > players in the pron biz sustain over 10 Gbps... but nobody is going to offer > you 95% billing for a 1.5s burst, of course. > > Jeff > > On Fri, May 20, 2011 at 1:19 AM, Brandon Wirtz <drak...@digerat.com> wrote: >> I wasn’t planning to use Python, I was using it as an example of the >> billing at the "high end" It would be interesting to see if Java can >> really get to that number. >> >> I use Instance memory in my existing solution, and agree it is fast... >> the real question is if in the time before I could get the file in to >> instance memory. Also there is just the whole would Google always >> have spare capacity for this. >> >> Likely I am porting my code to Java, to see if I can get the 20 >> threads per instance you are talking about. If so that would save me >> A Lot between now and Python 2.7 >> >> >> >> -----Original Message----- >> From: google-appengine@googlegroups.com >> [mailto:google-appengine@googlegroups.com] On Behalf Of Stephen >> Johnson >> Sent: Friday, May 20, 2011 12:52 AM >> To: google-appengine@googlegroups.com >> Subject: Re: [google-appengine] Static Files: Here is a real Problem. >> Can I solve it with GAE? >> >> As a theoretical exercise and not knowing for one second if this would >> in anyway work, but... >> Don't use Python. Use Java. Store file in instance memory. Should be >> able to serve it with latency of 25ms or even less. So 40 requests per >> second for just one thread. Should be able to get at least 20 threads >> going per instance or more if new scheduler is good since their will >> be zero cpu utilization. So you're looking at 2500 instances max plus >> or minus. I'd also prime the pump so to speak. When you know it's >> going to happen start hitting your instances from several other places >> like perhaps AWS to start kicking up instances. You're paying for the >> 15 minutes anyway. Get as many instances pre-started as possible and >> ready to go. >> >> On Fri, May 20, 2011 at 12:25 AM, Jeff Schnitzer <j...@infohazard.org> >> wrote: >>> ...and by S3 I mean CloudFront... >>> >>> On Fri, May 20, 2011 at 12:19 AM, Jeff Schnitzer >>> <j...@infohazard.org> >> wrote: >>>> S3 couldn't handle this? >>>> >>>> Jeff >>>> >>>> On Thu, May 19, 2011 at 11:21 PM, Brandon Wirtz >>>> <drak...@digerat.com> >> wrote: >>>>> While I do a Lot of things on GAE, I do a lot of things not on GAE. >>>>> I have a Client they are a BIG client. You may have their service >>>>> in >> your home. >>>>> >>>>> They need a solution that will serve a file between 3 and 15k in >>>>> size to 2M devices in about 1500ms. >>>>> >>>>> >>>>> >>>>> The file will only come in to existence 15 seconds before hand. >>>>> >>>>> >>>>> >>>>> Python, on Future price model, not looking so affordable. 100ms per >>>>> request, is 15 requests per “window” 2m/15 is 133k instances. With >>>>> a minimum bill of >>>>> 15 minutes at 5 cents per hour… that’s 33k hours $1,666 per >>>>> occurrence before bandwidth. And assuming everyone lines up in a >>>>> nice line and the scheduler does the right thing. >>>>> >>>>> >>>>> >>>>> This will happen about 3 times a week. (and the $2k price is not >>>>> outside the realm of possibility but I’d like to mark it up and I >>>>> think they are thinking 1/6 that to get the price to $75k after >>>>> mark >>>>> up.) >>>>> >>>>> >>>>> >>>>> By the way Amazon Laughed at me. >>>>> >>>>> When I told Akamai I wanted 95th percentile billing they stopped >>>>> laughing at me. >>>>> >>>>> >>>>> >>>>> What are the theoretical and real limits of Static files. If I >>>>> created a python application that published the file as static will >>>>> Static Scale to 2M requests a second? Or 1.3M would likely do me. >>>>> >>>>> >>>>> >>>>> Would Google kick me off the service for abusing the .12 cents a >>>>> gig in ways they never imagined? >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Google App Engine" group. >>>>> To post to this group, send email to google-appengine@googlegroups.com. >>>>> To unsubscribe from this group, send email to >>>>> google-appengine+unsubscr...@googlegroups.com. >>>>> For more options, visit this group at >>>>> http://groups.google.com/group/google-appengine?hl=en. >>>>> >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups >> "Google App Engine" group. >>> To post to this group, send email to google-appengine@googlegroups.com. >>> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >>> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >>> >>> >> >> -- >> You received this message because you are subscribed to the Google >> Groups "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> >> >> -- >> You received this message because you are subscribed to the Google Groups > "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. >> >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
