Hi Wally, Happy to help (if I did).
> ...you have certainly > covered all the technical bases of implementing a proxy. Oh, I very much doubt it. :-) (BTW, I don't know where that "six" came from in my earlier post. You'll incur 2-3 times, not 2-6 times, as much transfer on requests for dynamic content through the proxy.) > 1. I have experienced being blocked by the app engine (try again in an > hour etc.), so I could reasonably assume that it would be likely that > a lot of traffic coming from one source may be blocked. Perhaps Brett Slatkin or someone else from Google's technical wing could comment on this. I don't know about AppEngine, but Google does place rate limits on end users' use of various apps they provide (such as Google Groups!), and so this is something to be aware of. But I'd be surprised if those rate-limits are naive enough to be confused by requests from a properly-configured proxy. A request from a properly-configured proxy includes the original source of the request as well as the proxy (or proxies) through which it's passed.[1] Proxies are widely used across the web, including by ISPs with hundreds of thousands of end users or more. To lump them all together under one rate limit (or at least under a rate limit intended for individuals) would be inappropriate. [1] http://tools.ietf.org/html/rfc2616#section-14.45 This also applies to your points 2 and 3; the original request's origin is preserved across the proxy (in the normal case; we're not talking about intentionally non-compliant -- but useful! -- proxies such as anonymizers and the like). In any case, the adsense stuff won't go through your proxy, remember that the script comes directly from googlesyndication.com. > Google has also said nothing about the China block, which again means > to expect the worst. Wally, I'm quite certain that any time China blocks the whole of AppEngine (which they don't appear to be doing currently, from other comments), Google is aware of it very quickly and does everything they reasonably can to clear up the problem working through channels with the appropriate Chinese officials. They cannot afford to be closed to China. Now, the degree to which they'll succeed largely depends on the Chinese government. AppEngine is a bit of a problem for them, it's just ridiculously easy to throw together an app that provides a way for Chinese citizens to break through the great firewall and get unfiltered information. I'm not surprised the whole of AppEngine was blocked for a time last year, and I'm not surprised it got unblocked -- presumably the result of discussion and negotiation between the Chinese government, U.S. government, and Google. If Google haven't commented on the situation, FWIW I wouldn't take that as evidence of their not being concerned about and actively engaged in addressing the problem. Public statements can sometimes cause trouble in sensitive negotiations. But hey, not like I'm an expert on international business and government relations. ;-) And I didn't mean to get into the politics; mainly I was trying to address your question about how to go about getting a proxy set up. Good luck, -- T.J. Crowder tj / crowder software / com Independent Software Engineer, consulting services available On Apr 10, 1:08 am, WallyDD <shaneb...@gmail.com> wrote: > Hi TJ, > > That really is an amazing post. I'm impressed, you have certainly > covered all the technical bases of implementing a proxy. > > My biggest concern is that Googles behaviour is unpredictable and I > not entirely sure how well they will respond to something like this > being implemented. > > 1. I have experienced being blocked by the app engine (try again in an > hour etc.), so I could reasonably assume that it would be likely that > a lot of traffic coming from one source may be blocked. > 2. A large portion of the revenue comes from Google adsense/adwords. > Google uses a variety of mechanisms to check for invalid clicks, so > all the clicks coming from one source would no doubt raise some red > flags. > 3. The traffic statistics would be almost useless (there is probably a > workaround... but a lot of work). > 4. Google has deliberately and intentionally blocked traffic > originating from Sudan, Syria, Cuba, Iran and North Korea(not really > sure if they have internet there). From the legal discourse I have > read it would appear google is obligated to block any proxies where > traffic is coming from these countries. I don't really understand this > one as the USA changed their political administration in January 2009 > and the block went in two weeks later. There has to be some politics > behind this which I am unaware of. Google has decided to say nothing > on this subject so I can only assume the worst. > > Google has also said nothing about the China block, which again means > to expect the worst. > I am also far from convinced that Google has figured out China (like a > lot of western companies). From the look of their developer > bloghttp://www.developer.googlechinablog.com/, only 16 people read this as > the RSS feed. > I can't really expect any Chinese to have faith in Google with not > only that their country has blocked, but more importantly that google > itself has actively blocked other countries. > > Google will do what Google wants to do and fail to communicate. I > can't see this strategy doing anything other than annoying the Chinese > further. And back to China I go next week (luckily on unrelated > business). > > And TJ, I like your post, if I can get some (positive) answers I will > be putting in a proxy just as you have outlined. Keep up the great > work. > > On Apr 9, 10:35 am, "T.J. Crowder" <t...@crowdersoftware.com> wrote: > > > Hi Wally, > > > Sorry to hear about the block. > > > > The internet is indeed a funny place. > > > I did respond with a question on how to set this up but have received > > > no answer? > > > > Any ideas anyone? > > > Setting up a proxy server is a non-trivial task (I'm not saying it's > > hard, just non-trivial) so you're not likely to get a lot of dedicated > > help for it here. May be worth seeking out other newsgroups for the > > technical details (if you haven't already!). > > > Most commercial-grade web software such as Apache[1] or nginx[2] can > > be set up to proxy, and there are several dedicated proxy packages as > > well (such as Squid[3]). I've been hearing very good things about > > nginx the last year or so, but have virtually no direct experience > > with it (and not that much experience setting up proxies at all, so > > take all of this with a grain of salt). > > > [1]http://httpd.apache.org/ > > [2]http://nginx.net/ > > [3]http://www.squid-cache.org/ > > > But since you'll need a hosting provider of some sort for the proxy, > > and it sounds as though this is going to be your main reason for > > having that other hosting service, it may be worth considering > > approaching hosting providers who will set up and maintain the proxy > > for you, rather than doing it yourself. I searched for "proxy > > hosting" and there's a whole industry out there you can tap into. It > > depends on whether this is something you want to add to your set of > > skills. Naturally, you'll want to be sure that the proxy hosting > > company itself isn't blocked in China! Given what they do, I suspect > > a fair number of them are, but the censors can't keep on top of all of > > them, and you can switch as necessary (the joys of proxying!). > > > A downside of the proxy approach is that you'll end up paying anywhere > > from twice to six times as much for at least some of your site's > > traffic -- the parts that can't be cached. Say you host the proxy at > > Acme Hosting Company. Where before your traffic costs on a request > > for dynamic content were: > > > * Inbound cost at AppEngine (receiving request from end user's > > browser) > > * Outbound cost at AppEngine (sending reply to end user's browser) > > > with a proxy you'll be paying: > > > * Inbound cost at Acme (receiving request from end user's browser) > > * Outbound cost at Acme (sending request to AppEngine) > > * Inbound cost at AppEngine (receiving request from proxy) > > * Outbound cost at AppEngine (sending reply to proxy) > > * Inbound cost at Acme (receiving reply from AppEngine) > > * Outbound cost at Acme (sending reply to end user's browser) > > > So you'll need to shop around with that in mind. Again, that's only > > the dynamic content; if the proxy can satisfy the request from cache, > > it will, and so you wouldn't end up paying AppEngine transfer costs > > (or CPU time costs) on that particular request at all. > > > Some suggestions related to that: > > > * Provide a transparent redirect mechanism or some such for users who > > can go direct, so avoid putting unnecessary load and throughput on the > > proxy. > > > * Be sure that your site's content is as cacheable as possible (but > > this is always a good idea). The more cacheable your site, the faster > > it seems to be, because there's a fair bit of caching that goes on out > > in the cloud if you let it; caching doesn't only happen at the end > > user's browser. > > > * Make sure all of the links in the chain are using compression (gzip, > > etc.) whenever possible. > > > Wow, longer post than I intended. Anyway, FWIW, and good luck, > > -- > > T.J. Crowder > > tj / crowder software / com > > Independent Software Engineer, consulting services available > > > On Apr 6, 5:35 pm, WallyDD <shaneb...@gmail.com> wrote: > > > > The internet is indeed a funny place. > > > I did respond with a question on how to set this up but have received > > > no answer? > > > > Any ideas anyone? > > > > On Apr 6, 3:03 am, Paddy Foran <foran.pa...@gmail.com> wrote: > > > > > I'd just like to point out how funny it is that people keep banging on > > > > for Google to respond, and in their banging on for Google to respond, > > > > they missed Google's actual response. > > > > > >> Is there any google staff who is responsible for GAE promotion and > > > > >> technology to say something here? > > > > > >> How can I access to my Google Apps via my own domain directly, e.g. > > > > >> how can access via mail.my_domain.com instead of mail.google.com/a/ > > > > >> my_domain.com? > > > > > >One way to address this is to run a proxy server elsewhere, which will > > > > >allow your site to have it's own unique IP, rather than the shared IPs > > > > >of Google. > > > > > >-Brett > > > > >App Engine Team > > > > > Please note the "App Engine Team" signature. That means Brett (at > > > > least claims he) is from Google. > > > > > Poor Brett was ignored, as people clamoured for Brett to comment. > > > > > This is why I love the internet. It amuses me to no end. > > > > > On Apr 6, 12:48 am, Andy Freeman <ana...@earthlink.net> wrote: > > > > > > > No company is willing to be a pawn in the game of politics between > > > > > > Google and China. > > > > > > That sounds reasonable, but what can Google do to stop the Chinese > > > > > govt from blocking? > > > > > > (1) Google can't tell the Chinese govt what to do. > > > > > > (2) The Chinese govt appears to be technically competent and controls > > > > > the relevant connections, both from the outside and from internal > > > > > datacenters. > > > > > > (3) Google can propose agreements, but China is a soverign entity and > > > > > and can do what it pleases wrt internal matters. (Other posters have > > > > > suggested that buying dinner for the appropriate official would cause > > > > > the blocking to go away. I don't see why the Chinese govt would find > > > > > such an agreement binding.) > > > > > > Yes, one can argue that Google "needs" the Chinese govt to not block, > > > > > but that doesn't imply that Google can do anything to stop the Chinese > > > > > govt from blocking. Google's needs do not obligate the Chinese govt. > > > > > > On Apr 5, 3:16 pm, WallyDD <shaneb...@gmail.com> wrote: > > > > > > > Google is more or less obligated to solve this issue. > > > > > > > No company is willing to be a pawn in the game of politics between > > > > > > Google and China. > > > > > > Name a single company (that has any international presence) who > > > > > > would > > > > > > be willing to use GAE knowing full well that it is blocked in its > > > > > > current form? > > > > > > This issue has nothing to do with the Chinese government and there > > > > > > is > > > > > > no way Google will point the finger at them. > > > > > > > Perhaps google can also take on all the other countries that are > > > > > > blocking GAE and while they are at it they can point fingers at > > > > > > corporate america and their firewalls? > > > > > > You have to remember that at the moment this is a "preview release". > > > > > > > I don't really understand why you persist with this argument. You > > > > > > have > > > > > > raised some valid points which should be looked at and considered in > > > > > > the scheme of things but most of the diatribe you present here seems > > > > > > aimed at China/Chinese Government. I have always found prejudices > > > > > > cloud peoples judgement. > > > > > > > To sumarise how this problem will probably be viewed; > > > > > > Google created a dns based system (for GAE addressing) which puts > > > > > > everything though ghs.google.com. This system works really well and > > > > > > from my experience it was very clever > > ... > > read more » --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
