[google-appengine] How can I re-enable my application
I disabled an application and cannot re-enable it. The reason I disabled it was that I am seeing old versions of my sire after deploying HRD. Now even though I go to settings and click on re-enable, it stays disabled. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/4f1f6640-243a-477d-bcac-bade278cd8af%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] re-enabling an application
Thank you Alex. I had tried that earlier. But although when I I change the name in app.yaml it allows me to upload without errors, when I try to view the site at http://aussieclouds-hrd.appspot.com/ I get a 500 server error. I think it is because I disabled aussieclouds. On Wednesday, March 18, 2015 at 7:42:40 PM UTC-4, Alex Martelli wrote: As I just answered you in StackOverflow, I suspect the reason you can't upload new versions of your app is that in your `application:` line in app.yaml it still has the old app id (the version upgraded to HRD was given a new app id when you made it -- the old app id was aliased to access the new one). If that's the case you can fix app.yaml and upload the app but you'll need to access it only as newappid.appspot.com since you've disabled the oldappid.appspot.com that (before the disable) used to be aliased to it. Alex On Wed, Mar 18, 2015 at 4:04 PM, Kate mss@gmail.com javascript: wrote: I disabled an application and cannot re-enable it. The reason I disabled it was that I am seeing old versions of my sire after deploying HRD. Now even though I go to settings and click on re-enable, it stays disabled. I am stumped. I upgraded to HRD and now cannot upload with appcfg.py. It either fivers the error that it needs python 2.7 for HRD, and if I use python 2.7 it says I need HRD. I have HRD. I have python27. I thought perhaps I needed to delete the source application, but disabling it has only made things worse. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com javascript:. To post to this group, send email to google-a...@googlegroups.com javascript:. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/0ef35c33-3907-4b87-a8a2-21af475bb0bc%40googlegroups.com https://groups.google.com/d/msgid/google-appengine/0ef35c33-3907-4b87-a8a2-21af475bb0bc%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/40cb333b-7d59-43b0-a428-4ba324ebfb26%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[google-appengine] re-enabling an application
I disabled an application and cannot re-enable it. The reason I disabled it was that I am seeing old versions of my sire after deploying HRD. Now even though I go to settings and click on re-enable, it stays disabled. I am stumped. I upgraded to HRD and now cannot upload with appcfg.py. It either fivers the error that it needs python 2.7 for HRD, and if I use python 2.7 it says I need HRD. I have HRD. I have python27. I thought perhaps I needed to delete the source application, but disabling it has only made things worse. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/0ef35c33-3907-4b87-a8a2-21af475bb0bc%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[google-appengine] How do I re-enable an application?
I disabled an application and cannot re-enable it. The reason I disabled it was that I am seeing old versions of my sire after deploying HRD. Now even though I go to settings and click on re-enable, it stays disabled. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/17019d44-4011-4156-ba80-dc208f845344%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Downloading my app The request is invalid for an unspecified reason
I get an error when downloading my app. Can anyone give me any pointers. Here is my code with the response. C:\google_appengineappcfg.py download_app -A aussieclouds.appspot.com -V 1 wwwhome\aussieclouds Host: appengine.google.com Fetching file list... Error 400: --- begin server output --- Client Error (400) The request is invalid for an unspecified reason. --- end server output --- -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Ydwml6M_GVIJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: How to blacklist planet-lab.org?
@David, No they weren't carted off in a divvy van. Not did I get any tinnies - not that there's any Melbourne Bitter here. But to update you, there were a number of emails back and forth between myself and Planetlab (Princeton) and quite frankly I was concerned with the attitude of entitlement from the researchers. I did get an Im sorry and some excuses, but I am surprised there was not a true regret conveyed, given that the behavior of the researchers was clearly against Planetlab's published Acceptable Use Policy, and that I lost money, and both myself and a number of members of the GAE community spent time trying to find a solution. I intend to take the matter higher up the food chain, as I believe that it is morally wrong to single out a site to test GAE without asking permission. Why didn't they set up their own site? On Monday, August 13, 2012 9:35:31 AM UTC-4, Cesium wrote: G'day Kate, Did the hoons end up in the divvy van? I'd be gobsmacked if you didn't leave them tall poppies all stonkered. At least demand a few tinnies of amber fluid. David PS Just testing my new RESTful API: convertposttoaussiespeak.appspot.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Eu0gxxQ4Ej0J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Our application is suffering from 4 different kinds of 500 errors, plus memory spikes. Are we the 1.7.1 guinea pig?
Has there been any change in your traffic? On Tuesday, August 14, 2012 7:47:40 AM UTC-4, Per wrote: Hi, Our application all of a sudden throws errors that I've not seen before. While we continuously deploy new minor features, we've not made any substantial changes that would explain either of these: * Datastore access misbehaving: Nested in javax.servlet.ServletException: com.google.apphosting.api.ApiProxy$ArgumentException: An error occurred parsing (locally or remotely) the arguments to DATASTORE_V3.Get().: com.google.apphosting.api.ApiProxy$ArgumentException: An error occurred parsing (locally or remotely) the arguments to DATASTORE_V3.Get(). at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.success(ApiProxyImpl.java:452) at com.google.apphosting.runtime.ApiProxyImpl$AsyncApiFuture.success(ApiProxyImpl.java:382) at com.google.net.rpc3.client.RpcStub$RpcCallbackDispatcher$1.runInContext(RpcStub.java:782) * Got out of Memory errors while saving the session (outside our control) java.lang.OutOfMemoryError at java.util.zip.Deflater.init(Native Method) at java.util.zip.Deflater.init(Deflater.java:177) at java.util.zip.GZIPOutputStream.init(GZIPOutputStream.java:63) at java.util.zip.GZIPOutputStream.init(GZIPOutputStream.java:75) at com.google.apphosting.runtime.HttpCompression.compress(HttpCompression.java:51) at com.google.apphosting.runtime.HttpCompression.attemptCompression(HttpCompression.java:205) at com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:482) at com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:449) at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:455) at com.google.tracing.TraceContext.runInContext(TraceContext.java:695) at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:333) at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:325) at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:453) at com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:251) at java.lang.Thread.run(Thread.java:679) * com.google.appengine.api.memcache.MemcacheServiceException: Memcache put: Error setting single item (_ahsahRx7dGRBNmJh6XPmzlfTg) -- This one is thrown so deep inside the bowels of Jetty that it's outside our application's reach, and thus a memcache error takes down our application once again: http://code.google.com/p/googleappengine/issues/detail?id=6463 * URL fetch throws errors we've not seen before: Caused by: com.google.apphosting.api.ApiProxy$ArgumentException: An error occurred parsing (locally or remotely) the arguments to URLFETCH.Fetch(). Also, memory is all of a sudden rising a lot, every now and then. Resetting the instance with the most instances helps for a while, but the problems resume a few hours later. I'd be tempted to think that the memory issues can cause some of the above errors (but not all). It's been a while since 1.7.0, so I wouldn't be surprised of this is related to internal 1.7.1 testing. If this is the case, please stop and move us back to a regular 1.7.0 environment. Cheers, Per PS: I've raised production issues, but since I have never heard back from a single one *ever*, I'm posting to this forum to state how unhappy I am. It seems to be the only way to get attention (yeah, I've tried getting premium support, but it's not available in Germany...) -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/2smGHP_wkysJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: How to blacklist planet-lab.org?
I know I can have CF manage dns for domains that are DIRECTED to GAE. However the attacks were going straight to aussiecloud.appspot.com. CF cannot manage that! I have a very good understanding of how DNS works. In any case the culprits were found and have stopped. Always best to fix at the source rather than add another layer, if possible. On Monday, August 13, 2012 2:15:16 AM UTC-4, Jeff Schnitzer wrote: On Fri, Aug 10, 2012 at 2:41 PM, Kate mss@gmail.com javascript: wrote: I explained earlier that cloudflare cannot help as they need me to change my dns and I cannot as it is a gae app. My netsol dns for australiansabroad.com is set up according to google specs and are not accepted by cloudflare.. You have some sort of mistaken understanding about how DNS works. There is no reason why you cannot have CF manage DNS for domains that are directed to GAE. I do this even for domains that I don't proxy through cloudflare just because their DNS tools don't suck. Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/GczMGNfW12QJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] How can I block curl requests
Thanks!@ I have used up my 100 entries and haven't got all of them! Haven't heard back from planet-lab.org! Will keep you informed! On Friday, August 10, 2012 4:41:06 AM UTC-4, alex wrote: - subnet: 134.117.0.0/16 description: Carleton University - subnet: 147.228.0.0/14 description: Brno University of Technology - subnet: 206.23.0.0/16 description: Tennessee Board of Regents - subnet: 160.193.0.0/16 description: Osaka City University - subnet: 138.246.0.0/16 description: Ludwig-Maximilians-Universitaet Muenchen - subnet: 129.242.0.0/16 description: University of Tromso - subnet: 206.12.0.0/16 description: BCnet Vancouver - subnet: 152.3.0.0/16 description: Duke University - subnet: 195.37.0.0/16 description: Extranet der Universitaet Passau On Friday, August 10, 2012 4:10:24 AM UTC+2, Kate wrote: Done, Actually I have some more. Can you subnet any of these? all planet-lab ones - subnet: 134.117.226.181 - subnet: 147.229.10.250 - subnet: 206.23.240.29 - subnet: 160.193.163.106 - subnet: 138.246.99.249 - subnet: 129.242.19.197 - subnet: 206.12.16.154 - subnet: 160.193.163.106 - subnet: 152.3.138.6 - subnet: 195.37.16.121 I am up to 100 now because of these individual ones. On Thursday, August 9, 2012 5:37:41 PM UTC-4, alex wrote: Actually, scratch last three lines (starting from # these belong to...) and replace with - subnet: 141.0.0.0/8 On Thursday, August 9, 2012 11:32:03 PM UTC+2, alex wrote: A lot of those IPs are assigned to universities, like almost literally all of them. More than 50% are US universities. This really looks like a big distributed bot network to me. Anyway, here you go (some IPs are from the same net block so there are less than 100 entries): - subnet: 82.179.176.0/20 - subnet: 83.230.96.0/19 - subnet: 88.2.0.0/16 - subnet: 128.10.0.0/16 - subnet: 128.84.0.0/16 - subnet: 128.42.0.0/16 - subnet: 128.36.0.0/16 - subnet: 128.111.0.0/16 - subnet: 128.114.0.0/16 - subnet: 128.138.0.0/16 - subnet: 128.227.0.0/16 - subnet: 128.208.0.0/16 - subnet: 128.151.0.0/16 - subnet: 128.223.0.0/16 - subnet: 128.232.0.0/16 - subnet: 129.10.0.0/16 - subnet: 129.15.0.0/16 - subnet: 129.74.0.0/16 - subnet: 129.82.0.0/16 - subnet: 129.93.0.0/16 - subnet: 129.97.0.0/16 - subnet: 129.108.0.0/16 - subnet: 129.130.0.0/16 - subnet: 129.237.0.0/16 - subnet: 130.37.0.0/16 - subnet: 130.83.0.0/16 - subnet: 130.104.0.0/16 - subnet: 130.216.0.0/16 - subnet: 130.253.0.0/16 - subnet: 130.195.4.0/24 - subnet: 130.237.0.0/18 - subnet: 131.179.0.0/16 - subnet: 131.188.0.0/16 - subnet: 132.64.0.0/13 - subnet: 132.72.0.0/14 - subnet: 134.151.0.0/16 - subnet: 138.4.0.0/16 - subnet: 138.250.0.0/15 - subnet: 138.48.0.0/16 - subnet: 139.165.0.0/16 - subnet: 140.109.0.0/16 - subnet: 143.225.0.0/16 - subnet: 155.245.0.0/16 - subnet: 160.80.0.0/16 - subnet: 161.106.0.0/16 - subnet: 192.16.124.0/22 - subnet: 192.38.0.0/17 - subnet: 192.41.132.0/22 - subnet: 192.42.42.0/23 - subnet: 193.1.0.0/16 - subnet: 193.138.2.0/24 - subnet: 193.196.0.0/15 - subnet: 193.166.0.0/15 - subnet: 193.136.0.0/15 - subnet: 193.166.0.0/15 - subnet: 193.204.0.0/15 - subnet: 193.226.0.0/19 - subnet: 194.29.176.0/22 - subnet: 194.167.0.0/16 - subnet: 194.254.0.0/16 - subnet: 195.130.124.0/22 - subnet: 203.110.240.0/24 - subnet: 212.51.208.0/20 - subnet: 213.73.32.0/19 - subnet: 213.131.0.0/19 - subnet: 136.159.0.0/16 - subnet: 132.170.0.0/16 - subnet: 132.181.0.0/16 - subnet: 133.0.0.0/8 - subnet: 203.0.0.0/8 - subnet: 198.82.0.0/16 - subnet: 200.0.0.0/8 - subnet: 192.107.171.0/24 - subnet: 140.112.0.0/12 - subnet: 140.123.0.0/16 - subnet: 142.103.0.0/16 - subnet: 143.89.0.0/16 - subnet: 139.78.0.0/16 - subnet: 155.246.0.0/16 - subnet: 156.56.0.0/16 - subnet: 156.62.0.0/16 - subnet: 157.92.0.0/16 - subnet: 169.226.0.0/16 - subnet: 165.91.0.0/16 - subnet: 165.230.0.0/16 - subnet: 192.1.0.0/16 - subnet: 192.12.33.0/24 - subnet: 169.229.0.0/16 - subnet: 141.219.0.0/16 # these belong to a too big block - subnet: 141.11.0.162 - subnet: 141.20.103.211 On Thursday, August 9, 2012 10:21:03 PM UTC+2, Kate wrote: I did that but don't think it will catch them all. I had a list of 100 individual ones, see below. Can you recognize subnets in them? blacklist: - subnet: 82.179.176.44 - subnet: 83.230.127.124 - subnet: 88.2.234.60 - subnet: 128.10.19.53 - subnet: 128.84.154.45 - subnet: 128.42.142.44 - subnet: 128.36.233.154 - subnet: 128.111.52.59 - subnet: 128.114.63.64 - subnet: 128.138.207.45 - subnet: 128.227.150.12 - subnet: 128.208.4.198 - subnet: 128.151.65.101 - subnet: 128.223.8.112 - subnet: 128.232.103.201 - subnet: 129.10.120.194 - subnet: 129.15.78.30 - subnet: 129.74.74.20 - subnet: 129.82.12.188 - subnet: 129.93.229.139 - subnet: 129.97.74.14 - subnet: 129.108.202.11 - subnet: 129.130.252.140 - subnet: 129.237.161.194 - subnet: 130.37.193.143
[google-appengine] Planet-lab.org bringing down my site - dodos attack
I have posted on a couple of other threads re curl requests coming from hundreds of different ip addresses and taking up my resources on gae to the extent that my site is down several hours a day. I have blocked as many of http://www.planet-lab.org/'s ips that I can but I can only have 100 entries in te blacklist and this isn't enough. People here have been helpful in identifying subnets but thousands of requests are still getting through. I have emailed planet-lab but would like to phone them.i suspect it's just a bunch of students making mistakes but there must be some action I can take as I can't keep paying for these curl requests and will have to close down permanently soon. Any help appreciated. Btw the ips are from within the US, Europe and Asia. Kate -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/f8hAr4ErNRoJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Planet-lab.org bringing down my site - ddos attack
I have posted about my site being bombarded by curl requests from hundreds of ip addresses in some other threads here and received a lot of help. It seems that most of the offenders are affiliated with planet-lab and possibly the attacks are mistakes. However as a result my site is being brought down for several hourse per day. I have contacted planet-lab by email with no luck, I cannot have more that100 subnet entries in my block file on gae, and short of paying more $$$ to google to support these unwanted curl requests I. Can think of nothing else to do. Can anyone think of how I can get planet-lab to stop? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/jjVHY8Iz0KUJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] How to blacklist planet-lab.org?
I have posted about my site being bombarded by curl requests from hundreds of ip addresses in some other threads here and received a lot of help. It seems that most of the offenders are affiliated with planet-lab and possibly the attacks are mistakes. However as a result my site is being brought down for several hourse per day. I have contacted planet-lab by email with no luck, I cannot have more that100 subnet entries in my block file on gae, and short of paying more $$$ to google to support these unwanted curl requests I. Can think of nothing else to do. Can anyone think of how I can get planet-lab to stop? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/1Tdo9PCo2uEJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Planet-lab.org bringing down my site - ddos attack
I have posted about my site being bombarded by curl requests from hundreds of ip addresses in some other threads here and received a lot of help. It seems that most of the offenders are affiliated with planet-lab and possibly the attacks are mistakes. However as a result my site is being brought down for several hourse per day. I have contacted planet-lab by email with no luck, I cannot have more that100 subnet entries in my block file on gae, and short of paying more $$$ to google to support these unwanted curl requests I. Can think of nothing else to do. Can anyone think of how I can get planet-lab to stop? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Et7YzHkkLrsJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: How to blacklist planet-lab.org?
I do not need multiple threads. My last three were in error as I was having problems with my iPad and didn't think they were posted. My apologies. I explained earlier that cloudflare cannot help as they need me to change my dns and I cannot as it is a gae app. My netsol dns for australiansabroad.com is set up according to google specs and are not accepted by cloudflare.. The correct course of action is to have planet-lab stop sending tens of thousand of curl requests. My site has been running since 1996 and I should not have to spend hours figuring out how to block attacks. If planet-lab will not stop the attacks I will take the course suggested by other posters and report them to the appropriate authorities, -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/7bFtnBjjihgJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: How to blacklist planet-lab.org?
I received an answer from planet-lab. Hi Kathleen, Apologies for not resolving this sooner. We believe we found a likely source behind the traffic you've been referring to. As you may know, PlanetLab is a distributed systems research test bed with 1000+ machines world wide. These machines may share access to both research, local and public Internet. These services are actively managed by researchers granted access to PlanetLab accounts. Since your site is hosted by Google, the IP addresses that you use are not unique to you, but are shared among many Google hosted services. Many experiments on PlanetLab nodes sent significant volume of legitimate traffic to these IP addresses and finding the subset of this traffic that corresponds to your service is a bit more involved. We have however identified a likely experiment that is responsible, it is ucr_web slice, run by researchers at University of California, Riverside, who are cc'ed on this email. The researchers provide a description of their work as: This slice is being used to perform measurements to detect outages on paths on which traffic is served from PlanetLab. We passively observe traffic outgoing from PlanetLab to see which prefixes are receiving TCP traffic from PlanetLab, and then use a combination of passive monitoring and active probing to detect outages on paths to these prefixes.brWe ensure that we only send active probes to prefixes that receive traffic from PlanetLab, and we probe every prefix at most once every 5 minutes if the prefix is reachable and at most three times in a 5 minute period if we do not receive responses to our probes.' On Friday, August 10, 2012 5:41:36 PM UTC-4, Kate wrote: I do not need multiple threads. My last three were in error as I was having problems with my iPad and didn't think they were posted. My apologies. I explained earlier that cloudflare cannot help as they need me to change my dns and I cannot as it is a gae app. My netsol dns for australiansabroad.com is set up according to google specs and are not accepted by cloudflare.. The correct course of action is to have planet-lab stop sending tens of thousand of curl requests. My site has been running since 1996 and I should not have to spend hours figuring out how to block attacks. If planet-lab will not stop the attacks I will take the course suggested by other posters and report them to the appropriate authorities, -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/GqLd2DT2fZwJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] How can I block curl requests
Heard back from planet-lab: See below. However I don't buy the We ensure that we only send active probes to prefixes that receive traffic from PlanetLab, and we probe every prefix at most once every 5 minutes if the prefix is reachable and at most three times in a 5 minute period if we do not receive responses to our probes. Hi Kathleen, Apologies for not resolving this sooner. We believe we found a likely source behind the traffic you've been referring to. As you may know, PlanetLab is a distributed systems research test bed with 1000+ machines world wide. These machines may share access to both research, local and public Internet. These services are actively managed by researchers granted access to PlanetLab accounts. Since your site is hosted by Google, the IP addresses that you use are not unique to you, but are shared among many Google hosted services. Many experiments on PlanetLab nodes sent significant volume of legitimate traffic to these IP addresses and finding the subset of this traffic that corresponds to your service is a bit more involved. We have however identified a likely experiment that is responsible, it is ucr_web slice, run by researchers at University of California, Riverside, who are cc'ed on this email. The researchers provide a description of their work as: This slice is being used to perform measurements to detect outages on paths on which traffic is served from PlanetLab. We passively observe traffic outgoing from PlanetLab to see which prefixes are receiving TCP traffic from PlanetLab, and then use a combination of passive monitoring and active probing to detect outages on paths to these prefixes.brWe ensure that we only send active probes to prefixes that receive traffic from PlanetLab, and we probe every prefix at most once every 5 minutes if the prefix is reachable and at most three times in a 5 minute period if we do not receive responses to our probes. On Friday, August 10, 2012 1:37:33 PM UTC-4, Kate wrote: Thanks!@ I have used up my 100 entries and haven't got all of them! Haven't heard back from planet-lab.org! Will keep you informed! On Friday, August 10, 2012 4:41:06 AM UTC-4, alex wrote: - subnet: 134.117.0.0/16 description: Carleton University - subnet: 147.228.0.0/14 description: Brno University of Technology - subnet: 206.23.0.0/16 description: Tennessee Board of Regents - subnet: 160.193.0.0/16 description: Osaka City University - subnet: 138.246.0.0/16 description: Ludwig-Maximilians-Universitaet Muenchen - subnet: 129.242.0.0/16 description: University of Tromso - subnet: 206.12.0.0/16 description: BCnet Vancouver - subnet: 152.3.0.0/16 description: Duke University - subnet: 195.37.0.0/16 description: Extranet der Universitaet Passau On Friday, August 10, 2012 4:10:24 AM UTC+2, Kate wrote: Done, Actually I have some more. Can you subnet any of these? all planet-lab ones - subnet: 134.117.226.181 - subnet: 147.229.10.250 - subnet: 206.23.240.29 - subnet: 160.193.163.106 - subnet: 138.246.99.249 - subnet: 129.242.19.197 - subnet: 206.12.16.154 - subnet: 160.193.163.106 - subnet: 152.3.138.6 - subnet: 195.37.16.121 I am up to 100 now because of these individual ones. On Thursday, August 9, 2012 5:37:41 PM UTC-4, alex wrote: Actually, scratch last three lines (starting from # these belong to...) and replace with - subnet: 141.0.0.0/8 On Thursday, August 9, 2012 11:32:03 PM UTC+2, alex wrote: A lot of those IPs are assigned to universities, like almost literally all of them. More than 50% are US universities. This really looks like a big distributed bot network to me. Anyway, here you go (some IPs are from the same net block so there are less than 100 entries): - subnet: 82.179.176.0/20 - subnet: 83.230.96.0/19 - subnet: 88.2.0.0/16 - subnet: 128.10.0.0/16 - subnet: 128.84.0.0/16 - subnet: 128.42.0.0/16 - subnet: 128.36.0.0/16 - subnet: 128.111.0.0/16 - subnet: 128.114.0.0/16 - subnet: 128.138.0.0/16 - subnet: 128.227.0.0/16 - subnet: 128.208.0.0/16 - subnet: 128.151.0.0/16 - subnet: 128.223.0.0/16 - subnet: 128.232.0.0/16 - subnet: 129.10.0.0/16 - subnet: 129.15.0.0/16 - subnet: 129.74.0.0/16 - subnet: 129.82.0.0/16 - subnet: 129.93.0.0/16 - subnet: 129.97.0.0/16 - subnet: 129.108.0.0/16 - subnet: 129.130.0.0/16 - subnet: 129.237.0.0/16 - subnet: 130.37.0.0/16 - subnet: 130.83.0.0/16 - subnet: 130.104.0.0/16 - subnet: 130.216.0.0/16 - subnet: 130.253.0.0/16 - subnet: 130.195.4.0/24 - subnet: 130.237.0.0/18 - subnet: 131.179.0.0/16 - subnet: 131.188.0.0/16 - subnet: 132.64.0.0/13 - subnet: 132.72.0.0/14 - subnet: 134.151.0.0/16 - subnet: 138.4.0.0/16 - subnet: 138.250.0.0/15 - subnet: 138.48.0.0/16 - subnet: 139.165.0.0/16 - subnet: 140.109.0.0/16 - subnet: 143.225.0.0/16 - subnet: 155.245.0.0/16 - subnet: 160.80.0.0/16 - subnet: 161.106.0.0/16 - subnet: 192.16.124.0/22 - subnet: 192.38.0.0/17 - subnet: 192.41.132.0
[google-appengine] Re: Low hitting site experiencing denial of service attack - site opffline!
I made a cloudflare account but it can't resolve my domain name, www.australiansabroad.com as I don't have a dns entry at network solutions.com where I register my sites. I have a special entry that google resolves. And if I put in my appspot site name cloudflare says it cannot accept that. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/UfV1hfuhhs8J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Low hitting site experiencing denial of service attack - site opffline!
I will have a go and then I will have to give up on google apps. I am very surprised they allow dos attacks. No web provider has allowed this to happen before. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/0RfQ_4DZg3oJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] How can I block curl requests
Hi Sergey, Here is a typical example 2012-08-09 06:51:16.597 / 302 30ms 0kb curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3 libidn/0.6.14 libssh2/0.18 202.125.215.12 - - [09/Aug/2012:04:51:16 -0700] HEAD / HTTP/1.1 302 153 - curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3 libidn/0.6.14 libssh2/0.18 aussieclouds.appspot.com ms=31 cpu_ms=0 api_cpu_ms=0 cpm_usd=0.49 instance=00c61b117c2f994812ed63184c9c5544dea738 But the ip address varies. My code forces 302 response. Before I added the code they were throwing errors head method not found. But even though I am doing the 303 I am still getting front end time exceeded and these requests are taking up about 95% of my quota. So to keep the site alive I would have to pay for them, I have lost most of my European and Australian visitors because the site is down every night during those places daylight hours. Obviously I can't continue like this and so will have to move to a provider capable of blocking these requests, -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/tC4bapnRFMAJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] How can I block curl requests
Hi Sergey, Here is a typical example 2012-08-09 06:51:16.597 / 302 30ms 0kb curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3 libidn/0.6.14 libssh2/0.18 202.125.215.12 - - [09/Aug/2012:04:51:16 -0700] HEAD / HTTP/1.1 302 153 - curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3 libidn/0.6.14 libssh2/0.18 aussieclouds.appspot.com ms=31 cpu_ms=0 api_cpu_ms=0 cpm_usd=0.49 instance=00c61b117c2f994812ed63184c9c5544dea738 But the ip address varies. My code forces 302 response. Before I added the code they were throwing errors head method not found. But even though I am doing the 303 I am still getting front end time exceeded and these requests are taking up about 95% of my quota. So to keep the site alive I would have to pay for them, I have lost most of my European and Australian visitors because the site is down every night during those places daylight hours. Obviously I can't continue like this and so will have to move to a provider capable of blocking these requests, -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/MHFuAqELNlEJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] How can I block curl requests
No cloudfare requires ands info which I do not have. I made a cloudflare account but it can't resolve my domain name, www.australiansabroad.com as I don't have a dns entry at network solutions.com where I register my sites. I have a special entry that google resolves. And if I put in my appspot site name cloudflare says it cannot accept that. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/_O6arbPIFb8J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Low hitting site experiencing denial of service attack - site opffline!
I tried sending back a 302 when I get a head request and that didn't help. All the ips asre different and not all come from planet lab. But I will contact planet lab. You would think they would stop however anyway as my site is down about 1/8th of the day now. I will try the suggestions to date and if that doesn't work will seek out a new provider or just rename the site and get a new ip. I don't think it is planet lab looping as why would they keep changing the ip address? I googled the problem and most people suggest getting the provider to help but google isn't interested as they have nothing to gain. I can ask new providers their policy before inning up. I did find another person on GAE with an identical problem but apparently he gave up. Today I am already in to 20% of my quota and will soon lose. My only advertiser. I have already lost one, and my Adsense revenue is right down because of all the downtime. I also use other google services not related to this site being attacked so if no resolution I will vote with my feet. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/OSxYXB57MkkJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Low hitting site experiencing denial of service attack - site opffline!
I tried sending back a 302 when I get a head request and that didn't help. All the ips asre different and not all come from planet lab. But I will contact planet lab. You would think they would stop however anyway as my site is down about 1/8th of the day now. I will try the suggestions to date and if that doesn't work will seek out a new provider or just rename the site and get a new ip. I don't think it is planet lab looping as why would they keep changing the ip address? I googled the problem and most people suggest getting the provider to help but google isn't interested as they have nothing to gain. I can ask new providers their policy before inning up. I did find another person on GAE with an identical problem but apparently he gave up. Today I am already in to 20% of my quota and will soon lose. My only advertiser. I have already lost one, and my Adsense revenue is right down because of all the downtime. I also use other google services not related to this site being attacked so if no resolution I will vote with my feet. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/2Bk3u1tVsmsJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Low hitting site experiencing denial of service attack - site opffline!
I made a cloudflare account but it can't resolve my domain name, www.australiansabroad.com as I don't have a dns entry at network solutions.com where I register my sites. I have a special entry that google resolves. And if I put in my appspot site name cloudflare says it cannot accept that. On Thursday, August 9, 2012 6:38:16 AM UTC-4, Ernesto Oltra wrote: Have you tried CloudFlare? It's basically the same, a proxy to your app, and you don't have to make/admin it. It has a reputation system, and some others pretty good settings to deal with DOS attacks. BTW, I'm not affiliated at all with the company, I'm only other web dev. On Thursday, August 9, 2012 5:12:17 AM UTC+2, Kate wrote: My site has been on GAE for several years but am now experiencing denial of service attacks. I don't want to pay for them so my site is now down. I put in a production issue last week and have tried blocking the user agent (curl). But I can't block by IP as all IPs are different. Apart from moving from GAE what can I do. Thanks in advance, Kate -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Ayv0M-GUEmIJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] How can I block curl requests
They are not coming from the same IP. They are mostly in Europe but there are no subnets. There are hundreds of them and google only lets you block 100. eg 132.65.240.100 133.15.59.2 193.136.19.13 139.165.12.211 193.166.167.5 141.219.252.133 200.17.202.195 195.130.124.1 193.1.201.27 138.48.3.202 136.159.220.40 138.251.214.78 all these and more within a minute. They are all different. Kate On Thursday, August 9, 2012 9:55:29 AM UTC-4, alex wrote: Kate, If barryhunter is right and all the IPs are coming from the same ISP anyway, you can simply block the whole subnetwork ranges of that ISP (at least temporary) using dos.yaml: https://developers.google.com/appengine/docs/python/config/dos It'll be pain in the ass updating the file every time you encounter new subnets but at least you could probably save some quota 'till you move somewhere else or figure something out. -- alex On Thursday, August 9, 2012 1:59:57 PM UTC+2, Kate wrote: Hi Sergey, Here is a typical example 2012-08-09 06:51:16.597 / 302 30ms 0kb curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3 libidn/0.6.14 libssh2/0.18 202.125.215.12 - - [09/Aug/2012:04:51:16 -0700] HEAD / HTTP/1.1 302 153 - curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0zlib/1.2.3 libidn/0.6.14 libssh2/0.18 aussieclouds.appspot.com ms=31 cpu_ms=0 api_cpu_ms=0 cpm_usd=0.49 instance=00c61b117c2f994812ed63184c9c5544dea738 But the ip address varies. My code forces 302 response. Before I added the code they were throwing errors head method not found. But even though I am doing the 303 I am still getting front end time exceeded and these requests are taking up about 95% of my quota. So to keep the site alive I would have to pay for them, I have lost most of my European and Australian visitors because the site is down every night during those places daylight hours. Obviously I can't continue like this and so will have to move to a provider capable of blocking these requests, -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/g_yrLQPI49cJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Low hitting site experiencing denial of service attack - site opffline!
My site is in Python. I am afraid the attackers have won. It seems nothing can be done and we on GAE have to put up with it. Or pay. I will increase my quota while I move my site and cop it sweet as they say in Australia. Thanks for everyone who tried to help. Kate On Thursday, August 9, 2012 2:13:01 AM UTC-4, Kristopher Giesing wrote: There's a short example of a node.js proxy server here: http://www.catonmat.net/http-proxy-in-nodejs/ It might not be as hard as you imagine to get this working. I'd hate to see the attackers win :( - Kris On Wednesday, August 8, 2012 10:06:45 PM UTC-7, Kate wrote: I don't know how to write the node.js. I am surprised that google cannot help me here. I do appreciate your help but I just can't do this without spending hours of study. To answer your other question, the blocking attempt didn't solve the problem. The curl requests come about 3 times per second. I have been working on trying to stop them for over a week now. I suppose I can just change my domain name or move to another hosting service but it seems a pity. I have had this site since 1996 with never a problem. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/5y6f_3Sj3ooJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] How can I block curl requests
Thanks, I emailed support. Tried to look up their steering committee but their database is dowwn. Will let you all know what I hear back! On Thursday, August 9, 2012 1:03:49 PM UTC-4, Joshua Smith wrote: If you put those into an IP lookup utility, you'll find that they are actually all planetlab related. I believe that what is happening is you are being DOS'd by a botnet created by these guys: http://www.planet-lab.org To report a suspected violation of this policy, contact PlanetLab Support ( sup...@planet-lab.org javascript:). If that doesn't stop it, sue them. Call the FBI. Contact everyone on the steering committe: http://www.planet-lab.org/consortium On Aug 9, 2012, at 12:13 PM, Kate mss@gmail.com javascript: wrote: They are not coming from the same IP. They are mostly in Europe but there are no subnets. There are hundreds of them and google only lets you block 100. eg 132.65.240.100 133.15.59.2 193.136.19.13 139.165.12.211 193.166.167.5 141.219.252.133 200.17.202.195 195.130.124.1 193.1.201.27 138.48.3.202 136.159.220.40 138.251.214.78 all these and more within a minute. They are all different. Kate On Thursday, August 9, 2012 9:55:29 AM UTC-4, alex wrote: Kate, If barryhunter is right and all the IPs are coming from the same ISP anyway, you can simply block the whole subnetwork ranges of that ISP (at least temporary) using dos.yaml: https://developers.google.com/appengine/docs/python/config/dos It'll be pain in the ass updating the file every time you encounter new subnets but at least you could probably save some quota 'till you move somewhere else or figure something out. -- alex On Thursday, August 9, 2012 1:59:57 PM UTC+2, Kate wrote: Hi Sergey, Here is a typical example 2012-08-09 06:51:16.597 / 302 30ms 0kb curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3 libidn/0.6.14 libssh2/0.18 202.125.215.12 - - [09/Aug/2012:04:51:16 -0700] HEAD / HTTP/1.1 302 153 - curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0zlib/1.2.3 libidn/0.6.14 libssh2/0.18 aussieclouds.appspot.com ms=31 cpu_ms=0 api_cpu_ms=0 cpm_usd=0.49 instance=00c61b117c2f994812ed63184c9c5544dea738 But the ip address varies. My code forces 302 response. Before I added the code they were throwing errors head method not found. But even though I am doing the 303 I am still getting front end time exceeded and these requests are taking up about 95% of my quota. So to keep the site alive I would have to pay for them, I have lost most of my European and Australian visitors because the site is down every night during those places daylight hours. Obviously I can't continue like this and so will have to move to a provider capable of blocking these requests, -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/g_yrLQPI49cJ. To post to this group, send email to google-a...@googlegroups.comjavascript: . To unsubscribe from this group, send email to google-appengi...@googlegroups.com javascript:. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/FIuRs3_jpfEJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] How can I block curl requests
I did that but don't think it will catch them all. I had a list of 100 individual ones, see below. Can you recognize subnets in them? blacklist: - subnet: 82.179.176.44 - subnet: 83.230.127.124 - subnet: 88.2.234.60 - subnet: 128.10.19.53 - subnet: 128.84.154.45 - subnet: 128.42.142.44 - subnet: 128.36.233.154 - subnet: 128.111.52.59 - subnet: 128.114.63.64 - subnet: 128.138.207.45 - subnet: 128.227.150.12 - subnet: 128.208.4.198 - subnet: 128.151.65.101 - subnet: 128.223.8.112 - subnet: 128.232.103.201 - subnet: 129.10.120.194 - subnet: 129.15.78.30 - subnet: 129.74.74.20 - subnet: 129.82.12.188 - subnet: 129.93.229.139 - subnet: 129.97.74.14 - subnet: 129.108.202.11 - subnet: 129.130.252.140 - subnet: 129.237.161.194 - subnet: 130.37.193.143 - subnet: 130.83.166.243 - subnet: 130.104.72.213 - subnet: 130.216.1.22 - subnet: 130.253.21.123 - subnet: 130.195.4.68 - subnet: 130.237.50.125 - subnet: 131.179.150.72 - subnet: 131.188.44.102 - subnet: 132.65.240.100 - subnet: 132.72.23.10 - subnet: 132.170.3.32 - subnet: 132.181.10.56 - subnet: 133.1.74.163 - subnet: 133.15.59.2 - subnet: 133.68.253.242 - subnet: 134.151.255.181 - subnet: 138.4.0.120 - subnet: 138.251.214.78 - subnet: 136.159.220.40 - subnet: 138.48.3.202 - subnet: 139.78.141.245 - subnet: 139.165.12.211 - subnet: 140.109.17.181 - subnet: 140.112.107.82 - subnet: 140.123.230.248 - subnet: 141.11.0.162 - subnet: 141.20.103.211 - subnet: 141.219.252.133 - subnet: 142.103.2.2 - subnet: 143.89.49.73 - subnet: 143.225.229.238 - subnet: 155.245.47.225 - subnet: 155.246.12.163 - subnet: 156.56.250.226 - subnet: 156.62.231.244 - subnet: 157.92.44.101 - subnet: 160.80.221.39 - subnet: 161.106.240.19 - subnet: 169.226.40.2 - subnet: 169.229.50.15 - subnet: 165.91.55.8 - subnet: 165.230.49.115 - subnet: 192.1.249.138 - subnet: 192.12.33.102 - subnet: 192.16.125.11 - subnet: 192.38.109.144 - subnet: 192.41.135.219 - subnet: 192.42.43.23 - subnet: 192.107.171.145 - subnet: 193.1.201.27 - subnet: 193.138.2.13 - subnet: 193.196.39.9 - subnet: 193.167.187.185 - subnet: 193.136.19.13 - subnet: 193.166.167.5 - subnet: 193.205.215.74 - subnet: 193.226.19.31 - subnet: 194.29.178.13 - subnet: 194.167.254.19 - subnet: 194.254.215.12 - subnet: 195.130.124.1 - subnet: 198.82.160.221 - subnet: 200.0.206.137 - subnet: 200.0.206.168 - subnet: 200.17.202.195 - subnet: 200.129.132.19 - subnet: 202.23.159.52 - subnet: 202.125.215.12 - subnet: 202.237.248.222 - subnet: 202.249.37.67 - subnet: 203.110.240.190 - subnet: 203.178.133.2 - subnet: 212.51.218.235 - subnet: 213.73.40.106 - subnet: 213.131.1.101 Thanks Kate On Thursday, August 9, 2012 2:37:48 PM UTC-4, alex wrote: well, while you're notifying planetlab and whatnot you could create and upload a dos.yaml for the time being with a content similar to this: blacklist: - subnet: 132.65.0.0/16 - subnet: 133.0.0.0/8 description: somewhere in china - subnet: 136.159.0.0/16 - subnet: 138.250.0.0/15 - subnet: 138.48.0.0/16 - subnet: 139.165.0.0/16 - subnet: 141.219.252.0/24 - subnet: 193.1.0.0/16 description: planetlab - subnet: 193.136.16.0/24 - subnet: 193.166.167.0/24 - subnet: 195.130.124.0/22 - subnet: 200.17.192.0/19 - put that file in your app root dir and do something like this from a terminal: appcfg.py update_dos . On Thursday, August 9, 2012 6:13:14 PM UTC+2, Kate wrote: They are not coming from the same IP. They are mostly in Europe but there are no subnets. There are hundreds of them and google only lets you block 100. eg 132.65.240.100 133.15.59.2 193.136.19.13 139.165.12.211 193.166.167.5 141.219.252.133 200.17.202.195 195.130.124.1 193.1.201.27 138.48.3.202 136.159.220.40 138.251.214.78 all these and more within a minute. They are all different. Kate On Thursday, August 9, 2012 9:55:29 AM UTC-4, alex wrote: Kate, If barryhunter is right and all the IPs are coming from the same ISP anyway, you can simply block the whole subnetwork ranges of that ISP (at least temporary) using dos.yaml: https://developers.google.com/appengine/docs/python/config/dos It'll be pain in the ass updating the file every time you encounter new subnets but at least you could probably save some quota 'till you move somewhere else or figure something out. -- alex On Thursday, August 9, 2012 1:59:57 PM UTC+2, Kate wrote: Hi Sergey, Here is a typical example 2012-08-09 06:51:16.597 / 302 30ms 0kb curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3 libidn/0.6.14 libssh2/0.18 202.125.215.12 - - [09/Aug/2012:04:51:16 -0700] HEAD / HTTP/1.1 302 153 - curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0zlib/1.2.3 libidn/0.6.14 libssh2/0.18 aussieclouds.appspot.com ms=31 cpu_ms=0 api_cpu_ms=0 cpm_usd=0.49 instance=00c61b117c2f994812ed63184c9c5544dea738 But the ip address varies. My code forces 302 response. Before I added the code they were throwing errors head method not found. But even
Re: [google-appengine] How can I block curl requests
Terrific Alex Thank you On Thursday, August 9, 2012 5:32:03 PM UTC-4, alex wrote: A lot of those IPs are assigned to universities, like almost literally all of them. More than 50% are US universities. This really looks like a big distributed bot network to me. Anyway, here you go (some IPs are from the same net block so there are less than 100 entries): - subnet: 82.179.176.0/20 - subnet: 83.230.96.0/19 - subnet: 88.2.0.0/16 - subnet: 128.10.0.0/16 - subnet: 128.84.0.0/16 - subnet: 128.42.0.0/16 - subnet: 128.36.0.0/16 - subnet: 128.111.0.0/16 - subnet: 128.114.0.0/16 - subnet: 128.138.0.0/16 - subnet: 128.227.0.0/16 - subnet: 128.208.0.0/16 - subnet: 128.151.0.0/16 - subnet: 128.223.0.0/16 - subnet: 128.232.0.0/16 - subnet: 129.10.0.0/16 - subnet: 129.15.0.0/16 - subnet: 129.74.0.0/16 - subnet: 129.82.0.0/16 - subnet: 129.93.0.0/16 - subnet: 129.97.0.0/16 - subnet: 129.108.0.0/16 - subnet: 129.130.0.0/16 - subnet: 129.237.0.0/16 - subnet: 130.37.0.0/16 - subnet: 130.83.0.0/16 - subnet: 130.104.0.0/16 - subnet: 130.216.0.0/16 - subnet: 130.253.0.0/16 - subnet: 130.195.4.0/24 - subnet: 130.237.0.0/18 - subnet: 131.179.0.0/16 - subnet: 131.188.0.0/16 - subnet: 132.64.0.0/13 - subnet: 132.72.0.0/14 - subnet: 134.151.0.0/16 - subnet: 138.4.0.0/16 - subnet: 138.250.0.0/15 - subnet: 138.48.0.0/16 - subnet: 139.165.0.0/16 - subnet: 140.109.0.0/16 - subnet: 143.225.0.0/16 - subnet: 155.245.0.0/16 - subnet: 160.80.0.0/16 - subnet: 161.106.0.0/16 - subnet: 192.16.124.0/22 - subnet: 192.38.0.0/17 - subnet: 192.41.132.0/22 - subnet: 192.42.42.0/23 - subnet: 193.1.0.0/16 - subnet: 193.138.2.0/24 - subnet: 193.196.0.0/15 - subnet: 193.166.0.0/15 - subnet: 193.136.0.0/15 - subnet: 193.166.0.0/15 - subnet: 193.204.0.0/15 - subnet: 193.226.0.0/19 - subnet: 194.29.176.0/22 - subnet: 194.167.0.0/16 - subnet: 194.254.0.0/16 - subnet: 195.130.124.0/22 - subnet: 203.110.240.0/24 - subnet: 212.51.208.0/20 - subnet: 213.73.32.0/19 - subnet: 213.131.0.0/19 - subnet: 136.159.0.0/16 - subnet: 132.170.0.0/16 - subnet: 132.181.0.0/16 - subnet: 133.0.0.0/8 - subnet: 203.0.0.0/8 - subnet: 198.82.0.0/16 - subnet: 200.0.0.0/8 - subnet: 192.107.171.0/24 - subnet: 140.112.0.0/12 - subnet: 140.123.0.0/16 - subnet: 142.103.0.0/16 - subnet: 143.89.0.0/16 - subnet: 139.78.0.0/16 - subnet: 155.246.0.0/16 - subnet: 156.56.0.0/16 - subnet: 156.62.0.0/16 - subnet: 157.92.0.0/16 - subnet: 169.226.0.0/16 - subnet: 165.91.0.0/16 - subnet: 165.230.0.0/16 - subnet: 192.1.0.0/16 - subnet: 192.12.33.0/24 - subnet: 169.229.0.0/16 - subnet: 141.219.0.0/16 # these belong to a too big block - subnet: 141.11.0.162 - subnet: 141.20.103.211 On Thursday, August 9, 2012 10:21:03 PM UTC+2, Kate wrote: I did that but don't think it will catch them all. I had a list of 100 individual ones, see below. Can you recognize subnets in them? blacklist: - subnet: 82.179.176.44 - subnet: 83.230.127.124 - subnet: 88.2.234.60 - subnet: 128.10.19.53 - subnet: 128.84.154.45 - subnet: 128.42.142.44 - subnet: 128.36.233.154 - subnet: 128.111.52.59 - subnet: 128.114.63.64 - subnet: 128.138.207.45 - subnet: 128.227.150.12 - subnet: 128.208.4.198 - subnet: 128.151.65.101 - subnet: 128.223.8.112 - subnet: 128.232.103.201 - subnet: 129.10.120.194 - subnet: 129.15.78.30 - subnet: 129.74.74.20 - subnet: 129.82.12.188 - subnet: 129.93.229.139 - subnet: 129.97.74.14 - subnet: 129.108.202.11 - subnet: 129.130.252.140 - subnet: 129.237.161.194 - subnet: 130.37.193.143 - subnet: 130.83.166.243 - subnet: 130.104.72.213 - subnet: 130.216.1.22 - subnet: 130.253.21.123 - subnet: 130.195.4.68 - subnet: 130.237.50.125 - subnet: 131.179.150.72 - subnet: 131.188.44.102 - subnet: 132.65.240.100 - subnet: 132.72.23.10 - subnet: 132.170.3.32 - subnet: 132.181.10.56 - subnet: 133.1.74.163 - subnet: 133.15.59.2 - subnet: 133.68.253.242 - subnet: 134.151.255.181 - subnet: 138.4.0.120 - subnet: 138.251.214.78 - subnet: 136.159.220.40 - subnet: 138.48.3.202 - subnet: 139.78.141.245 - subnet: 139.165.12.211 - subnet: 140.109.17.181 - subnet: 140.112.107.82 - subnet: 140.123.230.248 - subnet: 141.11.0.162 - subnet: 141.20.103.211 - subnet: 141.219.252.133 - subnet: 142.103.2.2 - subnet: 143.89.49.73 - subnet: 143.225.229.238 - subnet: 155.245.47.225 - subnet: 155.246.12.163 - subnet: 156.56.250.226 - subnet: 156.62.231.244 - subnet: 157.92.44.101 - subnet: 160.80.221.39 - subnet: 161.106.240.19 - subnet: 169.226.40.2 - subnet: 169.229.50.15 - subnet: 165.91.55.8 - subnet: 165.230.49.115 - subnet: 192.1.249.138 - subnet: 192.12.33.102 - subnet: 192.16.125.11 - subnet: 192.38.109.144 - subnet: 192.41.135.219 - subnet: 192.42.43.23 - subnet: 192.107.171.145 - subnet: 193.1.201.27 - subnet: 193.138.2.13 - subnet: 193.196.39.9 - subnet: 193.167.187.185 - subnet: 193.136.19.13 - subnet: 193.166.167.5 - subnet
Re: [google-appengine] How can I block curl requests
Done, Actually I have some more. Can you subnet any of these? all planet-lab ones - subnet: 134.117.226.181 - subnet: 147.229.10.250 - subnet: 206.23.240.29 - subnet: 160.193.163.106 - subnet: 138.246.99.249 - subnet: 129.242.19.197 - subnet: 206.12.16.154 - subnet: 160.193.163.106 - subnet: 152.3.138.6 - subnet: 195.37.16.121 I am up to 100 now because of these individual ones. On Thursday, August 9, 2012 5:37:41 PM UTC-4, alex wrote: Actually, scratch last three lines (starting from # these belong to...) and replace with - subnet: 141.0.0.0/8 On Thursday, August 9, 2012 11:32:03 PM UTC+2, alex wrote: A lot of those IPs are assigned to universities, like almost literally all of them. More than 50% are US universities. This really looks like a big distributed bot network to me. Anyway, here you go (some IPs are from the same net block so there are less than 100 entries): - subnet: 82.179.176.0/20 - subnet: 83.230.96.0/19 - subnet: 88.2.0.0/16 - subnet: 128.10.0.0/16 - subnet: 128.84.0.0/16 - subnet: 128.42.0.0/16 - subnet: 128.36.0.0/16 - subnet: 128.111.0.0/16 - subnet: 128.114.0.0/16 - subnet: 128.138.0.0/16 - subnet: 128.227.0.0/16 - subnet: 128.208.0.0/16 - subnet: 128.151.0.0/16 - subnet: 128.223.0.0/16 - subnet: 128.232.0.0/16 - subnet: 129.10.0.0/16 - subnet: 129.15.0.0/16 - subnet: 129.74.0.0/16 - subnet: 129.82.0.0/16 - subnet: 129.93.0.0/16 - subnet: 129.97.0.0/16 - subnet: 129.108.0.0/16 - subnet: 129.130.0.0/16 - subnet: 129.237.0.0/16 - subnet: 130.37.0.0/16 - subnet: 130.83.0.0/16 - subnet: 130.104.0.0/16 - subnet: 130.216.0.0/16 - subnet: 130.253.0.0/16 - subnet: 130.195.4.0/24 - subnet: 130.237.0.0/18 - subnet: 131.179.0.0/16 - subnet: 131.188.0.0/16 - subnet: 132.64.0.0/13 - subnet: 132.72.0.0/14 - subnet: 134.151.0.0/16 - subnet: 138.4.0.0/16 - subnet: 138.250.0.0/15 - subnet: 138.48.0.0/16 - subnet: 139.165.0.0/16 - subnet: 140.109.0.0/16 - subnet: 143.225.0.0/16 - subnet: 155.245.0.0/16 - subnet: 160.80.0.0/16 - subnet: 161.106.0.0/16 - subnet: 192.16.124.0/22 - subnet: 192.38.0.0/17 - subnet: 192.41.132.0/22 - subnet: 192.42.42.0/23 - subnet: 193.1.0.0/16 - subnet: 193.138.2.0/24 - subnet: 193.196.0.0/15 - subnet: 193.166.0.0/15 - subnet: 193.136.0.0/15 - subnet: 193.166.0.0/15 - subnet: 193.204.0.0/15 - subnet: 193.226.0.0/19 - subnet: 194.29.176.0/22 - subnet: 194.167.0.0/16 - subnet: 194.254.0.0/16 - subnet: 195.130.124.0/22 - subnet: 203.110.240.0/24 - subnet: 212.51.208.0/20 - subnet: 213.73.32.0/19 - subnet: 213.131.0.0/19 - subnet: 136.159.0.0/16 - subnet: 132.170.0.0/16 - subnet: 132.181.0.0/16 - subnet: 133.0.0.0/8 - subnet: 203.0.0.0/8 - subnet: 198.82.0.0/16 - subnet: 200.0.0.0/8 - subnet: 192.107.171.0/24 - subnet: 140.112.0.0/12 - subnet: 140.123.0.0/16 - subnet: 142.103.0.0/16 - subnet: 143.89.0.0/16 - subnet: 139.78.0.0/16 - subnet: 155.246.0.0/16 - subnet: 156.56.0.0/16 - subnet: 156.62.0.0/16 - subnet: 157.92.0.0/16 - subnet: 169.226.0.0/16 - subnet: 165.91.0.0/16 - subnet: 165.230.0.0/16 - subnet: 192.1.0.0/16 - subnet: 192.12.33.0/24 - subnet: 169.229.0.0/16 - subnet: 141.219.0.0/16 # these belong to a too big block - subnet: 141.11.0.162 - subnet: 141.20.103.211 On Thursday, August 9, 2012 10:21:03 PM UTC+2, Kate wrote: I did that but don't think it will catch them all. I had a list of 100 individual ones, see below. Can you recognize subnets in them? blacklist: - subnet: 82.179.176.44 - subnet: 83.230.127.124 - subnet: 88.2.234.60 - subnet: 128.10.19.53 - subnet: 128.84.154.45 - subnet: 128.42.142.44 - subnet: 128.36.233.154 - subnet: 128.111.52.59 - subnet: 128.114.63.64 - subnet: 128.138.207.45 - subnet: 128.227.150.12 - subnet: 128.208.4.198 - subnet: 128.151.65.101 - subnet: 128.223.8.112 - subnet: 128.232.103.201 - subnet: 129.10.120.194 - subnet: 129.15.78.30 - subnet: 129.74.74.20 - subnet: 129.82.12.188 - subnet: 129.93.229.139 - subnet: 129.97.74.14 - subnet: 129.108.202.11 - subnet: 129.130.252.140 - subnet: 129.237.161.194 - subnet: 130.37.193.143 - subnet: 130.83.166.243 - subnet: 130.104.72.213 - subnet: 130.216.1.22 - subnet: 130.253.21.123 - subnet: 130.195.4.68 - subnet: 130.237.50.125 - subnet: 131.179.150.72 - subnet: 131.188.44.102 - subnet: 132.65.240.100 - subnet: 132.72.23.10 - subnet: 132.170.3.32 - subnet: 132.181.10.56 - subnet: 133.1.74.163 - subnet: 133.15.59.2 - subnet: 133.68.253.242 - subnet: 134.151.255.181 - subnet: 138.4.0.120 - subnet: 138.251.214.78 - subnet: 136.159.220.40 - subnet: 138.48.3.202 - subnet: 139.78.141.245 - subnet: 139.165.12.211 - subnet: 140.109.17.181 - subnet: 140.112.107.82 - subnet: 140.123.230.248 - subnet: 141.11.0.162 - subnet: 141.20.103.211 - subnet: 141.219.252.133 - subnet: 142.103.2.2 - subnet: 143.89.49.73 - subnet: 143.225.229.238 - subnet: 155.245.47.225 - subnet: 155.246.12.163 - subnet: 156.56.250.226
Re: [google-appengine] How can I block curl requests
It isn't stopping them. I am just not getting errors. What is troubling is that these curl requests are counting as hits and there are so many of tens of thousands of them it is hard for me to analyze site traffic as genuine requests are buried in the curl stats. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/p4fPoQaAACIJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] How can I block curl requests
Thanks Kyle, it does help. But I still hate those nasty people! I opened a production issue with google a week ago so maybe they will help someday ... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Jg6pq5l490MJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Low hitting site experiencing denial of service attack - site opffline!
My site has been on GAE for several years but am now experiencing denial of service attacks. I don't want to pay for them so my site is now down. I put in a production issue last week and have tried blocking the user agent (curl). But I can't block by IP as all IPs are different. Apart from moving from GAE what can I do. Thanks in advance, Kate -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/_j6s1MEhVVQJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] How can I block curl requests
My site is now down as I'm over quota. I can't tun billing on as it is too expensive to pay for these dos attacks. Thanks everyone for being helpful but I think I'm beaten on this. It seems a pity that a non profit site could be brought down by this but that's the case. Google doesn't seem to care as there has been no response on my production issue. I suppose it isn't in their interest as I either pay for the attacks or lose my site. V discouraging. All attempts at blocking the attacks has only increased their volume. Kate On Wednesday, August 8, 2012 1:24:36 PM UTC-4, Kyle Finley wrote: In the admin logs - under options - you can filter by regular expression. Does that help? On Aug 8, 2012, at 12:11 PM, Kate wrote: It isn't stopping them. I am just not getting errors. What is troubling is that these curl requests are counting as hits and there are so many of tens of thousands of them it is hard for me to analyze site traffic as genuine requests are buried in the curl stats. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/p4fPoQaAACIJ. To post to this group, send email to google-a...@googlegroups.comjavascript:. To unsubscribe from this group, send email to google-appengi...@googlegroups.com javascript:. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/0uMr6RgJyxwJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Low hitting site experiencing denial of service attack - site opffline!
I don't know how to write the node.js. I am surprised that google cannot help me here. I do appreciate your help but I just can't do this without spending hours of study. To answer your other question, the blocking attempt didn't solve the problem. The curl requests come about 3 times per second. I have been working on trying to stop them for over a week now. I suppose I can just change my domain name or move to another hosting service but it seems a pity. I have had this site since 1996 with never a problem. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/RLTiLlM5cTsJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] How can I block curl requests
@Kyle
I changed to 503 and didn't update my python.
Is this good or bad
C:\inetpubcurl -v http://www.coolabah.com
* About to connect() to www.coolabah.com port 80 (#0)
* Trying 205.178.189.131...
* connected
* Connected to www.coolabah.com (205.178.189.131) port 80 (#0)
GET / HTTP/1.1
User-Agent: curl/7.26.0
Host: www.coolabah.com
Accept: */*
HTTP/1.1 302 Moved Temporarily
Content-Length: 0
Location: /?bee68f00
* Connection #0 to host www.coolabah.com left intact
* Closing connection #0
\inetpub
On Friday, August 3, 2012 11:52:09 AM UTC-4, Kyle Finley wrote:
@Brandon
Yes, 503 would probably be better then 401.
@Joshua
No 429 doesn't work. I don't know if the allowed return values are
documented, but here's the source:
http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/ext/webapp/_webapp25.py#345
On Aug 3, 2012, at 10:45 AM, Joshua Smith wrote:
I would have thought self.error(429). That doesn't work? Is there a doc
that says what codes are are allowed to return?
On Aug 3, 2012, at 11:02 AM, Kyle Finley kylefin...@gmail.com wrote:
Yes, thank you. Do you have any thoughts on how to return error code 429?
On Aug 3, 2012, at 9:51 AM, Joshua Smith wrote:
There are couple problems with your snippet.
First, she's getting HEAD not GET requests, so you need to use different
handler.
Also, you aren't returning, so if you were in a GET request, it would
proceed to handle the request regardless.
Something more like this (untested):
class MainHandler(webapp.RequestHandler):
def head(self):
self.error(401)
def get(self):
if (self.request.headers['User-Agent'].startswith('curl'))
self.error(401)
return
# rest of the get handler
On Aug 3, 2012, at 10:34 AM, Kyle Finley kylefin...@gmail.com wrote:
Hi Joshua,
Thank you, that's a good thought.
Kate sent me some files offline, and I believe we've figured out the
problem. For the middleware to work you must be using WSGI not CGI. Someone
please correct me if I'm wrong, but I believe she would have to upgrade
here App to python27 to use it. The alternative is to do the check in the
webapp request handler:
def check_for_curl(self):
if self.request.environ['HTTP_USER_AGENT'].startswith('curl'):
return self.error(401)
class MainHandler(webapp.RequestHandler):
def get(self):
check_for_curl(self)
# handle request
The problem is that webapp doesn't recognize error code 429 so we have to
use something else. Unless there's a simple way to make it write 429?
- Kyle
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/TQuZYYR0wrAJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/4GmoCKWhujAJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] How can I block curl requests
OOPs. my error.
If I curl to http://aussieclouds.appspot.com or to
http://www.australiansabroad.com it gets through still. I will update
pyphon. coolabah.com points to australiansabroad.com and I thought they
resolved to the same address. Apparently not.
On Monday, August 6, 2012 2:26:17 PM UTC-4, Kate wrote:
@Kyle
I changed to 503 and didn't update my python.
Is this good or bad
C:\inetpubcurl -v http://www.coolabah.com
* About to connect() to www.coolabah.com port 80 (#0)
* Trying 205.178.189.131...
* connected
* Connected to www.coolabah.com (205.178.189.131) port 80 (#0)
GET / HTTP/1.1
User-Agent: curl/7.26.0
Host: www.coolabah.com
Accept: */*
HTTP/1.1 302 Moved Temporarily
Content-Length: 0
Location: /?bee68f00
* Connection #0 to host www.coolabah.com left intact
* Closing connection #0
\inetpub
On Friday, August 3, 2012 11:52:09 AM UTC-4, Kyle Finley wrote:
@Brandon
Yes, 503 would probably be better then 401.
@Joshua
No 429 doesn't work. I don't know if the allowed return values are
documented, but here's the source:
http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/ext/webapp/_webapp25.py#345
On Aug 3, 2012, at 10:45 AM, Joshua Smith wrote:
I would have thought self.error(429). That doesn't work? Is there a doc
that says what codes are are allowed to return?
On Aug 3, 2012, at 11:02 AM, Kyle Finley kylefin...@gmail.com wrote:
Yes, thank you. Do you have any thoughts on how to return error code 429?
On Aug 3, 2012, at 9:51 AM, Joshua Smith wrote:
There are couple problems with your snippet.
First, she's getting HEAD not GET requests, so you need to use different
handler.
Also, you aren't returning, so if you were in a GET request, it would
proceed to handle the request regardless.
Something more like this (untested):
class MainHandler(webapp.RequestHandler):
def head(self):
self.error(401)
def get(self):
if (self.request.headers['User-Agent'].startswith('curl'))
self.error(401)
return
# rest of the get handler
On Aug 3, 2012, at 10:34 AM, Kyle Finley kylefin...@gmail.com wrote:
Hi Joshua,
Thank you, that's a good thought.
Kate sent me some files offline, and I believe we've figured out the
problem. For the middleware to work you must be using WSGI not CGI. Someone
please correct me if I'm wrong, but I believe she would have to upgrade
here App to python27 to use it. The alternative is to do the check in the
webapp request handler:
def check_for_curl(self):
if self.request.environ['HTTP_USER_AGENT'].startswith('curl'):
return self.error(401)
class MainHandler(webapp.RequestHandler):
def get(self):
check_for_curl(self)
# handle request
The problem is that webapp doesn't recognize error code 429 so we have to
use something else. Unless there's a simple way to make it write 429?
- Kyle
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/TQuZYYR0wrAJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/PZ0hWrvigZEJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http
Re: [google-appengine] How can I block curl requests
Just read this. Thanks.
Looks like it works now.
Thanks to all!
Kate
On Friday, August 3, 2012 10:51:23 AM UTC-4, Joshua Smith wrote:
There are couple problems with your snippet.
First, she's getting HEAD not GET requests, so you need to use different
handler.
Also, you aren't returning, so if you were in a GET request, it would
proceed to handle the request regardless.
Something more like this (untested):
class MainHandler(webapp.RequestHandler):
def head(self):
self.error(401)
def get(self):
if (self.request.headers['User-Agent'].startswith('curl'))
self.error(401)
return
# rest of the get handler
On Aug 3, 2012, at 10:34 AM, Kyle Finley kylefin...@gmail.com wrote:
Hi Joshua,
Thank you, that's a good thought.
Kate sent me some files offline, and I believe we've figured out the
problem. For the middleware to work you must be using WSGI not CGI. Someone
please correct me if I'm wrong, but I believe she would have to upgrade
here App to python27 to use it. The alternative is to do the check in the
webapp request handler:
def check_for_curl(self):
if self.request.environ['HTTP_USER_AGENT'].startswith('curl'):
return self.error(401)
class MainHandler(webapp.RequestHandler):
def get(self):
check_for_curl(self)
# handle request
The problem is that webapp doesn't recognize error code 429 so we have to
use something else. Unless there's a simple way to make it write 429?
- Kyle
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/TQuZYYR0wrAJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/hI3PU4EwpG0J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] How can I block curl requests
But I still don't like them hitting my site every 500 ms! e.g.
1.
1. 2012-08-06 13:56:02.725 / 401 32ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
2.
1. 2012-08-06 13:56:02.279 / 401 70ms 0kb curl/7.19.7
(i386-redhat-linux-gnu) libcurl/7.19.7 NSS/3.12.6.2 zlib/1.2.3 libidn/1.9
libssh2/1.2.4
3.
1. 2012-08-06 13:55:57.921 / 401 29ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
4.
1. 2012-08-06 13:55:55.403 / 401 9ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
5.
1. 2012-08-06 13:55:54.323 / 401 52ms 0kb curl/7.19.7
(i386-redhat-linux-gnu) libcurl/7.19.7 NSS/3.12.6.2 zlib/1.2.3 libidn/1.9
libssh2/1.2.4
6.
1. 2012-08-06 13:55:54.283 / 401 33ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
7.
1. 2012-08-06 13:55:52.814 / 401 82ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
8.
1. 2012-08-06 13:55:52.437 / 401 50ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
9.
1. 2012-08-06 13:55:49.063 / 401 33ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
10.
1. 2012-08-06 13:55:45.986 / 401 10ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
11.
1. 2012-08-06 13:55:43.610 / 401 18ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
12.
1. 2012-08-06 13:55:42.189 / 401 29ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
13.
1. 2012-08-06 13:55:42.114 / 401 76ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
14.
1. 2012-08-06 13:55:39.592 / 401 51ms 0kb curl/7.21.0
(x86_64-redhat-linux-gnu) libcurl/7.21.0 NSS/3.12.10.0 zlib/1.2.5
libidn/1.18 libssh2/1.2.4
15.
1. 2012-08-06 13:55:38.948 / 401 85ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
16.
1. 2012-08-06 13:55:38.945 / 401 130ms 0kb curl/7.19.7
(i386-redhat-linux-gnu) libcurl/7.19.7 NSS/3.12.6.2 zlib/1.2.3 libidn/1.9
libssh2/1.2.4
17.
1. 2012-08-06 13:55:38.944 / 401 123ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
On Monday, August 6, 2012 2:50:55 PM UTC-4, Kate wrote:
Just read this. Thanks.
Looks like it works now.
Thanks to all!
Kate
On Friday, August 3, 2012 10:51:23 AM UTC-4, Joshua Smith wrote:
There are couple problems with your snippet.
First, she's getting HEAD not GET requests, so you need to use different
handler.
Also, you aren't returning, so if you were in a GET request, it would
proceed to handle the request regardless.
Something more like this (untested):
class MainHandler(webapp.RequestHandler):
def head(self):
self.error(401)
def get(self):
if (self.request.headers['User-Agent'].startswith('curl'))
self.error(401)
return
# rest of the get handler
On Aug 3, 2012, at 10:34 AM, Kyle Finley kylefin...@gmail.com wrote:
Hi Joshua,
Thank you, that's a good thought.
Kate sent me some files offline, and I believe we've figured out the
problem. For the middleware to work you must be using WSGI not CGI. Someone
please correct me if I'm wrong, but I believe she would have to upgrade
here App to python27 to use it. The alternative is to do the check in the
webapp request handler:
def check_for_curl(self):
if self.request.environ['HTTP_USER_AGENT'].startswith('curl'):
return self.error(401)
class MainHandler(webapp.RequestHandler):
def get(self):
check_for_curl(self)
# handle request
The problem is that webapp doesn't recognize error code 429 so we have to
use something else. Unless there's a simple way to make it write 429?
- Kyle
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/TQuZYYR0wrAJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more
Re: [google-appengine] How can I block curl requests
I like it!!
On Monday, August 6, 2012 3:53:46 PM UTC-4, Joshua Smith wrote:
You could have some fun with them. Instead of returning an error, you
could redirect them someplace:
Replace self.error(401) with:
self.redirect(http://localhost/;)
for example. Curl does follow redirects by default, but it doesn't have
to, so this may or may not have an effect.
If they are following redirects, you could set up a backend with a handler
that just sleeps for 30 seconds. Then redirect them all to that one
backend, where they can all sit in line waiting.
On Aug 6, 2012, at 2:58 PM, Kate mss.ka...@gmail.com wrote:
But I still don't like them hitting my site every 500 ms! e.g.
1.
1. 2012-08-06 13:56:02.725 / 401 32ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
2.
1. 2012-08-06 13:56:02.279 / 401 70ms 0kb curl/7.19.7
(i386-redhat-linux-gnu) libcurl/7.19.7 NSS/3.12.6.2 zlib/1.2.3
libidn/1.9 libssh2/1.2.4
3.
1. 2012-08-06 13:55:57.921 / 401 29ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
4.
1. 2012-08-06 13:55:55.403 / 401 9ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
5.
1. 2012-08-06 13:55:54.323 / 401 52ms 0kb curl/7.19.7
(i386-redhat-linux-gnu) libcurl/7.19.7 NSS/3.12.6.2 zlib/1.2.3
libidn/1.9 libssh2/1.2.4
6.
1. 2012-08-06 13:55:54.283 / 401 33ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
7.
1. 2012-08-06 13:55:52.814 / 401 82ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
8.
1. 2012-08-06 13:55:52.437 / 401 50ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
9.
1. 2012-08-06 13:55:49.063 / 401 33ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
10.
1. 2012-08-06 13:55:45.986 / 401 10ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
11.
1. 2012-08-06 13:55:43.610 / 401 18ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
12.
1. 2012-08-06 13:55:42.189 / 401 29ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
13.
1. 2012-08-06 13:55:42.114 / 401 76ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
14.
1. 2012-08-06 13:55:39.592 / 401 51ms 0kb curl/7.21.0
(x86_64-redhat-linux-gnu) libcurl/7.21.0 NSS/3.12.10.0 zlib/1.2.5
libidn/1.18 libssh2/1.2.4
15.
1. 2012-08-06 13:55:38.948 / 401 85ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
16.
1. 2012-08-06 13:55:38.945 / 401 130ms 0kb curl/7.19.7
(i386-redhat-linux-gnu) libcurl/7.19.7 NSS/3.12.6.2 zlib/1.2.3
libidn/1.9 libssh2/1.2.4
17.
1. 2012-08-06 13:55:38.944 / 401 123ms 0kb curl/7.18.2
(i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3
libidn/0.6.14 libssh2/0.18
On Monday, August 6, 2012 2:50:55 PM UTC-4, Kate wrote:
Just read this. Thanks.
Looks like it works now.
Thanks to all!
Kate
On Friday, August 3, 2012 10:51:23 AM UTC-4, Joshua Smith wrote:
There are couple problems with your snippet.
First, she's getting HEAD not GET requests, so you need to use different
handler.
Also, you aren't returning, so if you were in a GET request, it would
proceed to handle the request regardless.
Something more like this (untested):
class MainHandler(webapp.RequestHandler):
def head(self):
self.error(401)
def get(self):
if (self.request.headers['User-Agent'].startswith('curl'))
self.error(401)
return
# rest of the get handler
On Aug 3, 2012, at 10:34 AM, Kyle Finley kylefin...@gmail.com wrote:
Hi Joshua,
Thank you, that's a good thought.
Kate sent me some files offline, and I believe we've figured out the
problem. For the middleware to work you must be using WSGI not CGI. Someone
please correct me if I'm wrong, but I believe she would have to upgrade
here App to python27 to use it. The alternative is to do the check in the
webapp request handler:
def check_for_curl(self):
if self.request.environ['HTTP_USER_AGENT'].startswith('curl
[google-appengine] Re: How can I block curl requests
Well I tried this by testing user agent but it passes the test and the page
loads correctly, which it shouldn't in this example.
Am I meant to pass 'app' as such? I am not sure of this parameter.
Here is my code
from webob import Response
class AntiCurlMiddleware(object):
def __init__(self, app):
self.app = app
def __call__(self, environ, start_response):
ua = os.environ.get('HTTP_USER_AGENT', unknown)
sindex = string.find(ua,'Win',0)
if sindex 0:
resp = Response('Too many requests!')
resp.status_code = 423
return resp(environ, start_response)
return self.app(environ, start_response)
def webapp_add_wsgi_middleware(app):
return AntiCurlMiddlewarey(app)
On Wednesday, August 1, 2012 4:43:58 PM UTC-4, Kate wrote:
I am getting tens of thousands of curl requests - many thousands time
browser requests and I want to block them. I'm using python. They coming
from many different IPs most in Europe. If I can't stop them I will have
to close my site or go to a new provider.
Thank in advance,
Kate
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/kRNatAqarF8J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: How can I block curl requests
I am looking for 'win' in user agent as that IS the USER Agent I am using
to test. It should evaluate to true and give a 423 error.
I do not want to test it using curl. I am trying to see if I can return a
423 error. It is a TEST!
I am running it locally to see if it will block.
It doesn't.
'Win' is in my user agent - as I am testing.
On Thursday, August 2, 2012 3:43:44 PM UTC-4, hyperflame wrote:
You're trying to find curl requests by looking for a win string?
IIRC, curl uses curl as its default user-agent, you have to look for
that. Also, it's a bad idea to look for Win, as legitimate requests
(users using WINdows) will be blocked.
Also, it should be noted (and I believe a number of people have
already mentioned) that curl allows a person to change their user-
agent; just looking for a curl user-agent may not stop the problem.
On Aug 2, 1:55 pm, Kate mss.ka...@gmail.com wrote:
Well I tried this by testing user agent but it passes the test and the
page
loads correctly, which it shouldn't in this example.
Am I meant to pass 'app' as such? I am not sure of this parameter.
Here is my code
from webob import Response
class AntiCurlMiddleware(object):
def __init__(self, app):
self.app = app
def __call__(self, environ, start_response):
ua = os.environ.get('HTTP_USER_AGENT', unknown)
sindex = string.find(ua,'Win',0)
if sindex 0:
resp = Response('Too many requests!')
resp.status_code = 423
return resp(environ, start_response)
return self.app(environ, start_response)
def webapp_add_wsgi_middleware(app):
return AntiCurlMiddlewarey(app)
On Wednesday, August 1, 2012 4:43:58 PM UTC-4, Kate wrote:
I am getting tens of thousands of curl requests - many thousands time
browser requests and I want to block them. I'm using python. They
coming
from many different IPs most in Europe. If I can't stop them I will
have
to close my site or go to a new provider.
Thank in advance,
Kate
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/_ihaQnsMLZ8J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] trying to send HTTP ERROR 429 for a certain user agent
How can I trap a certain user agent and send a 429 response.
I have the following in my appengine_config.py and it always defaults to a
200.
(Note I do not intend to block all Windows user agent requests bit have the
code sindex = string.find(ua,'Win',0) there for testing only).
from webob import Response
import os
class AntiCurlMiddleware(object):
def __init__(self, app):
self.app = app
def __call__(self, environ, start_response):
ua = os.environ.get('HTTP_USER_AGENT', unknown)
sindex = string.find(ua,'Win',0)
if sindex 0:
resp = Response('Too many requests!')
resp.status_code =429
return resp(environ, start_response)
else:
return self.app(environ, start_response)
def webapp_add_wsgi_middleware(app):
return AntiCurlMiddlewarey(app)
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/3Hoo-P6wYj0J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Google App Engine, rogue crawlers, and PageSpeed Insights
I am having a similar problem and still cannot find an answer. The requests are all curl requests and I have tried everything I can think of. I tried using appengine_config.py and checking for a user agent but that didn't work. All the IP addresses are different. Surely there must be a solution for this sort of problem. On Friday, July 27, 2012 4:59:04 PM UTC-4, Jeff Schnitzer wrote: On Thu, Jul 26, 2012 at 8:45 PM, Drake drak...@digerat.com wrote: And then when Google Spam team bot shows up you would be delisted... That would Rock... It's highly improbable that anyone in an official capacity at Google will ever view your page with the exact User-Agent: AppEngine-Google; (+http://code.google.com/appengine; appid: s~steprep) Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/-qNFr7Za1sUJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Google App Engine, rogue crawlers, and PageSpeed Insights
How can I block the following curl requests. Not every IP is different and I get 10s of 1000s of them every day. Honestly I do not know HOW to block them. What method/code? 2012-08-02 15:03:21.103 / 405 55ms 0kb curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3 libidn/0.6.14 libssh2/0.18 132.72.23.10 - - [02/Aug/2012:13:03:21 -0700] HEAD / HTTP/1.1 405 124 - curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3 libidn/0.6.14 libssh2/0.18 aussieclouds.appspot.com ms=56 cpu_ms=0 api_cpu_ms=0 cpm_usd=0.45 instance=00c61b117c41a67b1b944a189d7cc38d5365564c https://appengine.google.com/instances?app_id=aussiecloudsversion_id=1.360754534133043769key=00c61b117c41a67b1b944a189d7cc38d5365564c#00c61b117c41a67b1b944a189d7cc38d5365564c On Thursday, July 26, 2012 5:27:27 PM UTC-4, Jeff Schnitzer wrote: Every fetch request from GAE includes the appid as a header... you obviously see it yourself, which is how you know the appid of the crawler. This is how Google enables you to block applications; just block all requests with that particular header. Jeff On Wed, Jul 25, 2012 at 9:35 AM, jswap js...@yahoo.com wrote: I run a website containing lots of doctor-related data. We get crawled by rogue crawlers from thousands of IP addresses DAILY (mostly in Russia) and we sometimes see our content show up on other websites. I define a crawler as rogue when it does not obey robots.txt exclusions, and the crawling company offers no benefit to us and just sucks up system resources. Google App Engine is hosting a crawler (appid: s~steprep) that is similar to the Russian ones we block. This crawler crawls us aggressively, sucks up system resources, ignores the robots.txt file, and offers no benefit to us. Per our usual policy, we have been blocking the hundreds of Google IP addresses that this crawler is crawling from. The problem is that one or more of these IP addresses also hosts Google's PageSpeed Insights page, located here: https://developers.google.com/speed/pagespeed/insights My questions for Google are: 1 - Is it your intention that websites be unable to block crawlers that you host? 2 - Is it your intention that websites must allow the steprep crawler in exchange for using the PageSpeed Insights tool? Some people may suggest why not just ask the company crawling you to stop crawling you? 1 - Some companies ignore the request. 2 - Some companies temporarily stop crawling, then show up again a few days or weeks later, at which point I have to waste time dealing with it all over again. If we were to allow every crawler to crawl our site, our server would be brought to its knees. I'm not going to waste money on increasing server resources just so more crawlers can scrape our data. Website owners need a mechanism for blocking rogue crawlers, even when they are hosted by Google App Engine. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Bo8u134CRr8J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/uFcISD9ePFgJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: How can I block curl requests
Thanks Kyle,
I don't have a system with access to curl and will download it for Windows
later. Meanwhile I've put your code on my site.
Can you test it with curl for me? I tried testing for user agent starting
with Mozilla , from Firefox but it didn't work.
http://www.australiansabroad.com
Many thanks!!
Kate
On Thursday, August 2, 2012 4:42:44 PM UTC-4, Kyle Finley wrote:
Kate, sorry there were some errors
Here's a working example:
https://github.com/scotch/operation_curl_block/blob/master/appengine_config.py
It should be status not status_code, AntiCurlMiddleware not
AntiCurlMiddleware*y *and I used startswith *instead* of *is*
You can test it here:
*curl -v http://anticurl.scotch-media.appspot.com*
will return:
* About to connect() to anticurl.scotch-media.appspot.com port 80 (#0)
* Trying 173.194.77.141... connected
* Connected to anticurl.scotch-media.appspot.com (173.194.77.141) port 80
(#0)
GET / HTTP/1.1
User-Agent: curl/7.21.4 (universal-apple-darwin11.0) libcurl/7.21.4
OpenSSL/0.9.8r zlib/1.2.5
Host: anticurl.scotch-media.appspot.com
Accept: */*
HTTP/1.1 *423* Locked
Content-Type: text/html; charset=UTF-8
Vary: Accept-Encoding
Date: Thu, 02 Aug 2012 20:40:13 GMT
Server: Google Frontend
Cache-Control: private
Transfer-Encoding: chunked
* Connection #0 to host anticurl.scotch-media.appspot.com left intact
* Closing connection #0
*Too many requests!*
I hope that helps
- Kyle
On Thursday, August 2, 2012 2:47:57 PM UTC-5, Kate wrote:
I am looking for 'win' in user agent as that IS the USER Agent I am using
to test. It should evaluate to true and give a 423 error.
I do not want to test it using curl. I am trying to see if I can return a
423 error. It is a TEST!
I am running it locally to see if it will block.
It doesn't.
'Win' is in my user agent - as I am testing.
On Thursday, August 2, 2012 3:43:44 PM UTC-4, hyperflame wrote:
You're trying to find curl requests by looking for a win string?
IIRC, curl uses curl as its default user-agent, you have to look for
that. Also, it's a bad idea to look for Win, as legitimate requests
(users using WINdows) will be blocked.
Also, it should be noted (and I believe a number of people have
already mentioned) that curl allows a person to change their user-
agent; just looking for a curl user-agent may not stop the problem.
On Aug 2, 1:55 pm, Kate mss.ka...@gmail.com wrote:
Well I tried this by testing user agent but it passes the test and the
page
loads correctly, which it shouldn't in this example.
Am I meant to pass 'app' as such? I am not sure of this parameter.
Here is my code
from webob import Response
class AntiCurlMiddleware(object):
def __init__(self, app):
self.app = app
def __call__(self, environ, start_response):
ua = os.environ.get('HTTP_USER_AGENT', unknown)
sindex = string.find(ua,'Win',0)
if sindex 0:
resp = Response('Too many requests!')
resp.status_code = 423
return resp(environ, start_response)
return self.app(environ, start_response)
def webapp_add_wsgi_middleware(app):
return AntiCurlMiddlewarey(app)
On Wednesday, August 1, 2012 4:43:58 PM UTC-4, Kate wrote:
I am getting tens of thousands of curl requests - many thousands
time
browser requests and I want to block them. I'm using python. They
coming
from many different IPs most in Europe. If I can't stop them I will
have
to close my site or go to a new provider.
Thank in advance,
Kate
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/OQh8YTtF26cJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: How can I block curl requests
Yes it is in the root directory. I am stumped! I didn't think it was working as I tried testing for different browsers and it didn't catch them. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/zMRl_ScuwoMJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: How can I block curl requests
How do I restart the instance? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/wQ-LIZ8Y3NoJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] How can I block curl requests
I restarted the instance. I also got a copy of curl.exe for windows and it
lets me through! :-(
My code is exactly this (below) and is in the file appengine_config.py in
the root directory.
I also tested your example from my machine and got blocked.
I can't think of why this could be! I even altered appengine_config.py to
have a syntax error and it picked it up, so I know it is loading.
from webob import Response
class AntiCurlMiddleware(object):
def __init__(self, app):
self.app = app
def __call__(self, environ, start_response):
if environ['HTTP_USER_AGENT'].startswith('curl'):
resp = Response('Too many requests!')
resp.status = 423
return resp(environ, start_response)
return self.app(environ, start_response)
def webapp_add_wsgi_middleware(app):
return AntiCurlMiddleware(app)
On Thursday, August 2, 2012 6:55:37 PM UTC-4, Kyle Finley wrote:
How do I restart the instance?
at appengine.google.com in the instance section you should see a list of
instance. They each have a Shutdown button
Yes it is in the root directory. I am stumped! I didn't think it was
working as I tried testing for different browsers and it didn't catch them.
I don't know, that's strange. I'ts working here:
http://anticurl.scotch-media.appspot.com/
And if you have included the file in your project, it should at the very
least stop *me* from using curl to access your site.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/1jEROOXORQMJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Why I am I getting these HEAD / HTTP/1.1 errors?
The problem is I am getting tens of thousands of these requests per day and they ARE impacting my quota and they come from different IPs tho most seem to be in Europe. They are all curl requests so they are not genuine. I'll have to move to a new ISP who can help me in blocking them as I can't afford this. My real requests are about .01% of these curl requests! On Thursday, August 2, 2012 6:45:04 PM UTC-4, Jay wrote: HEAD requests like this often come from social sites or search engines to test a url. This may or may not be the case for you, but that is my experience. Depending on the framework you are using , you should be able to handle HEAD requests if you want to. Handling the request, even if it errors does consume some resources, so there is likely some small impact to quota. If you don't want the systems hitting your site, probably the best option is robots.txt On Wednesday, August 1, 2012 3:02:02 PM UTC-5, Kate wrote: Thanks for that, but these requests are coming from many different ip addresses and I can see no pattern to them. I am just wondering why there are so many and if it is affecting my quota. Also I do not have a main.py. I have an index.py. I'm wondering now how it works at all! On Wednesday, August 1, 2012 3:36:55 PM UTC-4, Joshua Smith wrote: Looks to me like somebody is doing head requests to your app using curl. curl is a command line program that downloads web pages, similar to wget. This particular user is coming from linux, possibly from here: Portugal, Braga, Universidade do Minho (although reverse IP search results aren't always accurate). If I were you, I'd just ignore the errors. You aren't in business to make some college kid's half-baked web scraper work correctly. However, if you do want to handle the HEAD requests, you just add a method analogous to the post or get handlers you have implemented already in your main.py. -Joshua On Aug 1, 2012, at 3:26 PM, Kate mss.ka...@gmail.com wrote: I can't work out why I am getting these errors. In 13 hours I have 26,205 of them. I have no forms in my code. I do have urlfetch If as I have been told the error means that the head method is noot implemented, how do I set up my app to support the Head methiod? I cannot find anywhere I am using it, Here is an example of the error message. 2012-08-01 14:25:52.668 / 405 51ms 0kb curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3 libidn/0.6.14 libssh2/0.18 193.136.19.13 - - [01/Aug/2012:12:25:52 -0700] HEAD / HTTP/1.1 405 124 - curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0zlib/1.2.3 libidn/0.6.14 libssh2/0.18 aussieclouds.appspot.com ms=51 cpu_ms=0 api_cpu_ms=0 cpm_usd=0.45 instance=00c61b117c01c1a2dec223c75dcfeaab60fbd7 Kate -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/9NUDM-mRDTcJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] HEAD request not supported
I am getting the error bellow which I understand refers to an unsupported HEAD request. However the only form I have is a google search form. So I do not understand why I am getting this error. What should I look for? 128.227.150.12 - - [01/Aug/2012:11:17:36 -0700] HEAD / HTTP/1.1 405 124 - curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3 libidn/0.6.14 libssh2/0.18 aussieclouds.appspot.com ms=13 cpu_ms=21 api_cpu_ms=0 cpm_usd=0.000624 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/juBLpyKbGwoJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Why I am I getting these HEAD / HTTP/1.1 errors?
I can't work out why I am getting these errors. In 13 hours I have 26,205 of them. I have no forms in my code. I do have urlfetch If as I have been told the error means that the head method is noot implemented, how do I set up my app to support the Head methiod? I cannot find anywhere I am using it, Here is an example of the error message. 2012-08-01 14:25:52.668 / 405 51ms 0kb curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3 libidn/0.6.14 libssh2/0.18 193.136.19.13 - - [01/Aug/2012:12:25:52 -0700] HEAD / HTTP/1.1 405 124 - curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3 libidn/0.6.14 libssh2/0.18 aussieclouds.appspot.com ms=51 cpu_ms=0 api_cpu_ms=0 cpm_usd=0.45 instance=00c61b117c01c1a2dec223c75dcfeaab60fbd7 https://appengine.google.com/instances?app_id=aussiecloudsversion_id=1.360736682426762400key=00c61b117c01c1a2dec223c75dcfeaab60fbd7#00c61b117c01c1a2dec223c75dcfeaab60fbd7 Kate -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/TG2a3CId3bwJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Why I am I getting these HEAD / HTTP/1.1 errors?
Thanks for that, but these requests are coming from many different ip addresses and I can see no pattern to them. I am just wondering why there are so many and if it is affecting my quota. Also I do not have a main.py. I have an index.py. I'm wondering now how it works at all! On Wednesday, August 1, 2012 3:36:55 PM UTC-4, Joshua Smith wrote: Looks to me like somebody is doing head requests to your app using curl. curl is a command line program that downloads web pages, similar to wget. This particular user is coming from linux, possibly from here: Portugal, Braga, Universidade do Minho (although reverse IP search results aren't always accurate). If I were you, I'd just ignore the errors. You aren't in business to make some college kid's half-baked web scraper work correctly. However, if you do want to handle the HEAD requests, you just add a method analogous to the post or get handlers you have implemented already in your main.py. -Joshua On Aug 1, 2012, at 3:26 PM, Kate mss.ka...@gmail.com wrote: I can't work out why I am getting these errors. In 13 hours I have 26,205 of them. I have no forms in my code. I do have urlfetch If as I have been told the error means that the head method is noot implemented, how do I set up my app to support the Head methiod? I cannot find anywhere I am using it, Here is an example of the error message. 2012-08-01 14:25:52.668 / 405 51ms 0kb curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3 libidn/0.6.14 libssh2/0.18 193.136.19.13 - - [01/Aug/2012:12:25:52 -0700] HEAD / HTTP/1.1 405 124 - curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0zlib/1.2.3 libidn/0.6.14 libssh2/0.18 aussieclouds.appspot.com ms=51 cpu_ms=0 api_cpu_ms=0 cpm_usd=0.45 instance=00c61b117c01c1a2dec223c75dcfeaab60fbd7 Kate -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/ftljM_kj7qgJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Thousands of head requests to my app using curl
See example below. I am getting thousands of these per day and want to stop them. They are throwing HEAD / HTTP/1/1 errors. They come from different IP addresses. I'm stumped! 2012-08-01 14:25:52.668 / 405 51ms 0kb curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3 libidn/0.6.14 libssh2/0.18 193.136.19.13 - - [01/Aug/2012:12:25:52 -0700] HEAD / HTTP/1.1 405 124 - curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3 libidn/0.6.14 libssh2/0.18 aussieclouds.appspot.com ms=51 cpu_ms=0 api_cpu_ms=0 cpm_usd=0.45 instance=00c61b117c01c1a2dec223c75dcfeaab60fbd7 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Ey2V8HeuggQJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] How can I block curl requests
I am getting tens of thousands of curl requests - many thousands time browser requests and I want to block them. I'm using python. They coming from many different IPs most in Europe. If I can't stop them I will have to close my site or go to a new provider. Thank in advance, Kate -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/L7wDtd5rrUcJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] My site bombarded with thousands of URL requests from different IPs - how to stop them
I've posted about this problem before but I don't think I have explained it adequately. I am getting tens of thousands of CURL requests throwing HEAD method errors. This is thousands of times more than normal browser requests. I can't block them as they come from different IPs and you are only allowed to block 100 IPs. So - is there way of stopping them with code. If not, I can set up my site somewhere else but it seems a pity to have to do this. First I was concerned that these requests were throwing errors and it was suggested I enable the head method, but I can't see any point in doing this as the CURL requests are no no use to me. I have stopped being a billable site as I can't afford to support these requests. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Kc8kRRoDKKkJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] How can I block curl requests
Yes I can but there's a limit to the number - 100 I think, and there are tens of thousands of them. I guess I'll just have to move from GAE as there doesn't seem any way of doing this and I can't be paying for these requests. On Wednesday, August 1, 2012 4:51:04 PM UTC-4, rerngvit yanggratoke wrote: I think you can blacklist those IPs. Have a look in https://developers.google.com/appengine/docs/java/config/dos#About_dos_xmlor https://developers.google.com/appengine/docs/python/config/dos. On Wed, Aug 1, 2012 at 10:43 PM, Kate mss.ka...@gmail.com wrote: I am getting tens of thousands of curl requests - many thousands time browser requests and I want to block them. I'm using python. They coming from many different IPs most in Europe. If I can't stop them I will have to close my site or go to a new provider. Thank in advance, Kate -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/L7wDtd5rrUcJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Best Regards, Rerngvit Yanggratoke -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/X1_iYMXzpKEJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] My site bombarded with thousands of URL requests from different IPs - how to stop them
Thanks Joshue, How do I file a production issue? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/xPmxxyiQfPsJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] My site bombarded with thousands of URL requests from different IPs - how to stop them
Thanks. Done. Issue Issue 7916http://code.google.com/p/googleappengine/issues/detail?id=7916 On Wednesday, August 1, 2012 7:01:10 PM UTC-4, Joshua Smith wrote: http://code.google.com/p/googleappengine/issues/list New Issue Production Issue template Follow the instructions in the template On Aug 1, 2012, at 6:56 PM, Kate mss.ka...@gmail.com wrote: Thanks Joshue, How do I file a production issue? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/xPmxxyiQfPsJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/9vsWBdZec0AJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] How can I block curl requests
Yes Brandon, the user agent is the same, but how do I issue the denied message? Thanks, Kate -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/xF8LX7mt1v8J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] How to find the CAUSE of an error reported in the dashboard
My site appears to load OK but I am getting thousands of errors reported in the dashboard for /_ which I assume is the root directory. How can I find what is causing the error? Thanks in advance! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/AoFcTROWV6cJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Error message CURL ???
I am getting errors every few seconds being reported on my dashboard. However my pages load OK without error. When I delve from the error which is reported at /_ I get the following type of log entry. What can this mean? 012-07-30 07:49:22.860 / 405 68ms 0kb curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3 libidn/0.6.14 libssh2/0.18 128.233.252.12 - - [30/Jul/2012:07:49:22 -0700] HEAD / HTTP/1.1 405 124 - curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3 libidn/0.6.14 libssh2/0.18 aussieclouds.appspot.com ms=69 cpu_ms=0 api_cpu_ms=0 cpm_usd=0.46 instance=00c61b117cc74495d13a7c9bd0b786398b9deda2 https://appengine.google.com/instances?app_id=aussiecloudsversion_id=1.360685861039161880key=00c61b117cc74495d13a7c9bd0b786398b9deda2#00c61b117cc74495d13a7c9bd0b786398b9deda2 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/5x3ySkcl13oJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Error message CURL ???
The weird thing is, that this large number of errors 22119 in the dashboard summary is not at all reflected in the number of requests. And if I go to logs and do a search on errors I only get 20 errors! Why is the dashboard number so huge. Even the total number of requests is only round 200! (last 24 hours). On Monday, July 30, 2012 10:52:46 AM UTC-4, Kate wrote: I am getting errors every few seconds being reported on my dashboard. However my pages load OK without error. When I delve from the error which is reported at /_ I get the following type of log entry. What can this mean? 012-07-30 07:49:22.860 / 405 68ms 0kb curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3 libidn/0.6.14 libssh2/0.18 128.233.252.12 - - [30/Jul/2012:07:49:22 -0700] HEAD / HTTP/1.1 405 124 - curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0 zlib/1.2.3 libidn/0.6.14 libssh2/0.18 aussieclouds.appspot.com ms=69 cpu_ms=0 api_cpu_ms=0 cpm_usd=0.46 instance=00c61b117cc74495d13a7c9bd0b786398b9deda2 https://appengine.google.com/instances?app_id=aussiecloudsversion_id=1.360685861039161880key=00c61b117cc74495d13a7c9bd0b786398b9deda2#00c61b117cc74495d13a7c9bd0b786398b9deda2 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/q0Vk8oajZbQJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Huge difference (8,400%) between billing received and information in the dashboard
I'm searching for a very similar issue - I've been charge $2.10/week for no activity, and there is nothing on my billing that includes the $2.10/week minimum - is this something that applies to all app engine accounts? I can't find documentation of it anywhere in the information I've received. Thank you. Kate On Thursday, June 14, 2012 11:53:55 PM UTC-7, hugues2 wrote: Hi Greg, Thanks for your very fast feedback. Indeed I completely overlooked the weekly charge of 2.10$. Sorry for the big mistake. Hugues Le vendredi 15 juin 2012 08:30:46 UTC+2, hugues2 a écrit : Gi, Has anyone the same issue ? -- In the admin dashboard, I see that I should be billed 0,03 $ (see below, Annex 1) -- I received a billing for 2,54$ (see below, Annex 2) == Difference of : 8,400% Also the number of instance hours has a 100% difference between the dashboard (15.62) and the billing (33.42) == Which information is the correct one ? *Annex 1 : Dashboard :* Frontend Instance Hours15.62 Instance Hours0.00$0.08/ Hour$0.00Backend Instance Hours0.00 Instance Hours0.00$0.08/ Hour$0.00Datastore Stored Data0.04 GBytes0.00$0.008/ GByte-day$0.00Logs Stored Data0.00 GBytes0.00$0.008/ GByte-day$0.00Task Queue Stored Task Bytes0.00 GBytes0.00$0.008/ GByte-day$0.00Blobstore Stored Data0.02 GBytes0.00$0.0043/ GByte-day$0.00Datastore Write Operations0.03 Million Ops0.00$1.00/ Million Ops$0.00Datastore Read Operations0.09 Million Ops0.04$0.70/ Million Ops$0.03Datastore Small Operations0.00 Million Ops0.00$0.10/ Million Ops$0.00Outgoing Bandwidth0.01 GBytes0.00$0.12/ GByte$0.00Recipients Emailed10$0.01/ 100 Recipients$0.00Stanzas Sent00$0.10/ 100K Stanzas$0.00Channels Created [image: 0%] 0%0 of 95,0400$0.01/ 100 Opens$0.00Estimated cost for the last 23 hours: [image: 1%] *$0.03* / $2.00* *Annex 2 : Billing : * ResourceUsedCharge*Frontend Instance Hours* $0.08/Hour33.48$0.00*Discounted Instance Hour* $0.05/Hour0.00$0.00*Unused Discounted Instance Hour* $0.05/Hour0.00$0.00*Backend Instance Hours* $0.08/Hour0.00$0.00*Datastore Storage* $0.008/GByte-day0.30$0.00*Logs Storage* $0.008/GByte-day0.03$0.00*Taskqueue Storage* $0.008/GByte-day0.00$0.00*Blobstore Storage* $0.0043/GByte-day0.14$0.00*Datastore Writes* $1.00/Million Ops0.02$0.00*Datastore Reads* $0.70/Million Ops0.05$0.00*Small Datastore Operations* $0.10/Million Ops0.01$0.00*Bandwidth Out* $0.12/GByte0.03$0.00*Emails* $0.01/100 Recipients0.05$0.00*XMPP Stanzas* $0.10/100K Stanzas0.00$0.00*Opened Channels* $0.01/100 Opens0.00$0.00*Minimum Spend* $2.10/Week$2.10*Total:*$2.10 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/BRuuZnpE5KAJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Confusion re Idle Instances and Pending Latency Options
I am confused. I have a low volume (in terms of requests) site. At the Applications Settings page there are two sliders. I can understand the top one and have set the number of instances v low. But on researching latency issues some articles imply to set minimum latency high, others to set it low. Can someone please help me out here. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Still getting hits (instances) on deleted page
I had add url pages that were getting a lot of hits but most were unwanted from some spammers. As this was costing me $$$ with the new pricing, I deleted the pages and set the link to point to an external URL. However I am stil getting hits recorded for my addurl page as it is still loaded in order to redirect. If I take out the redirect so that I would expect a 404, there is still a redirect, this time to my home page. How can I force a 404? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: We're down hard because the payments system is failing! HELP!
I am having a similar problem. I cannot change my max $ amount. It tells me to email a google billing address which I did. 48 hours ago. To date no answer ... On Nov 14, 6:46 pm, Ikai Lan (Google) ika...@google.com wrote: I just checked on your app and it looks okay to me. For anyone else having issues with this, the best place to get help is to email the address provided in the mass email that's also available on the billing history page. It should be appengine_update_pricing at google. We're working through this queue as quickly as we can to help folks transition to the new billing. -- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com | twitter.com/ikai On Tue, Nov 15, 2011 at 9:40 AM, Mike Prince m...@mikeprince.com wrote: For the app yumyumlabs, we ran an entity delete process on 4m records and it ended up eating $40 of our budget in a few minutes (36M database writes!) When we tried to increase our budget the payments system is failing with: Hello michael prince, Google Services has cancelled your order. You have not been charged. Reason: Other (describe below) Comments from Google Services: This inactive subscription has been canceled Order date: Nov 14, 2011 1:39 PM PST Google order number: 810629956315305 Shipping Status Qty Item Price Cancelled Items 1 App Engine Budget - Maximum weekly budget for application 'yumyumlabs', including applicable tax. $0.00 Tax (CA) : $0.00 Total : $0.00 now $740.00 per week Order cancelled - Your card was not charged Now all our requests are failing and the billing system keeps rejecting our payments! This system supports a featured app on android with up to 30k new users a day. This is bad. HELP! Thanks, Mike -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Charged for style sheets?
Thanks for that. Another related question A static file handler associates a URL pattern with paths to static files uploaded with the application. Lets say I have the following directories /stylesheets /static/images /static/images/ads Can I do the following - url: /images static_dir: static/images - url: /images/ads static_dir: static/images/ads /*that is do subdirectories have the be explicitly listed - url /stylesheets static_dir /stylesheets /* that is can you have more than one virtual static_dir? */ On Nov 15, 11:41 am, Barry Hunter barrybhun...@gmail.com wrote: Set caching headers on the file, so that every page doesnt have to load it http://code.google.com/appengine/docs/python/config/appconfig.html#St... http://code.google.com/appengine/docs/python/gettingstarted/staticfil... Static files dont use instances. You still of course pay for bandwidth On Tue, Nov 15, 2011 at 4:36 PM, Kate mss.ka...@gmail.com wrote: I have a css file that loads for every page. It shows up as a request in my billing summary. Are we charged for this? Is it an instance? If so, I will just include it in each page! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Static files question
Lets say I have the following directories /stylesheets /static/images /static/images/ads Can I do the following - url: /images static_dir: static/images - url: /images/ads static_dir: static/images/ads /*that is do subdirectories have the be explicitly listed - url /stylesheets static_dir /stylesheets /* that is can you have more than one virtual static_dir? */ -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Instances questions - static files cost $$$!
Even though I have complied with the python definition for my static files diectoroes the static files (style sheets, images) are STILL showinfg as instance requests on my dashboard! Due to the way estimates are calculated, the sum of individual resource costs may not match the displayed total exactly. and then the requests are listed, including my images and stylesheets directories declared as - url: /stylesheets static_dir: templates/stylesheets - url: /static/images static_dir: static/images - url: /static static_dir: static - url: /robots.txt static_files: robots.txt upload: robots.txt WHY is this??? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Static files question
Thanks Kylie, but why are these files still showing as requests on my summary page? On Nov 15, 2:44 pm, Kyle Finley kylefin...@gmail.com wrote: - url: /images static_dir: static/images *Yes* - url: /images/ads static_dir: static/images/ads /* that is do subdirectories have the be explicitly listed *No, all subdirectories will be included:* *images/ads* *images/cats* - url /stylesheets static_dir /stylesheets /* that is can you have more than one virtual static_dir? */ *Yes* - Kyle -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: New pricing structure infinity increase
But that article says to DECREASE latency whereas the two sliders in the applications seyting page would indicate that you should INCREASE latency. Now I am confused! On Nov 14, 1:34 am, Gregory D'alesandre gr...@google.com wrote: Hi Kate, check out the article that we wrote about optimizing your app to minimize costs:http://code.google.com/appengine/articles/managing-resources.html Without looking at your app its hard to know what the right settings should be although you are only charged for the number of idle instances up to the max you set as you Max Idle Instances. Max Idle Instances also controls how quickly the scheduler tears instances down for you though, so it you set it to 1, it means that the scheduler will aggressively take down any additional instances which could impact your performance. Greg On Sun, Nov 13, 2011 at 3:42 PM, Kate mss.ka...@gmail.com wrote: I AM getting charged for instances! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/2ZHYpc8KRU8J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] New pricing structure infinity increase
I cannot work out how to set my instances and latency settings but I need to do something. Until 3 days ago my site cost nothing. Then 0,42 0.33 0.37 0.86 4.80 How can this be? I had not altered any settings and I had not uploaded any new pages. Certainly I cannot be paying $35 per week for a small site that has only 1600 requests per day. Can anyone tell me what values I should have, min and max for the lyle instances and pending latency sliders? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Disable billing - is this a silly question?
Why WOULD'T a person disable billing. My site cost nothing till a few days ago. Should I try it? I know this sounds a very silly question, but why would I go from 0 to several dollars a day. If I turn off billing what will happen? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Default page ... 404 error redirecting and I want the 404 response
I recently deleted a page from my site as it was getting too many inappropriate hits. I also deleted al links to it but still there are many attempted hits coming. When the page is requested the request is redirected to the homepage. I do not want this and cannot remember how I set it up. Anyone have any ideas as to how I can force anyone requesting this non existent page to get a 404. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: New pricing structure infinity increase
I AM getting charged for instances! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/2ZHYpc8KRU8J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Default page ... 404 error redirecting and I want the 404 response
I worked it out
Using a handler
class addURLHandler(webapp.RequestHandler):
def get(self):
path = self.request.path
if doRender(self,path):
return
self.response.out.write(htmlheadmeta http-equiv=\refresh\
content=\0;url=%s\/headbody/body/html % ('redirectURL',))
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/XaSyNSSgcBUJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] My site no longer working
I haven't touched a thing, yet suddenly my site throws an error when the url does not contain the filename. e.g. http://www.coolabah.com gives a server error while http://www.coolabah.com/index.htm works fine. Can anyone help me! The error is The website cannot display the page HTTP 500 Most likely causes: •The website is under maintenance. •The website has a programming error. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: My site no longer working
This is what I am getting. Traceback (most recent call last): File C:\google_appengine\google\appengine\ext\webapp\_webapp25.py, line 701, in __call__ handler.get(*groups) File C:\wwwhome\aussieclouds\index.py, line 396, in get doRender(self,'index.htm') I read the blogs and found that a new version of the SDK was deployed yesterday. So I downloaded the new version to help me troubleshoot but it was of no help. I do not have any idea as to how to proceed with this! On Oct 12, 8:25 pm, Nick Johnson nickjohn...@google.com wrote: Hi Kate, Your app appears to be returning a 500 server error (with no text in the body) when fetching the base URL (/) of your app. It's impossible to say why this is without seeing your code and logs. Have you checked the logs in the App Engine admin console to see what is being logged for these requests? -Nick Johnson On Thu, Oct 13, 2011 at 10:42 AM, Kate mss.ka...@gmail.com wrote: I haven't touched a thing, yet suddenly my site throws an error when the url does not contain the filename. e.g. http://www.coolabah.comgives a server error while http://www.coolabah.com/index.htmworks fine. Can anyone help me! The error is The website cannot display the page HTTP 500 Most likely causes: •The website is under maintenance. •The website has a programming error. -- You received this message because you are subscribed to theGoogleGroups GoogleApp Engine group. To post to this group, send email togoogle-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Nick Johnson, Developer Programs Engineer, App Engine -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: My site no longer working
This is the code that now fails - though it has worked for
18months
class MainHandler(webapp.RequestHandler):
def get(self):
path = self.request.path
isgeneric=path.rfind('generic')
isinformation=path.rfind('information')
isexpat=path.rfind('expat')
temp = os.path.join (
os.path.dirname(__file__),
'templates' + path)
if doRender(self,path):
return
doRender(self,'index.htm')
On Oct 12, 8:25 pm, Nick Johnson nickjohn...@google.com wrote:
Hi Kate,
Your app appears to be returning a 500 server error (with no text in the
body) when fetching the base URL (/) of your app. It's impossible to say why
this is without seeing your code and logs. Have you checked the logs in the
App Engine admin console to see what is being logged for these requests?
-Nick Johnson
On Thu, Oct 13, 2011 at 10:42 AM, Kate mss.ka...@gmail.com wrote:
I haven't touched a thing, yet suddenly my site throws an error when
the url does not contain the filename.
e.g.
http://www.coolabah.comgives a server error while
http://www.coolabah.com/index.htmworks fine.
Can anyone help me!
The error is
The website cannot display the page
HTTP 500
Most likely causes:
•The website is under maintenance.
•The website has a programming error.
--
You received this message because you are subscribed to theGoogleGroups
GoogleApp Engine group.
To post to this group, send email togoogle-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
Nick Johnson, Developer Programs Engineer, App Engine
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: My site no longer working
This is the code that now fails - though it has worked for
18months
class MainHandler(webapp.RequestHandler):
def get(self):
path = self.request.path
isgeneric=path.rfind('generic')
isinformation=path.rfind('information')
isexpat=path.rfind('expat')
temp = os.path.join (
os.path.dirname(__file__),
'templates' + path)
if doRender(self,path):
return
doRender(self,'index.htm')
On Oct 12, 8:25 pm, Nick Johnson nickjohn...@google.com wrote:
Hi Kate,
Your app appears to be returning a 500 server error (with no text in the
body) when fetching the base URL (/) of your app. It's impossible to say why
this is without seeing your code and logs. Have you checked the logs in the
App Engine admin console to see what is being logged for these requests?
-Nick Johnson
On Thu, Oct 13, 2011 at 10:42 AM, Kate mss.ka...@gmail.com wrote:
I haven't touched a thing, yet suddenly my site throws an error when
the url does not contain the filename.
e.g.
http://www.coolabah.comgives a server error while
http://www.coolabah.com/index.htmworks fine.
Can anyone help me!
The error is
The website cannot display the page
HTTP 500
Most likely causes:
•The website is under maintenance.
•The website has a programming error.
--
You received this message because you are subscribed to theGoogleGroups
GoogleApp Engine group.
To post to this group, send email togoogle-appeng...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
Nick Johnson, Developer Programs Engineer, App Engine
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: My site no longer working
my site was about 5 days ago. All was fine then. How can I see the live log? On Oct 12, 8:36 pm, Nick Johnson nickjohn...@google.com wrote: Hi Kate, You haven't included the complete stacktrace, including the all important last line that tells us what actually went wrong. Can you paste the whole thing, please? -Nick Johnson On Thu, Oct 13, 2011 at 11:35 AM, Kate mss.ka...@gmail.com wrote: This is what I am getting. Traceback (most recent call last): File C:\google_appengine\google\appengine\ext\webapp\_webapp25.py, line 701, in __call__ handler.get(*groups) File C:\wwwhome\aussieclouds\index.py, line 396, in get doRender(self,'index.htm') I read the blogs and found that a new version of the SDK was deployed yesterday. So I downloaded the new version to help me troubleshoot but it was of no help. I do not have any idea as to how to proceed with this! On Oct 12, 8:25 pm, Nick Johnson nickjohn...@google.com wrote: Hi Kate, Your app appears to be returning a 500 server error (with no text in the body) when fetching the base URL (/) of your app. It's impossible to say why this is without seeing your code and logs. Have you checked the logs in the App Engine admin console to see what is being logged for these requests? -Nick Johnson On Thu, Oct 13, 2011 at 10:42 AM, Kate mss.ka...@gmail.com wrote: I haven't touched a thing, yet suddenly my site throws an error when the url does not contain the filename. e.g. http://www.coolabah.comgivesa server error while http://www.coolabah.com/index.htmworksfine. Can anyone help me! The error is The website cannot display the page HTTP 500 Most likely causes: •The website is under maintenance. •The website has a programming error. -- You received this message because you are subscribed to theGoogleGroups GoogleApp Engine group. To post to this group, send email togoogle-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Nick Johnson, Developer Programs Engineer, App Engine -- You received this message because you are subscribed to theGoogleGroups GoogleApp Engine group. To post to this group, send email togoogle-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Nick Johnson, Developer Programs Engineer, App Engine -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: My site no longer working
my site was about 5 days ago. All was fine then. How can I see the live log? On Oct 12, 8:36 pm, Nick Johnson nickjohn...@google.com wrote: Hi Kate, You haven't included the complete stacktrace, including the all important last line that tells us what actually went wrong. Can you paste the whole thing, please? -Nick Johnson On Thu, Oct 13, 2011 at 11:35 AM, Kate mss.ka...@gmail.com wrote: This is what I am getting. Traceback (most recent call last): File C:\google_appengine\google\appengine\ext\webapp\_webapp25.py, line 701, in __call__ handler.get(*groups) File C:\wwwhome\aussieclouds\index.py, line 396, in get doRender(self,'index.htm') I read the blogs and found that a new version of the SDK was deployed yesterday. So I downloaded the new version to help me troubleshoot but it was of no help. I do not have any idea as to how to proceed with this! On Oct 12, 8:25 pm, Nick Johnson nickjohn...@google.com wrote: Hi Kate, Your app appears to be returning a 500 server error (with no text in the body) when fetching the base URL (/) of your app. It's impossible to say why this is without seeing your code and logs. Have you checked the logs in the App Engine admin console to see what is being logged for these requests? -Nick Johnson On Thu, Oct 13, 2011 at 10:42 AM, Kate mss.ka...@gmail.com wrote: I haven't touched a thing, yet suddenly my site throws an error when the url does not contain the filename. e.g. http://www.coolabah.comgivesa server error while http://www.coolabah.com/index.htmworksfine. Can anyone help me! The error is The website cannot display the page HTTP 500 Most likely causes: •The website is under maintenance. •The website has a programming error. -- You received this message because you are subscribed to theGoogleGroups GoogleApp Engine group. To post to this group, send email togoogle-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Nick Johnson, Developer Programs Engineer, App Engine -- You received this message because you are subscribed to theGoogleGroups GoogleApp Engine group. To post to this group, send email togoogle-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Nick Johnson, Developer Programs Engineer, App Engine -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: My site no longer working
Nick, I have my site up now - thank you. However I cannot see WHICH line was causing the problem. Can you point it out to me? I only got it working by deleting all XML calls. All of them appeared to be fine, but I couldn't take a chance. It would really help me if you could let me know how you zeroed in on the problem! On Oct 12, 9:04 pm, Kate mss.ka...@gmail.com wrote: Nick, thanks for the quick reply. Here is the log of when I run it locally. 2011-10-12 20:04:27 Running command: ['C:\\Python27\\pythonw.exe', 'C: \\google_appengine\\dev_appserver.py', '--admin_console_server=', '-- port=8080', u'C:\\wwwhome\\aussieclouds'] Warning: You are using a Python runtime (2.7) that is more recent than the production runtime environment (2.5). Your application may use features that are not available in the production environment and may not work correctly when deployed to production. INFO 2011-10-13 00:04:35,358 rdbms_sqlite.py:58] Connecting to SQLite database '' with file 'c:\\docume~1\\kjuliff\\locals~1\\temp\ \dev_appserver.rdbms' WARNING 2011-10-13 00:04:35,578 dev_appserver.py:4648] Could not initialize images API; you are likely missing the Python PIL module. ImportError: No module named _imaging INFO 2011-10-13 00:04:35,967 dev_appserver_multiprocess.py:637] Running application aussieclouds on port 8080:http://localhost:8080 WARNING 2011-10-13 00:05:21,046 __init__.py:97] You are using the default Django version (0.96). The default Django version will change in an App Engine release in the near future. Please call use_library() to explicitly select a Django version. For more information seehttp://code.google.com/appengine/docs/python/tools/libraries.html#Django ERROR 2011-10-13 00:05:23,390 _webapp25.py:464] no element found: line 1, column 0 Traceback (most recent call last): File C:\google_appengine\google\appengine\ext\webapp\_webapp25.py, line 701, in __call__ handler.get(*groups) File C:\wwwhome\aussieclouds\index.py, line 396, in get doRender(self,'index.htm') File C:\wwwhome\aussieclouds\index.py, line 111, in doRender theevents=frontpage.getevents() File C:\wwwhome\aussieclouds\frontpage.py, line 240, in getevents dom = parse(http://www.gumnut.com/clouds/getevents.cfm;) File C:\wwwhome\aussieclouds\frontpage.py, line 229, in parse return minidom.parseString(r.content) File C:\Python27\lib\xml\dom\minidom.py, line 1924, in parseString return expatbuilder.parseString(string) File C:\Python27\lib\xml\dom\expatbuilder.py, line 940, in parseString return builder.parseString(string) File C:\Python27\lib\xml\dom\expatbuilder.py, line 223, in parseString parser.Parse(string, True) ExpatError: no element found: line 1, column 0 INFO 2011-10-13 00:05:23,405 dev_appserver.py:4143] GET / HTTP/ 1.1 500 - INFO 2011-10-13 00:05:23,421 dev_appserver_index.py:255] Updating C:\wwwhome\aussieclouds\index.yaml INFO 2011-10-13 00:07:26,358 dev_appserver.py:4143] GET /_ah/ admin HTTP/1.1 302 - INFO 2011-10-13 00:07:26,500 dev_appserver.py:4143] GET /_ah/ admin/datastore HTTP/1.1 200 - INFO 2011-10-13 00:07:26,515 dev_appserver_index.py:255] Updating C:\wwwhome\aussieclouds\index.yaml INFO 2011-10-13 00:07:26,703 dev_appserver.py:4143] GET /_ah/ admin/images/google.gif HTTP/1.1 200 - 2011-10-12 20:09:43 (Process exited with code -1) 2011-10-12 20:09:47 Running command: ['C:\\Python27\\pythonw.exe', 'C: \\google_appengine\\dev_appserver.py', '--admin_console_server=', '-- port=8080', u'C:\\wwwhome\\aussieclouds'] Warning: You are using a Python runtime (2.7) that is more recent than the production runtime environment (2.5). Your application may use features that are not available in the production environment and may not work correctly when deployed to production. INFO 2011-10-13 00:09:50,842 rdbms_sqlite.py:58] Connecting to SQLite database '' with file 'c:\\docume~1\\kjuliff\\locals~1\\temp\ \dev_appserver.rdbms' WARNING 2011-10-13 00:09:50,905 dev_appserver.py:4648] Could not initialize images API; you are likely missing the Python PIL module. ImportError: No module named _imaging INFO 2011-10-13 00:09:50,983 dev_appserver_multiprocess.py:637] Running application aussieclouds on port 8080:http://localhost:8080 WARNING 2011-10-13 00:10:32,078 __init__.py:97] You are using the default Django version (0.96). The default Django version will change in an App Engine release in the near future. Please call use_library() to explicitly select a Django version. For more information seehttp://code.google.com/appengine/docs/python/tools/libraries.html#Django INFO 2011-10-13 00:10:32,203 dev_appserver.py:4143] GET /_ah/ admin HTTP/1.1 302 - INFO 2011-10-13 00:10:32,203 dev_appserver_index.py:255] Updating C:\wwwhome\aussieclouds\index.yaml INFO 2011-10-13 00:10:32,250 dev_appserver.py:4143] GET /_ah/ admin HTTP/1.1 302 - INFO
[google-appengine] 24 hours in SF: A Geolocation App
The link to the source code is incorrect. Can someone direct me to the repository for this code. The article at http://code.google.com/appengine/articles/geosearch.html just points to the index summary of apps. I can't find 24 hours in SF: A Geolocation App there. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] two applications two domains - please help
OK. I have a number of applications. I successfully set up one to be accessed via one of my domains. Now I have a second app I want to map top a different domain. I'm stumped. If I have one domain with a cname rcord pointing to ghs.google.com and another pointing to ghs.google.com, how does google know WHICH app it's meant to bring up? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
