[google-appengine] Re: Some information in spanish?
Hello, César. Please, make a question about app.yaml and I will try to answer in simple English y añadir alguna nota en Español. Best, saludos. Javier On Wednesday, April 22, 2015 at 4:17:23 PM UTC+2, César Mejías Narciso wrote: Hi everybody! I'm looking for some information in spanish about how can I configure de app.yaml, due to my english isn't fluid to understand everythings. Thanks a lot -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/f433d4af-bdba-4944-9fa1-9043751eed0e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] So many spam bots are hitting my website hosted on Google App Engine
What rate are these queries arriving? Are they all from the same IP address? Are they scanning your site or hitting one or two pages over and over? One request is not useful. Assuming this is just a poorly behaved bot and not a DOS attack, the simplest solution is to install a servlet filter at the top of your stack. Inspect the request and if you don't like something about it (IP address, user agent, etc) and return a blank page (or goatse, or whatever). Short of a real DDOS, this will convert your expensive 1100ms requests into almost-free 10ms requests and mitigate the issue. Jeff On Wed, Apr 22, 2015 at 11:42 PM, Ashutosh Mishra ashutosh.narayan3...@gmail.com wrote: Hi Jeff, Thanks for your harsh word suggestion, Please have a look to attached log file snap shot you can have IP of the ahref bot, it use to regularly coming on my site. I think this real information will be enough, so please let me know concrete solution to overcome this problem. On Thursday, April 23, 2015 at 11:43:08 AM UTC+5:30, Jeff Schnitzer wrote: I'm calling bullshit. You have a website developed on GAE/Java but you don't understand what .htaccess is and why it doesn't apply? If you're having problems with your website, why don't you ask the people who developed it? I don't get it. The advice you have been offered here (all of which is reasonable) requires more technical sophistication than you exhibit. Possibly this is a bot doing normal things. Possibly this is a real DOS attack of some kind. Post some real information like IP addresses and the actual rate of requests, and maybe we can help you with an appropriate mitigation strategy. You have said a bunch of technically dumb things with an accusatory tone of voice (spam bots are attacking me!). This happens a lot, and usually it means _you_ just screwed something up. If you want help, post more information and be less arrogant about it. You don't know what you think you know. Jeff On Wed, Apr 22, 2015 at 7:08 AM, Barry Hunter barryb...@gmail.com wrote: Have you cross checked the IP(s) of the bot? The User-Agent is easily spoofed, it might be some other bot just pertending to be a ahrefbot. Regardless, as already mentioned can put handlers in your code to 'trap' bad actors. Check the useagent, and do something different. (can't totally block this way, but can minimise damage -make the requests very quick/short. And by not returning further links, stop them finding yet more pages to index). ... or use an external service to 'firewall' such requests - as already mentioned Cloudflare offer this. On 22 April 2015 at 15:02, Ashutosh Mishra ashutosh.n...@gmail.com wrote: Hi Vinny, thanks for your comment I have done the changes in myhotelcar.com/bobots.txt file as you have mentioned but issue is still not resolved as per my analysis the bots hiiting specifically ahref has increased day by day an now issue seems critical. Please hep me to get out of this situation. I will happy to have your advice on this. On Tuesday, April 21, 2015 at 10:07:28 PM UTC+5:30, Vinny P wrote: On Mon, Apr 20, 2015 at 11:32 PM, Ashutosh Mishra ashutosh.n...@gmail.com wrote: I have also searched so many thing and I found the Ahref bot doesn't obey robots principal. Many people has suggested that I can prohibit them via htaccess file, I don't want to use that way as in google app engine hosting I didn't find htaccess file. So please provide me any way to filter out these spam bots. The .htaccess file isn't supported in App Engine. If this is the real Ahref bot, it should support robots.txt. I looked in your robots.txt file: I see you disallowing Baidu, Yandex and a wildcard disallow, but not specifically ahrefbot. Try adding the following to your robots file: *user-agent: AhrefsBot* *disallow: /* According to the ahrefbot robot page, you can also email them directly to ask them to stop; see https://ahrefs.com/robot On Mon, Apr 20, 2015 at 11:36 PM, Ashutosh Mishra ashutosh.n...@gmail.com wrote: I think you have picked the issue correctly they are hitting particular set of pages regularly hotel pages which were dynamically generated, you are correct about rss and sitemap feed. So please tell me the way to overcome this issue as these spam bots specially ahref bot is consuming my server bandwidth a lot un-necessarily. I want a good solution so that I will not face any spam bot hurdle in future. This happens to a lot of websites with a large set of dynamically generated pages. Honestly the best solution would be to sign up for Cloudflare ( https://www.cloudflare.com/google ) and use their tools to help filter incoming traffic. You can also do what Barry suggested earlier, and start blocking the IPs that ahrefsbot is using. If you're willing to do some coding, you can write a filter into your application to check for the useragent and kick back a 429 HTTP status code (Too Many
[google-appengine] mysql not able to java project deployed in google compute engine
I have created a compute engine instance and have installed MySQL also authorized to cloud sql instance. don`t know what other settings to be done using ssl, searched a lot but not getting the exact settings to be done in VM instance to connect my java project to mysql. Initially getting this error: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure The last packet sent successfully to the server was 0 milliseconds ago. The driver has not received any packets from the server. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/65206750-efcd-45f7-8a59-5731ac2bb8f0%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[google-appengine] FeatureNotEnabledError on Managed VM for Image Service
Hi all, Wondering if anyone knows what the problem is here. Runtime: Python I have just changed a module over to Managed VMs, and it uses the Images Python API (https://cloud.google.com/appengine/docs/python/images/). The module was working fine before the change (been running for almost one year), but now on Managed VMs I get this error (in production only, dev server is working fine): *The API call images.Transform() is currently not enabled.* Traceback (most recent call last): File /home/vmagent/python_vm_runtime/lib/webapp2-2.5.2/webapp2.py, line 1529, in __call__ rv = self.router.dispatch(request, response) File /home/vmagent/python_vm_runtime/lib/webapp2-2.5.2/webapp2.py, line 1278, in default_dispatcher return route.handler_adapter(request, response) File /home/vmagent/python_vm_runtime/lib/webapp2-2.5.2/webapp2.py, line 1102, in __call__ return handler.dispatch() File /home/vmagent/app/some/file/path.py, line 333, in dispatch result = super(Controller, self).dispatch() File /home/vmagent/python_vm_runtime/lib/webapp2-2.5.2/webapp2.py, line 572, in dispatch return self.handle_exception(e, self.app.debug) File /home/vmagent/python_vm_runtime/lib/webapp2-2.5.2/webapp2.py, line 570, in dispatch return method(*args, **kwargs) File /home/vmagent/app/some/other/file.py, line 113, in get ret = self. some_method(self.ai, **options) File /home/vmagent/app/some/other/file.py, line 888, in some_method tsr=tsr) File /home/vmagent/python_vm_runtime/google/appengine/api/images/__init__.py, line 800, in execute_transforms return rpc.get_result() File /home/vmagent/python_vm_runtime/google/appengine/api/apiproxy_stub_map.py, line 613, in get_result return self.__get_result_hook(self) File /home/vmagent/python_vm_runtime/google/appengine/api/images/__init__.py, line 885, in execute_transforms_hook rpc.check_success() File /home/vmagent/python_vm_runtime/google/appengine/api/apiproxy_stub_map.py, line 579, in check_success self.__rpc.CheckSuccess() File /home/vmagent/python_vm_runtime/google/appengine/ext/vmruntime/vmstub.py, line 311, in _WaitImpl raise self._TranslateToError(parsed_response) File /home/vmagent/python_vm_runtime/google/appengine/ext/vmruntime/vmstub.py, line 164, in _TranslateToError raise self._ErrorException(exception_type, msg) *FeatureNotEnabledError: The API call images.Transform() is currently not enabled.* Billing is enabled on this project/app. Any Googlers who have been recently active on this list have any idea? Do I need to request the Images API to be switched on for my Managed VM module? I can PM the app, module and version if needed. Thanks, Nick -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/b1bfc38c-afe6-447a-bdfc-bd1e0184b95b%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Google Cloud Platform wants to hear from you
Chris - with the improvements you're suggesting to Managed VMs (20 sec deploys, 1 sec instance startup time) - will it be recommended to use Managed VMs to serve frontend traffic? Right now they seem to be more targetted to backend processing because of the slow scaling. If this is true, and you are targeting it to serve front end traffic, does that mean we'll be able to deploy Managed VMs to our default modules, perform traffic splitting, access the logging api's and realtime api's, etc? These are the things that have been traditionally missing from managed vms. On Wednesday, April 22, 2015 at 10:33:46 AM UTC-7, Chris Ramsdale wrote: On Wed, Apr 22, 2015 at 2:21 AM, troberti tij...@firigames.com javascript: wrote: What a flurry of activity. :) Great to see. I have been using GCP( App Engine + BigQuery) in total for over 5 years, so not new, but I have seen plenty of new users make mistakes so let me chime in a bit: On App Engine (and GCP) there are a lot of ways to approach a problem, with the consequence that is very easy to choose the wrong solution. There is actually a rather steep learning curve to just know what is available. This is a problem, because the differences between various solutions can result in an order of magnitude difference in costs/latency/complexity etc. I stopped counting the amount of times I have seen models with every property indexed, resulting in huge datastore costs. Or where someone tries to put tons of data in the Datastore while BigQuery would be a much better fit for the problem. Every time this happens, the new user ends up disappointed. So guiding new users in the right direction when starting out on GCP seems very important. I agree with a lot in Karl's post, and especially the Roadmap. It doesn't need to be about features, but big ticket items like Python 3, Java 8, SSL etc should be communicated. It doesn't have to be an explicit list somewhere, just a PM chiming in regularly should be good enough. Now back to App Engine: Like I said in my post on the other thread https://cloud.google.com/appengine/forum/?place=msg%2Fgoogle-appengine%2FqTyc2E-0IXc%2FPxUzFMXhzjQJ, the trend towards managed VMs worries me a bit. For us, zero-configuration/no-maintenance is not just another feature of App Engine, it is one of the most important ones! We want to write our programs and then keep them running for years (5+) without having to do *anything*. Some of our apps are running like this for years now, and I want to ensure that this stays possible in the future. Managed VMs represent a new hosting environment that brings with it a set of feature benefits -- open source compatible runtimes, more CPU / memory configurations, access to native resources such as a file system and network stack. we'll be investing in this environment more and more over the coming months (we're ripping docker out of the getting started flow, getting deployment times to 20 secs, getting instance activation time to 1 sec, adding scale to/from zero instances, etc.) that said, don't worry, we'll absolutely keep your existing v1 apps running just as they have for years. Glad to see the Docker fad go, but please don't replace it with something where I need to choose my technology stack in some way. Just provide a set of stable APIs instead so we can consider everything else an implementation detail for App Engine to worry about :) exactly. if anything, we're going to make those same APIs available from other compute environments (e.g. Compute Engine and Container Engine). On Wednesday, April 22, 2015 at 8:31:47 AM UTC+2, Chris Ramsdale wrote: On Tue, Apr 21, 2015 at 11:22 PM, Vinny P vinn...@gmail.com wrote: Hi Katie, I think Karl's post hit a home run and I'm happy to see the positive response to his post. Let me just tack on a few items: *Managed VMs:* The development toolchain for Managed VMs can be a bit finicky. To be quite honest I have no idea how I got Managed VMs working on my laptop. Streamlining this would be a huge benefit to me, and probably a lot of first-timers. If you can convince one of the online IDE services to simplify creating Managed VM GAE apps, that would be super. For smaller or toy apps within Managed VMs: I shouldn't need to care about the Docker container running the application; I should be able to create an application using just Eclipse + Google Plugin, then be able to deploy straight to a Managed VM runtime *without* the intermediate step of having gcloud create and store a dockerfile https://cloud.google.com/appengine/docs/managed-vms/tutorial/step2#dockerfile . We're removing the Docker toolchain from the mix and will be surfacing a hosted build service that handles this on your behalf. Their toolchain is simply unstable. *Firebase:* I'm glad that Google bought up Firebase - they have a lot of great ideas
[google-appengine] Help : Share Datastore of App Engine with another app engine project
Anyone know of any implementation or api own google app engine of integration the datastore of the projects Google App Engine? Not using Web Services! Is it possible to make this integration datastore? Is there any API -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/672a6e37-1c0c-45df-b792-bfbb0e7e9498%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] So many spam bots are hitting my website hosted on Google App Engine
Hi Jeff, Thanks for your solution, but I can not go with filter, as filter will also increase cost and We are doing this only to reduce cost. Please suggest me some SEO prospective way to resolve this issue, as I have monitoring this bot user agen name remains same and IP too for some day around a week thereafter its IP changes, some time its IP shows France location some Time Brazil and Some time USA so its really difficult to block them on the basis of country traffic basis which I have already tried. Now as I have increased my pages of the website its crawling rate also increased exponentially, in a minute 10 times its used to index the site, its really causing loss as my server hosting cost has increased due to this. So filter is not an option. On Thursday, April 23, 2015 at 10:02:37 PM UTC+5:30, Jeff Schnitzer wrote: What rate are these queries arriving? Are they all from the same IP address? Are they scanning your site or hitting one or two pages over and over? One request is not useful. Assuming this is just a poorly behaved bot and not a DOS attack, the simplest solution is to install a servlet filter at the top of your stack. Inspect the request and if you don't like something about it (IP address, user agent, etc) and return a blank page (or goatse, or whatever). Short of a real DDOS, this will convert your expensive 1100ms requests into almost-free 10ms requests and mitigate the issue. Jeff On Wed, Apr 22, 2015 at 11:42 PM, Ashutosh Mishra ashutosh.n...@gmail.com javascript: wrote: Hi Jeff, Thanks for your harsh word suggestion, Please have a look to attached log file snap shot you can have IP of the ahref bot, it use to regularly coming on my site. I think this real information will be enough, so please let me know concrete solution to overcome this problem. On Thursday, April 23, 2015 at 11:43:08 AM UTC+5:30, Jeff Schnitzer wrote: I'm calling bullshit. You have a website developed on GAE/Java but you don't understand what .htaccess is and why it doesn't apply? If you're having problems with your website, why don't you ask the people who developed it? I don't get it. The advice you have been offered here (all of which is reasonable) requires more technical sophistication than you exhibit. Possibly this is a bot doing normal things. Possibly this is a real DOS attack of some kind. Post some real information like IP addresses and the actual rate of requests, and maybe we can help you with an appropriate mitigation strategy. You have said a bunch of technically dumb things with an accusatory tone of voice (spam bots are attacking me!). This happens a lot, and usually it means _you_ just screwed something up. If you want help, post more information and be less arrogant about it. You don't know what you think you know. Jeff On Wed, Apr 22, 2015 at 7:08 AM, Barry Hunter barryb...@gmail.com wrote: Have you cross checked the IP(s) of the bot? The User-Agent is easily spoofed, it might be some other bot just pertending to be a ahrefbot. Regardless, as already mentioned can put handlers in your code to 'trap' bad actors. Check the useagent, and do something different. (can't totally block this way, but can minimise damage -make the requests very quick/short. And by not returning further links, stop them finding yet more pages to index). ... or use an external service to 'firewall' such requests - as already mentioned Cloudflare offer this. On 22 April 2015 at 15:02, Ashutosh Mishra ashutosh.n...@gmail.com wrote: Hi Vinny, thanks for your comment I have done the changes in myhotelcar.com/bobots.txt file as you have mentioned but issue is still not resolved as per my analysis the bots hiiting specifically ahref has increased day by day an now issue seems critical. Please hep me to get out of this situation. I will happy to have your advice on this. On Tuesday, April 21, 2015 at 10:07:28 PM UTC+5:30, Vinny P wrote: On Mon, Apr 20, 2015 at 11:32 PM, Ashutosh Mishra ashutosh.n...@gmail.com wrote: I have also searched so many thing and I found the Ahref bot doesn't obey robots principal. Many people has suggested that I can prohibit them via htaccess file, I don't want to use that way as in google app engine hosting I didn't find htaccess file. So please provide me any way to filter out these spam bots. The .htaccess file isn't supported in App Engine. If this is the real Ahref bot, it should support robots.txt. I looked in your robots.txt file: I see you disallowing Baidu, Yandex and a wildcard disallow, but not specifically ahrefbot. Try adding the following to your robots file: *user-agent: AhrefsBot* *disallow: /* According to the ahrefbot robot page, you can also email them directly to ask them to stop; see https://ahrefs.com/robot On Mon, Apr 20, 2015 at 11:36 PM, Ashutosh Mishra ashutosh.n...@gmail.com wrote: I think
Re: [google-appengine] So many spam bots are hitting my website hosted on Google App Engine
Thanks for your solution, but I can not go with filter, as filter will also increase cost How so? Do you mean the developer time to make the filter? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CAJCAUuLU7p-4oe53gfSvvDdT8mopi13-TO5eRvuHMyABB5vLWw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Google Cloud Platform wants to hear from you
@chris, thanks for asking, 1) Sorry I might have had some confusion here, I thought traffic splitting/traffic migration didn't work for managed VM's but it in fact it doesn't work for any non-default module. We haven't yet tried to put our default module on managed vms because of stability concerns (may be outdated) and the api's listed below) 2) Re: APIs, we can't yet do the following in Managed VM's: - channel api: we use this to send realtime notifications to web/mobile clients - logging api: we use this to export our logs to BQ 3) The other things preventing us from moving default module over to managed vms: - cloud console dashboard doesn't show a lot of the aggregate metrics for a managed vm module, most importantly instance count over time. - cloud monitoring can't monitor managed vm instance count - seemed that instances would go into unhealthy state and not recover or get killed (this could be outdated) At the current moment there seems to be no major concerns with managed VM's (except for the things you're working on like startup time and deploy time and dev experience) but there are dozens of tiny gotchas which individually don't seem like much but together are a big enough deal that we don't serve front end traffic over it. For background processing the value is so high that we've moved all of our backend modules to managed vms (unless they need any of the above API's). On Thursday, April 23, 2015 at 2:24:59 PM UTC-7, Chris Ramsdale wrote: yes, the goal is to get App Engine Managed VMs (v2) to a state where they are ideal for frontend serving. couple of questions for you: (1) is traffic splitting and deploy-to-a-default-module not working for you? (2) re: APIs, which ones would you like to see enabled? you mentioned logging (and the goal is to move App Engine Logging API over to Google Cloud Logging all-up), but what else are you looking for? -- Chris On Thu, Apr 23, 2015 at 11:47 AM, Aleem Mawani al...@streak.com javascript: wrote: Chris - with the improvements you're suggesting to Managed VMs (20 sec deploys, 1 sec instance startup time) - will it be recommended to use Managed VMs to serve frontend traffic? Right now they seem to be more targetted to backend processing because of the slow scaling. If this is true, and you are targeting it to serve front end traffic, does that mean we'll be able to deploy Managed VMs to our default modules, perform traffic splitting, access the logging api's and realtime api's, etc? These are the things that have been traditionally missing from managed vms. On Wednesday, April 22, 2015 at 10:33:46 AM UTC-7, Chris Ramsdale wrote: On Wed, Apr 22, 2015 at 2:21 AM, troberti tij...@firigames.com wrote: What a flurry of activity. :) Great to see. I have been using GCP( App Engine + BigQuery) in total for over 5 years, so not new, but I have seen plenty of new users make mistakes so let me chime in a bit: On App Engine (and GCP) there are a lot of ways to approach a problem, with the consequence that is very easy to choose the wrong solution. There is actually a rather steep learning curve to just know what is available. This is a problem, because the differences between various solutions can result in an order of magnitude difference in costs/latency/complexity etc. I stopped counting the amount of times I have seen models with every property indexed, resulting in huge datastore costs. Or where someone tries to put tons of data in the Datastore while BigQuery would be a much better fit for the problem. Every time this happens, the new user ends up disappointed. So guiding new users in the right direction when starting out on GCP seems very important. I agree with a lot in Karl's post, and especially the Roadmap. It doesn't need to be about features, but big ticket items like Python 3, Java 8, SSL etc should be communicated. It doesn't have to be an explicit list somewhere, just a PM chiming in regularly should be good enough. Now back to App Engine: Like I said in my post on the other thread https://cloud.google.com/appengine/forum/?place=msg%2Fgoogle-appengine%2FqTyc2E-0IXc%2FPxUzFMXhzjQJ, the trend towards managed VMs worries me a bit. For us, zero-configuration/no-maintenance is not just another feature of App Engine, it is one of the most important ones! We want to write our programs and then keep them running for years (5+) without having to do *anything*. Some of our apps are running like this for years now, and I want to ensure that this stays possible in the future. Managed VMs represent a new hosting environment that brings with it a set of feature benefits -- open source compatible runtimes, more CPU / memory configurations, access to native resources such as a file system and network stack. we'll be investing in this environment more and more over the coming months (we're ripping docker out of the
Re: [google-appengine] Help : Share Datastore of App Engine with another app engine project
You can access Google Cloud Datastore from anywhere -- see https://cloud.google.com/datastore/docs . Alex On Thu, Apr 23, 2015 at 1:50 PM, Jorge William Rodrigues jorge...@gmail.com wrote: Anyone know of any implementation or api own google app engine of integration the datastore of the projects Google App Engine? Not using Web Services! Is it possible to make this integration datastore? Is there any API -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/672a6e37-1c0c-45df-b792-bfbb0e7e9498%40googlegroups.com https://groups.google.com/d/msgid/google-appengine/672a6e37-1c0c-45df-b792-bfbb0e7e9498%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CAE46Be86ZxkgYB4V_hFvUa8jYB6Y4W%3DsCe1mXHhHBFhv02jzVA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Google Cloud Platform wants to hear from you
yes, the goal is to get App Engine Managed VMs (v2) to a state where they are ideal for frontend serving. couple of questions for you: (1) is traffic splitting and deploy-to-a-default-module not working for you? (2) re: APIs, which ones would you like to see enabled? you mentioned logging (and the goal is to move App Engine Logging API over to Google Cloud Logging all-up), but what else are you looking for? -- Chris On Thu, Apr 23, 2015 at 11:47 AM, Aleem Mawani al...@streak.com wrote: Chris - with the improvements you're suggesting to Managed VMs (20 sec deploys, 1 sec instance startup time) - will it be recommended to use Managed VMs to serve frontend traffic? Right now they seem to be more targetted to backend processing because of the slow scaling. If this is true, and you are targeting it to serve front end traffic, does that mean we'll be able to deploy Managed VMs to our default modules, perform traffic splitting, access the logging api's and realtime api's, etc? These are the things that have been traditionally missing from managed vms. On Wednesday, April 22, 2015 at 10:33:46 AM UTC-7, Chris Ramsdale wrote: On Wed, Apr 22, 2015 at 2:21 AM, troberti tij...@firigames.com wrote: What a flurry of activity. :) Great to see. I have been using GCP( App Engine + BigQuery) in total for over 5 years, so not new, but I have seen plenty of new users make mistakes so let me chime in a bit: On App Engine (and GCP) there are a lot of ways to approach a problem, with the consequence that is very easy to choose the wrong solution. There is actually a rather steep learning curve to just know what is available. This is a problem, because the differences between various solutions can result in an order of magnitude difference in costs/latency/complexity etc. I stopped counting the amount of times I have seen models with every property indexed, resulting in huge datastore costs. Or where someone tries to put tons of data in the Datastore while BigQuery would be a much better fit for the problem. Every time this happens, the new user ends up disappointed. So guiding new users in the right direction when starting out on GCP seems very important. I agree with a lot in Karl's post, and especially the Roadmap. It doesn't need to be about features, but big ticket items like Python 3, Java 8, SSL etc should be communicated. It doesn't have to be an explicit list somewhere, just a PM chiming in regularly should be good enough. Now back to App Engine: Like I said in my post on the other thread https://cloud.google.com/appengine/forum/?place=msg%2Fgoogle-appengine%2FqTyc2E-0IXc%2FPxUzFMXhzjQJ, the trend towards managed VMs worries me a bit. For us, zero-configuration/no-maintenance is not just another feature of App Engine, it is one of the most important ones! We want to write our programs and then keep them running for years (5+) without having to do *anything*. Some of our apps are running like this for years now, and I want to ensure that this stays possible in the future. Managed VMs represent a new hosting environment that brings with it a set of feature benefits -- open source compatible runtimes, more CPU / memory configurations, access to native resources such as a file system and network stack. we'll be investing in this environment more and more over the coming months (we're ripping docker out of the getting started flow, getting deployment times to 20 secs, getting instance activation time to 1 sec, adding scale to/from zero instances, etc.) that said, don't worry, we'll absolutely keep your existing v1 apps running just as they have for years. Glad to see the Docker fad go, but please don't replace it with something where I need to choose my technology stack in some way. Just provide a set of stable APIs instead so we can consider everything else an implementation detail for App Engine to worry about :) exactly. if anything, we're going to make those same APIs available from other compute environments (e.g. Compute Engine and Container Engine). On Wednesday, April 22, 2015 at 8:31:47 AM UTC+2, Chris Ramsdale wrote: On Tue, Apr 21, 2015 at 11:22 PM, Vinny P vinn...@gmail.com wrote: Hi Katie, I think Karl's post hit a home run and I'm happy to see the positive response to his post. Let me just tack on a few items: *Managed VMs:* The development toolchain for Managed VMs can be a bit finicky. To be quite honest I have no idea how I got Managed VMs working on my laptop. Streamlining this would be a huge benefit to me, and probably a lot of first-timers. If you can convince one of the online IDE services to simplify creating Managed VM GAE apps, that would be super. For smaller or toy apps within Managed VMs: I shouldn't need to care about the Docker container running the application; I should be able to create an application using just Eclipse + Google Plugin, then be able to deploy straight to a Managed VM
Re: [google-appengine] So many spam bots are hitting my website hosted on Google App Engine
I'm calling bullshit. You have a website developed on GAE/Java but you don't understand what .htaccess is and why it doesn't apply? If you're having problems with your website, why don't you ask the people who developed it? I don't get it. The advice you have been offered here (all of which is reasonable) requires more technical sophistication than you exhibit. Possibly this is a bot doing normal things. Possibly this is a real DOS attack of some kind. Post some real information like IP addresses and the actual rate of requests, and maybe we can help you with an appropriate mitigation strategy. You have said a bunch of technically dumb things with an accusatory tone of voice (spam bots are attacking me!). This happens a lot, and usually it means _you_ just screwed something up. If you want help, post more information and be less arrogant about it. You don't know what you think you know. Jeff On Wed, Apr 22, 2015 at 7:08 AM, Barry Hunter barrybhun...@gmail.com wrote: Have you cross checked the IP(s) of the bot? The User-Agent is easily spoofed, it might be some other bot just pertending to be a ahrefbot. Regardless, as already mentioned can put handlers in your code to 'trap' bad actors. Check the useagent, and do something different. (can't totally block this way, but can minimise damage -make the requests very quick/short. And by not returning further links, stop them finding yet more pages to index). ... or use an external service to 'firewall' such requests - as already mentioned Cloudflare offer this. On 22 April 2015 at 15:02, Ashutosh Mishra ashutosh.narayan3...@gmail.com wrote: Hi Vinny, thanks for your comment I have done the changes in myhotelcar.com/bobots.txt file as you have mentioned but issue is still not resolved as per my analysis the bots hiiting specifically ahref has increased day by day an now issue seems critical. Please hep me to get out of this situation. I will happy to have your advice on this. On Tuesday, April 21, 2015 at 10:07:28 PM UTC+5:30, Vinny P wrote: On Mon, Apr 20, 2015 at 11:32 PM, Ashutosh Mishra ashutosh.n...@gmail.com wrote: I have also searched so many thing and I found the Ahref bot doesn't obey robots principal. Many people has suggested that I can prohibit them via htaccess file, I don't want to use that way as in google app engine hosting I didn't find htaccess file. So please provide me any way to filter out these spam bots. The .htaccess file isn't supported in App Engine. If this is the real Ahref bot, it should support robots.txt. I looked in your robots.txt file: I see you disallowing Baidu, Yandex and a wildcard disallow, but not specifically ahrefbot. Try adding the following to your robots file: *user-agent: AhrefsBot* *disallow: /* According to the ahrefbot robot page, you can also email them directly to ask them to stop; see https://ahrefs.com/robot On Mon, Apr 20, 2015 at 11:36 PM, Ashutosh Mishra ashutosh.n...@gmail.com wrote: I think you have picked the issue correctly they are hitting particular set of pages regularly hotel pages which were dynamically generated, you are correct about rss and sitemap feed. So please tell me the way to overcome this issue as these spam bots specially ahref bot is consuming my server bandwidth a lot un-necessarily. I want a good solution so that I will not face any spam bot hurdle in future. This happens to a lot of websites with a large set of dynamically generated pages. Honestly the best solution would be to sign up for Cloudflare ( https://www.cloudflare.com/google ) and use their tools to help filter incoming traffic. You can also do what Barry suggested earlier, and start blocking the IPs that ahrefsbot is using. If you're willing to do some coding, you can write a filter into your application to check for the useragent and kick back a 429 HTTP status code (Too Many Requests) if traffic is too high: http://tools.ietf.org/html/rfc6585#page-3 - -Vinny P Technology Media Consultant Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/181d93e6-b9e8-40e6-8a24-d883a2e315f8%40googlegroups.com https://groups.google.com/d/msgid/google-appengine/181d93e6-b9e8-40e6-8a24-d883a2e315f8%40googlegroups.com?utm_medium=emailutm_source=footer . For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe
Re: [google-appengine] So many spam bots are hitting my website hosted on Google App Engine
Hi Jeff, Thanks for your harsh word suggestion, Please have a look to attached log file snap shot you can have IP of the ahref bot, it use to regularly coming on my site. I think this real information will be enough, so please let me know concrete solution to overcome this problem. On Thursday, April 23, 2015 at 11:43:08 AM UTC+5:30, Jeff Schnitzer wrote: I'm calling bullshit. You have a website developed on GAE/Java but you don't understand what .htaccess is and why it doesn't apply? If you're having problems with your website, why don't you ask the people who developed it? I don't get it. The advice you have been offered here (all of which is reasonable) requires more technical sophistication than you exhibit. Possibly this is a bot doing normal things. Possibly this is a real DOS attack of some kind. Post some real information like IP addresses and the actual rate of requests, and maybe we can help you with an appropriate mitigation strategy. You have said a bunch of technically dumb things with an accusatory tone of voice (spam bots are attacking me!). This happens a lot, and usually it means _you_ just screwed something up. If you want help, post more information and be less arrogant about it. You don't know what you think you know. Jeff On Wed, Apr 22, 2015 at 7:08 AM, Barry Hunter barryb...@gmail.com javascript: wrote: Have you cross checked the IP(s) of the bot? The User-Agent is easily spoofed, it might be some other bot just pertending to be a ahrefbot. Regardless, as already mentioned can put handlers in your code to 'trap' bad actors. Check the useagent, and do something different. (can't totally block this way, but can minimise damage -make the requests very quick/short. And by not returning further links, stop them finding yet more pages to index). ... or use an external service to 'firewall' such requests - as already mentioned Cloudflare offer this. On 22 April 2015 at 15:02, Ashutosh Mishra ashutosh.n...@gmail.com javascript: wrote: Hi Vinny, thanks for your comment I have done the changes in myhotelcar.com/bobots.txt file as you have mentioned but issue is still not resolved as per my analysis the bots hiiting specifically ahref has increased day by day an now issue seems critical. Please hep me to get out of this situation. I will happy to have your advice on this. On Tuesday, April 21, 2015 at 10:07:28 PM UTC+5:30, Vinny P wrote: On Mon, Apr 20, 2015 at 11:32 PM, Ashutosh Mishra ashutosh.n...@gmail.com wrote: I have also searched so many thing and I found the Ahref bot doesn't obey robots principal. Many people has suggested that I can prohibit them via htaccess file, I don't want to use that way as in google app engine hosting I didn't find htaccess file. So please provide me any way to filter out these spam bots. The .htaccess file isn't supported in App Engine. If this is the real Ahref bot, it should support robots.txt. I looked in your robots.txt file: I see you disallowing Baidu, Yandex and a wildcard disallow, but not specifically ahrefbot. Try adding the following to your robots file: *user-agent: AhrefsBot* *disallow: /* According to the ahrefbot robot page, you can also email them directly to ask them to stop; see https://ahrefs.com/robot On Mon, Apr 20, 2015 at 11:36 PM, Ashutosh Mishra ashutosh.n...@gmail.com wrote: I think you have picked the issue correctly they are hitting particular set of pages regularly hotel pages which were dynamically generated, you are correct about rss and sitemap feed. So please tell me the way to overcome this issue as these spam bots specially ahref bot is consuming my server bandwidth a lot un-necessarily. I want a good solution so that I will not face any spam bot hurdle in future. This happens to a lot of websites with a large set of dynamically generated pages. Honestly the best solution would be to sign up for Cloudflare ( https://www.cloudflare.com/google ) and use their tools to help filter incoming traffic. You can also do what Barry suggested earlier, and start blocking the IPs that ahrefsbot is using. If you're willing to do some coding, you can write a filter into your application to check for the useragent and kick back a 429 HTTP status code (Too Many Requests) if traffic is too high: http://tools.ietf.org/html/rfc6585#page-3 - -Vinny P Technology Media Consultant Chicago, IL App Engine Code Samples: http://www.learntogoogleit.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com javascript:. To post to this group, send email to google-a...@googlegroups.com javascript:. Visit this group at http://groups.google.com/group/google-appengine. To view
[google-appengine] Re: OverQuotaException without being over quota
My app has the same problem - please fix. Thanks! Project ID: spheric-brand-800 On Friday, December 7, 2012 at 1:51:05 AM UTC-7, Lucian Baciu wrote: My app has just started throwing this: com.google.apphosting.api.ApiProxy$OverQuotaException: The API call datastore_v3.Put() required more quota than is available for all data store update operations. Everything is Okay in the Admin Console Quotas section. Please FIX! This is affecting my users. App id: timetonote -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/23058e8b-e422-489c-a263-5f41d836988b%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
