[google-appengine] Re: what tool would you use? ///My company needs a system that automatically sends messages to users to request their documents,
Hi, These are your requests: 1. automatically sends messages to users 2. users can answer whether or not they can 3. use that data to convert it into information For no1, I'm not sure if you require this to be time-based or not. It's not clear what the trigger is. However, since it's a push mechanism (out of the server first without a request), then you probably would want to use Pub/Sub[1] or Cloud Functions[2]. Pub/sub pushes out messages to users that are subscribed to a particular topic, while Cloud Functions gets triggered by Pub/Sub where the code is running. Although, you could technically have Pub/Sub trigger anything else that you want. For no2, you will need to add some logic to your code after the pub/sub so that it logs the response of their choice. For no3, it all depends which database you want to use. Cloud Functions or any other service would be connected to a database. This is more difficult for me to suggest because you may have a preference on the database type in the way you store your semantics. I suggest choosing the database first and then what possible dashboard to use with that database. For example, here's[3] an article on how to use BigQuery (for analytics) with Cloud SQL (our SQL databases). [1] https://cloud.google.com/pubsub/docs/overview [2] https://cloud.google.com/functions [3] https://cloud.google.com/bigquery/docs/cloud-sql-federated-queries On Thursday, June 10, 2021 at 6:12:03 PM UTC-4 jedc...@gmail.com wrote: > My company needs a system that automatically sends messages to users to > request their documents. and that users can answer whether or not they can > claim them. then use that data to convert it into information. there are > hundreds of users and hundreds of documents to deliver. I'm thinking of > making an AI system that sends messages to people's cell phones, but I > don't know what google clud tool to use. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/3e9e2977-9f65-443d-bcff-96c3f29139dcn%40googlegroups.com.
[google-appengine] Re: Laravel 8 - build failed in google app engine flexible environment
Hi,
I think this may be a duplicate post? I see another post here[1] about the
same thing
Please mark as resolved if it's for the same question, otherwise please
elaborate if it's a different issue. This also helps the community to find
the right solutions. Thank you.
[1] https://groups.google.com/g/google-appengine/c/gpDH9AK2WyA
On Wednesday, June 9, 2021 at 3:11:59 PM UTC-4 bha...@gmail.com wrote:
>
> Hello,
> I have a Laravel PHP app with the following:
>
> - app.yaml
> ```
> runtime: php
> env: flex
>
> runtime_config:
> document_root: public
> whitelist_functions: proc_open
> automatic_scaling:
> min_num_instances: 1
> max_num_instances: 1
> resources:
> cpu: 1
> memory_gb: 0.5
> disk_size_gb: 10
> env_variables:
> # See substitution variables for Google Cloud Build Trigger
> ```
>
> - cloud_build.yaml
> ```
> steps:
> - name: node:14.0.0
> entrypoint: npm
> args: ['install']
> - name: node:14.0.0
> entrypoint: npm
> args: ['run', 'prod']
> - name: 'gcr.io/cloud-builders/gcloud'
> args: ['app', 'deploy', '--project', '$PROJECT_ID', '-q',
> '$_GAE_PROMOTE', '-v', '$_GAE_VERSION']
> timeout: '3600s'
> ```
>
> - composer.json
> ```
> "require": {
> "php": "7.3.*",
> ...
> }
> ```
>
> When the Cloud Build Trigger runs, I get the following error:
>
> ```
> Step #0: Digest:
> sha256:1f43a583e4c10b1758647be47b9fb5f9659227ef266978694acbb7981e4ee093
> Step #0: Status: Downloaded newer image for
> gcr.io/gcp-runtimes/php/gen-dockerfile@sha256:1f43a583e4c10b1758647be47b9fb5f9659227ef266978694acbb7981e4ee093
> Step #0:
> gcr.io/gcp-runtimes/php/gen-dockerfile@sha256:1f43a583e4c10b1758647be47b9fb5f9659227ef266978694acbb7981e4ee093
> Step #0: + php /builder/create_dockerfile.php create --php73-image
> gcr.io/google-appengine/php73@sha256:bee53597a0df4167547fd9c2a501484e8c5d03831298f50c1db0e9cdcd915a88
>
> --php72-image
> gcr.io/google-appengine/php72@sha256:c3f3636b89aab3a83ab288a0108f767a4d56002c9a214b17b5fe93b9ba5f8bbf
>
> --php71-image
> gcr.io/google-appengine/php71@sha256:9d39f66c4e0b7c9a9590ed27373b8c18e79a6cde3dbf48c284ffc5f3fd99d12a
> Step #0: [09-Jun-2021 19:06:00 UTC] PHP Warning: array_key_exists()
> expects parameter 2 to be array, null given in
> /builder/src/Builder/GenFilesCommand.php on line 232
> Step #0:
> Step #0: Warning: array_key_exists() expects parameter 2 to be array, null
> given in /builder/src/Builder/GenFilesCommand.php on line 232
> Step #0: [09-Jun-2021 19:06:00 UTC] PHP Warning: array_key_exists()
> expects parameter 2 to be array, null given in
> /builder/src/Builder/GenFilesCommand.php on line 232
> Step #0:
> Step #0: Warning: array_key_exists() expects parameter 2 to be array, null
> given in /builder/src/Builder/GenFilesCommand.php on line 232
> Step #0: [09-Jun-2021 19:06:00 UTC] PHP Warning: array_key_exists()
> expects parameter 2 to be array, null given in
> /builder/src/Builder/GenFilesCommand.php on line 174
> Step #0:
> Step #0: Warning: array_key_exists() expects parameter 2 to be array, null
> given in /builder/src/Builder/GenFilesCommand.php on line 174
> Step #0: [09-Jun-2021 19:06:00 UTC] PHP Fatal error: Uncaught Error:
> Unsupported operand types in /builder/src/Builder/GenFilesCommand.php:279
> ```
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/google-appengine/1fb8de08-7448-4e10-b354-8b0149a6ce9en%40googlegroups.com.
[google-appengine] Re: Tasks duplicated when deploying new version of Python 3 service
Hi, When a new version is spun-up, it should not carry state. However, as mentioned previously, because the Cloud Tasks API is decoupled and you are keeping an older version alive, the new version will grab the current state of the API's queue that is decoupled and a duplicate situation will happen because the API hasn't received acknowledgment that this task in the queue is done. I think this happens particularly because it's just versioning and the way our load balancing works. This is why I don't think that keeping an older version is the best choice and it is expected behavior. However, you could capitalize on forcing them to not complete on the previous version by not using "-no-stop-previous-version" and somehow restart the task when that happens. There is a section here[1] for retrying upon failure. [1] https://cloud.google.com/tasks/docs/manage-cloud-task-scaling#retries On Wednesday, June 2, 2021 at 11:17:32 AM UTC-4 ludovic@lumapps.com wrote: > Hi, thanks for your reply. > > This seems to happen every time we deploy a new version while tasks are > executing on the previously serving version. We use "gcloud app deploy > --no-stop-previous-version" to let these tasks complete on the previous > version. Doing this seems to systematically cause Cloud Tasks to start a > new "execution" of some (or all) of these tasks on the new version. We > clearly see this in the logs, we see two requests with the same task > name/id running concurrently on the older and the new version. > > Perhaps this is normal, perhaps it is not. Is it? > > Many thanks > > > On Wednesday, June 2, 2021 at 4:39:49 PM UTC+2 Alexis (Cloud Platform > Support) wrote: > >> Hi, >> >> You asked if it's expected behavior and the article you linked says that >> it could happen and it's expected behavior, so "Developers should take >> steps to ensure that duplicate execution is not a catastrophic event" >> >> I think your question is more about frequency? Please tell us how often >> duplicates happen. >> >> On a side-note. Tasks are architecturally decoupled from the versions >> semantics since it's a separate API. It's technically part of the Cloud >> Tasks API as mentioned here[1]. So this means "gcloud app deploy >> --no-stop-previous-version" would probably entice more duplication as the >> state of the code is being scaled-up, the previous configs remain active >> because they are decoupled. In this case you'd have to ask yourself, what >> is the state or the condition of the code doing? And how often is it >> happening. This would happen in any microservice environment that scales >> and I think it's no2 "Asynchronicity" here[2]. Depending on your needs, you >> may require to code a completely separate service that is more elaborate >> than our tasks API. It's one of the pitfalls of having things decoupled. >> >> [1] https://cloud.google.com/tasks/docs/queue-yaml#introduction >> [2] >> https://blog.bernd-ruecker.com/3-common-pitfalls-in-microservice-integration-and-how-to-avoid-them-3f27a442cd07 >> >> >> On Tuesday, June 1, 2021 at 2:09:25 PM UTC-4 ludovic@lumapps.com >> wrote: >> >>> We noticed that when a new version of a "basic" (B4_1G) Python 3.9 >>> service is deployed, the running cloud tasks executing on the hitherto >>> latest version are sometimes (or perhaps always?) started again on the >>> newly deployed version. We noticed that the same task then runs on both the >>> older and the new version until their respective completions. >>> >>> This may be expected, yet the documentation seems to imply this should >>> now occur in general: >>> https://cloud.google.com/tasks/docs/common-pitfalls#duplicate_execution >>> >>> The new versions are deployed with "gcloud app deploy >>> --no-stop-previous-version ..." >>> >>> So my question is: is this normal behavior? >>> >>> >>> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/1a5b7d29-9b8a-4b53-8af9-32488c5cb6den%40googlegroups.com.
[google-appengine] Re: Tasks duplicated when deploying new version of Python 3 service
Hi, You asked if it's expected behavior and the article you linked says that it could happen and it's expected behavior, so "Developers should take steps to ensure that duplicate execution is not a catastrophic event" I think your question is more about frequency? Please tell us how often duplicates happen. On a side-note. Tasks are architecturally decoupled from the versions semantics since it's a separate API. It's technically part of the Cloud Tasks API as mentioned here[1]. So this means "gcloud app deploy --no-stop-previous-version" would probably entice more duplication as the state of the code is being scaled-up, the previous configs remain active because they are decoupled. In this case you'd have to ask yourself, what is the state or the condition of the code doing? And how often is it happening. This would happen in any microservice environment that scales and I think it's no2 "Asynchronicity" here[2]. Depending on your needs, you may require to code a completely separate service that is more elaborate than our tasks API. It's one of the pitfalls of having things decoupled. [1] https://cloud.google.com/tasks/docs/queue-yaml#introduction [2] https://blog.bernd-ruecker.com/3-common-pitfalls-in-microservice-integration-and-how-to-avoid-them-3f27a442cd07 On Tuesday, June 1, 2021 at 2:09:25 PM UTC-4 ludovic@lumapps.com wrote: > We noticed that when a new version of a "basic" (B4_1G) Python 3.9 service > is deployed, the running cloud tasks executing on the hitherto latest > version are sometimes (or perhaps always?) started again on the newly > deployed version. We noticed that the same task then runs on both the older > and the new version until their respective completions. > > This may be expected, yet the documentation seems to imply this should now > occur in general: > https://cloud.google.com/tasks/docs/common-pitfalls#duplicate_execution > > The new versions are deployed with "gcloud app deploy > --no-stop-previous-version ..." > > So my question is: is this normal behavior? > > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/cd8a83c3-6379-4758-8368-6f78edc4d7efn%40googlegroups.com.
[google-appengine] Re: How to host React and PHP application on Google App Engine?
Hello, Since React is not supported by Google, you may not find official Google documentation about it. However, based on other forum comments, I suspect you need to align the YAML file and build folder to be in the same path. For example, step 5 of this[1] tutorial mentions to create the YAML inside the build folder, and add some wildcard handlers (more about handlers here[2]). After this is done, deploying should work and align all resources. In addition, these[3][4] articles mention the same exact steps, with the latter as an automated script instead. I hope you find this useful. Sincerely, Alexis Google Cloud Platform Support [1] https://javascript.plainenglish.io/quickly-deploy-your-react-app-on-googles-app-engine-6bb97480cc9c [2] https://cloud.google.com/appengine/docs/standard/php7/config/appref [3] https://stackoverflow.com/questions/5369/app-engine-app-yaml-handlers-for-built-react-app [4] https://blog.doit-intl.com/deploying-a-react-app-to-googles-app-engine-6efa8f4732c7 On Saturday, May 1, 2021 at 9:11:38 AM UTC-4 smith.na...@portalwiz.com wrote: > I want to host an application google cloud that is built in React and the > backend is in PHP. I want to know how to do it? Is there any way to do > that? Can we write runtime: node and PHP both on the app.YAML file? > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/73e3e1c8-f344-4241-8eb2-1b085f69611bn%40googlegroups.com.
Re: [google-appengine] Re: App Engine - Enable Cloud Debugging for PHP
Hi Bilal, That is correct. The instructions for Cloud Debugger allows the link between your code and the graphical user interface of Google Cloud Platform, so that you can see information with the StackDriver/Debugging tool in our Dashboards. An example on how to view that is shown here https://cloud.google.com/debugger/docs/quickstart#view_deployed_source_code Sincerely, Alexis Google Cloud Platform Support On Thursday, April 15, 2021 at 3:27:54 AM UTC-4 bha...@gmail.com wrote: > Hello Alex, > Thanks again for the help. > > I understand from you that if I follow the instructions on how to enable > Cloud Debugger with the StackDriver on my PHP Larave app, including the > php.ini file, Is should be viewing my PHP Laravel app logs inside the Stack > Driver somewhere on GCP Dashboard. Correct? > > Regards > Bilal > > On Tue, Apr 13, 2021 at 9:46 PM 'Alexis (Google Cloud Platform Support)' > via Google App Engine wrote: > >> Hi Bilal, >> >> I suppose you are asking why use App Engine? >> >> The article you provided says that you can use App Engine, Compute >> Engine, Kubernetes Engine, etc... Your question was about App Engine, so I >> linked the App Engine documentation. >> >> In general, the Operational Suite (formerly "Stack Driver") requires some >> sort of VM to operate the logs. These VMs are hidden from you when you use >> a Google Service and it's included inside of it, along with the rest of the >> code. However, in this case I suspect you use your own service/framework >> for your code and the logs are not embedded as part of the Google >> infrastructure. So, you need a VM of some sort to merge the logs with GCP's >> infrastructure tools. Any of the selections above will do. >> >> Hope this answers your question. >> >> Sincerely, >> >> Alexis >> Google cloud Platform Support >> >> On Tuesday, April 13, 2021 at 1:11:32 PM UTC-4 bha...@gmail.com wrote: >> >>> Hi Alex, >>> Thanks a lot for assisting here. I will surely check it and try it out. >>> >>> I have one question though. Basically, I am enabling stack driver so >>> that my app, Laravel, would send its log to stack driver which is read by >>> GCP Cloud Debugger? Do I get the idea here? Maybe I miss the point, I am >>> just asking to get more info. >>> >>> Thanks >>> >>> On Tue, Apr 13, 2021 at 7:52 PM 'Alexis (Google Cloud Platform Support)' >>> via Google App Engine wrote: >>> >>>> Hi, >>>> >>>> It will be a pleasure to help. >>>> >>>> App Engine works by uploading/deploying your local configuration and >>>> code. This means you would start locally first. Therefore, if your php.ini >>>> is in the root folder of your local code, it should work. In the document >>>> you provided, some details are omitted for the entire App Engine process >>>> of >>>> configuration since it's mostly about Cloud Debugger. However, the rest of >>>> the details on how the configuration works for App Engine can be found >>>> here[1]. >>>> >>>> In your case, the php.ini is fairly standard to the runtime and not >>>> proprietary to App Engine's YAML, but more part of your codebase and the >>>> root folder structure. The explicit answer is located here[2] (where it >>>> says "If you need any of the above functions, add a php.ini file in the >>>> root of your application"). Also, please note the section below that, that >>>> also talks about describing the root folder in the YAML. You may find all >>>> the details on that page. >>>> >>>> Let us know if that solves your issue. Thank you. >>>> >>>> Sincerely, >>>> >>>> Alexis >>>> Google Cloud Platform Support >>>> >>>> [1] >>>> https://cloud.google.com/appengine/docs/flexible/php/configuration-files >>>> [2] >>>> https://cloud.google.com/appengine/docs/flexible/php/runtime#disabled_functions >>>> >>>> On Monday, April 12, 2021 at 5:37:03 PM UTC-4 bha...@gmail.com wrote: >>>> >>>>> >>>>> Hi, >>>>> I followed the instructions on the page >>>>> https://cloud.google.com/debugger/docs/setup/php?authuser=1 >>>>> >>>>> I am using App Engine Flex. At once the docs asks to do the following: >>>>> >>>>> >>>>> App Eng
Re: [google-appengine] Re: App Engine - Enable Cloud Debugging for PHP
Hi Bilal, I suppose you are asking why use App Engine? The article you provided says that you can use App Engine, Compute Engine, Kubernetes Engine, etc... Your question was about App Engine, so I linked the App Engine documentation. In general, the Operational Suite (formerly "Stack Driver") requires some sort of VM to operate the logs. These VMs are hidden from you when you use a Google Service and it's included inside of it, along with the rest of the code. However, in this case I suspect you use your own service/framework for your code and the logs are not embedded as part of the Google infrastructure. So, you need a VM of some sort to merge the logs with GCP's infrastructure tools. Any of the selections above will do. Hope this answers your question. Sincerely, Alexis Google cloud Platform Support On Tuesday, April 13, 2021 at 1:11:32 PM UTC-4 bha...@gmail.com wrote: > Hi Alex, > Thanks a lot for assisting here. I will surely check it and try it out. > > I have one question though. Basically, I am enabling stack driver so that > my app, Laravel, would send its log to stack driver which is read by GCP > Cloud Debugger? Do I get the idea here? Maybe I miss the point, I am just > asking to get more info. > > Thanks > > On Tue, Apr 13, 2021 at 7:52 PM 'Alexis (Google Cloud Platform Support)' > via Google App Engine wrote: > >> Hi, >> >> It will be a pleasure to help. >> >> App Engine works by uploading/deploying your local configuration and >> code. This means you would start locally first. Therefore, if your php.ini >> is in the root folder of your local code, it should work. In the document >> you provided, some details are omitted for the entire App Engine process of >> configuration since it's mostly about Cloud Debugger. However, the rest of >> the details on how the configuration works for App Engine can be found >> here[1]. >> >> In your case, the php.ini is fairly standard to the runtime and not >> proprietary to App Engine's YAML, but more part of your codebase and the >> root folder structure. The explicit answer is located here[2] (where it >> says "If you need any of the above functions, add a php.ini file in the >> root of your application"). Also, please note the section below that, that >> also talks about describing the root folder in the YAML. You may find all >> the details on that page. >> >> Let us know if that solves your issue. Thank you. >> >> Sincerely, >> >> Alexis >> Google Cloud Platform Support >> >> [1] >> https://cloud.google.com/appengine/docs/flexible/php/configuration-files >> [2] >> https://cloud.google.com/appengine/docs/flexible/php/runtime#disabled_functions >> >> On Monday, April 12, 2021 at 5:37:03 PM UTC-4 bha...@gmail.com wrote: >> >>> >>> Hi, >>> I followed the instructions on the page >>> https://cloud.google.com/debugger/docs/setup/php?authuser=1 >>> >>> I am using App Engine Flex. At once the docs asks to do the following: >>> >>> >>> App Engine flexible environment pecl install stackdriver_debugger-alpha >>> >>> If your php.ini file does not include extension=stackdriver_debugger.so >>> after running this step, add it manually. >>> >>> >>> On App Engine Flex, how can I access the php.ini to include that >>> extension? >>> >>> I did the rest of steps, however, when running cloud_build on App >>> Engine, I get error something like "install google/cloud or >>> google/error-reporting packages". I already have Google/cloud installed. >>> >>> Most probably, the extension is missing from php.ini and that's why it's >>> not working. >>> >>> >>> I appreciate your help. >>> >>> Thanks >>> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "Google App Engine" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/google-appengine/3zU3bF8O8sw/unsubscribe >> . >> To unsubscribe from this group and all its topics, send an email to >> google-appengi...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/google-appengine/dc1a4854-de71-4729-9e6c-f8b1193c25a1n%40googlegroups.com >> >> <https://groups.google.com/d/msgid/google-appengine/dc1a4854-de71-4729-9e6c-f8b1193c25a1n%40googlegroups.com?utm_medium=email_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/de6b2e4b-e192-48a7-92b6-c0ff58da9119n%40googlegroups.com.
[google-appengine] Re: App Engine - Enable Cloud Debugging for PHP
Hi, It will be a pleasure to help. App Engine works by uploading/deploying your local configuration and code. This means you would start locally first. Therefore, if your php.ini is in the root folder of your local code, it should work. In the document you provided, some details are omitted for the entire App Engine process of configuration since it's mostly about Cloud Debugger. However, the rest of the details on how the configuration works for App Engine can be found here[1]. In your case, the php.ini is fairly standard to the runtime and not proprietary to App Engine's YAML, but more part of your codebase and the root folder structure. The explicit answer is located here[2] (where it says "If you need any of the above functions, add a php.ini file in the root of your application"). Also, please note the section below that, that also talks about describing the root folder in the YAML. You may find all the details on that page. Let us know if that solves your issue. Thank you. Sincerely, Alexis Google Cloud Platform Support [1] https://cloud.google.com/appengine/docs/flexible/php/configuration-files [2] https://cloud.google.com/appengine/docs/flexible/php/runtime#disabled_functions On Monday, April 12, 2021 at 5:37:03 PM UTC-4 bha...@gmail.com wrote: > > Hi, > I followed the instructions on the page > https://cloud.google.com/debugger/docs/setup/php?authuser=1 > > I am using App Engine Flex. At once the docs asks to do the following: > > > App Engine flexible environment pecl install stackdriver_debugger-alpha > > If your php.ini file does not include extension=stackdriver_debugger.so > after running this step, add it manually. > > > On App Engine Flex, how can I access the php.ini to include that extension? > > I did the rest of steps, however, when running cloud_build on App Engine, > I get error something like "install google/cloud or google/error-reporting > packages". I already have Google/cloud installed. > > Most probably, the extension is missing from php.ini and that's why it's > not working. > > > I appreciate your help. > > Thanks > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/dc1a4854-de71-4729-9e6c-f8b1193c25a1n%40googlegroups.com.
[google-appengine] Re: How to install a fresh wordpress
Hi, With your picture, I can't tell if you are using an ephemeral IP. If you are, the first thing to do would be to promote your ephemeral external IP to a static external IP. There's more information about this here[1] Also, for resetting the data on your VM... Your Wordpress is just software you installed on our virtual machine (GCE instance). So, if you wipe the VM clean and use the same method to install as before, it should be fine. Unless you utilized a method that automatically installed WP along with the VM bundled with it? You will have to specify if you did it differently. In short, this[2] procedure allows you to delete the instance. You can experiment with stopping/restarting/delete instances and be free to do as you wish without loosing your IP *as long as it's depicted as a static IP in our systems*. Updating to a static IP (*and not choosing to deleted it in your configurations*) is your safest bet to do this properly. Once it's static, you can delete and recreate, this can sometimes be as fast as a manual format. If a manual format is what you want, you can log in using SSH into your VM and do a normal wipe like on any other machine. A search for you OS command line on how to wipe could help. However, most people prefer to completely delete the instance (once they have a static IP) since it's as fast and especially if you have a bundle of configuring the VM + Wordpress. [1] https://cloud.google.com/compute/docs/ip-addresses/reserve-static-external-ip-address#reserve_new_static [2] https://cloud.google.com/compute/docs/instances/deleting-instance On Monday, October 19, 2020 at 11:35:46 AM UTC-4 31th...@gmail.com wrote: > Hello there, > I would like to know how can I install a fresh wordpress on my VM instance > without changing the external IP address. > > https://prnt.sc/v1wtpx > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/f37933e8-415e-42d0-a8dc-c292e4157894n%40googlegroups.com.
[google-appengine] Re: App engine connection to cloud storage
Hi, I'll try to assist the best I can. The main issue here is that what you are doing is an http request and not local to the machine. For example "gs://" is essentially the gsutil tool wrapping-up an HTTP request and emulating locally, but it's not part of the local file system. Whereas the library you are using thinks it's manipulating the file system. Hence the problem you are having. The workaround would be this[1], which consists of "mounting" GCS to the filesystem using Cloud Storage FUSE. But the problem is that it's an anti-pattern on App Engine. App Engine is ephemeral and could have many instances with these mounts. You are bypassing a library with concurrency for when many instances would talk to the same database. This means it may work if you only max one instance, but you could have problems after more than one instance is talking to the same files. It just doesn't make sense architecturally, but it may work. Use at your own risk. Let me know if there's something I'm not seeing or why you can't use the library provided. [1] https://cloud.google.com/compute/docs/disks/gcs-buckets#mount_bucket On Thursday, October 8, 2020 at 10:57:10 AM UTC-4 punit...@gmail.com wrote: > In my Springboot based JAVA app, how can I specify the address of cloud > storage to access it directly without using "Storage" GCP Java classes. > With windows local app, I am using application.properties file to specify > the same, and access it directly. > > If I try to replace the same "E:/folder1/folder2" with > "gs://bucket1/folder1/folder2" > it gives me error: > Constructor threw exception; nested exception is > java.nio.file.FileSystemException: /workspace/gs:: Read-only file system. > Meaning it is trying to find the path in the same workspace as where the > jar/war file is placed. > > Any help? > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/26498f75-4686-4a9d-91f6-9af94ac3aa5en%40googlegroups.com.
[google-appengine] Re: My app verification is still pending
Hello, I will try to assist the best I can. There is an internal escalation process for this request. However, it requires a project ID, to which I strongly suggest you do NOT post this in this public forum. As George mentioned above, you may create a private request using the Public Issue Tracker link. The private request will not show your post publicly, but only with Google. Hope this helps. Sincerely, Alexis Google Cloud Platform Support On Wednesday, September 16, 2020 at 10:02:14 AM UTC-4 edav...@gmail.com wrote: > I have this same issue. Can you provide the correct link to create a > support ticket for this particular issue? > > On Wednesday, October 16, 2019 at 11:19:50 AM UTC-4 George (Cloud Platform > Support) wrote: > >> Has your app verification progressed meanwhile? You should keep in >> contact with the consent screen verification team, who is expected to >> verify and eventually approve your consent screen. You might be well >> entitled to support, so opening a support ticket in place of Groups >> postings should allow for direct and personalized help in this matter. You >> may also consider opening an issue in the Public Issue Tracker >> <https://issuetracker.google.com/>, if you need guidance in checking >> with the verification team. >> > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/e3c43641-3e05-457e-81ae-1ea646ff9a4en%40googlegroups.com.
[google-appengine] Re: Spending LImits Going Away :(
Hello everyone, To summarize this conversation, it is possible to set proactive limits by: - Maximum number of instance[1] in an app - Disable your app programmatically[2] for any resource consumption, etc.. - Cap API limits to prevent too many requests[3] If I understood properly, Joshua's situation is about caching. So in that case, we are talking about resource limits related to CPU cycles, or maybe amount of requests... I suspect that whatever resource spiked by this situation, can probably make use of the second option above or even the third if it's API consumption. But it all depends on how fast this happened and I would agree that having a hard cap feature would prevent the delay. In terms of instantaneous hard caps (with no delays), a feature request[4] can be done. However, I think it would be advantageous to clarify how such a cap can be helpful without impacting scalability. Please see below. When submitting a feature request: - Try to clarify how the limit should differentiate a CPU cycle that is legit and versus one that isn't legit. Would you want that on all your instances? Would you mind if it stopped all your services as a false alarm? (The current solution for that is max instances in my first point above because scaling is a horizontal concept and it doesn't completely stop things since it's a delta quantity. If some of the other issues mentioned were due to excessive instances or API requests , then please refer to the above). I hope this message consolidates the research and saves time for any new person reading this post. Thank you in advance. [1] https://cloud.google.com/appengine/docs/managing-costs#specify_the_maximum_number_of_instances [2] https://cloud.google.com/appengine/docs/managing-costs#disable_your_app_programmatically [3] https://cloud.google.com/apis/docs/capping-api-usage [4] https://cloud.google.com/support/docs/issue-trackers?hl=id#trackers-list On Wednesday, September 2, 2020 at 11:18:59 AM UTC-4 vit@gmail.com wrote: > Canceling a simple and straightforward option "daily spending limit" is a > very cruel action for beginners just starting to learn the platform. A > separate warning should be given in the starter guide: > > In order not to go bankrupt when using GCP, you first need to carefully > study the documentation section https://cloud.google.com/cost-management. > > My story. > > Ndb model with a lot of indexed fields (since the field property indexed = > True by default). The cost of writing one entry to such a model was very > high. > > The main program sent a message to the queue when a fairly rare event > occurred. The handler of such a message from the queue updated the > corresponding record in the model, and then an exception occurred due to an > error in the code, and the handler exited with an error code. > > The platform initiated several processing retries, which also failed. In > each attempt, there was a write to the model. > > The main program, not finding the results of processing, added a new > message to the queue, which caused a new series of attempts described > above. And so on. > > The daily budget was depleted within a few minutes and the program was > stopped by a fuse "daily spending limit". > > In the current reality, I would have received a bill for several thousand > dollars if I had reacted to this situation within 24 hours. > > Yes, then I changed the model, leaving indexed=True only for a couple of > fields where it was really necessary and changed the logic of both the main > program and the message handler from the queue. > > But all this happened later. > > вторник, 25 августа 2020 г. в 20:03:33 UTC+4, Joshua Smith: > >> Once again last night, my wallet was saved when a runaway bot chewed up >> my site’s whole daily spending limit. I got an email from a user, set up a >> firewall rule, and goosed my budget to get things going again. >> >> I’m *very* concerned about Google’s decision to remove this feature. >> Offering a cloud service that bills by usage without having a way to limit >> the spend shifts an unreasonable amount of risk onto the subscriber. >> >> I’ve set up budget alerts, as suggested, but I’m concerned that: >> >> - What if my bill shoots up really fast? How quickly is this alert going >> to go out? >> >> - What if I am away from the computer (remember when we used to be able >> to leave our houses? good times… good times…)? >> >> I run this particular site as a not-for-profit social good. (It’s a site >> that small town governments use to post their meetings.) I make *no* money >> on it. >> >> I’d be perfectly happy to handle this with self-set quotas on something >> other than dollars. For example, in my case the budget-buster is always >> “Cloud Datastore Read Operations.” If I could set a cap on that one thing, >> it’d give me the protection I need. >> >> -Joshua >> >> -- You received this message because you are subscribed to the Google Groups
[google-appengine] Re: Deploying to App Engine Standard Env + Laravel Project + Read-only file system error
Hi Hadil, The error says: "Failed to open stream: Read-only file system". I think this means that it's finding a file there already (under "/workspace/storage/framework/views/") where it shouldn't be. This could be a caching issue with Laravel where the files are upload by certain machines only. However, if it isn't a caching issue, then something is different about those local machines and what it uploads since one dev doesn't have this problem 100% of the time. It could also be a config issue in Laravel. Scenario 1: If all the devs are pulling from the same repo: - What if you try one of these[1] commands to clear view cache on non-working machines? - What if you try clearing the general[2] cache on non-working machines? - Check the gitignore file if it's the same across stations that work and don't work Scenario 2: If the devs are not pulling from same repo: - Where is the part of the code that should have VIEW_COMPILED_PATH? Is that code different across repos? - Check the gitignore file if it's the same across stations that work and don't work There's a lot of possibilities, but I suspect it might be caching being uploaded into the wrong path somehow. Probably due to local testing. [1] https://laravel.com/docs/7.x/views#optimizing-views [2] https://laravel.com/docs/7.x/configuration#configuration-caching On Wednesday, August 26, 2020 at 4:16:05 AM UTC-4 hadil.cha...@mmpww.com wrote: > Hello Mary, > > Thank you for your reply. > > The app.yaml file already includes "/*tmp*" for both "*VIEW_COMPILED_PATH*" > and "*APP_STORAGE*". For more permanent files, we're already using Cloud > Buckets. What Laravel is trying to do is store compiled views (which are > *temporary* files) onto "/*workspace*/storage/framework/views/". However, > from app.yaml we already set "VIEW_COMPILED_PATH" as *tmp*, why is the > code still trying to save onto the "*workspace*" directory? I need to > emphasize again that this project has been working fine for more than a > year on app engine and it started breaking only 2 weeks ago. I appreciate > the links you shared but if you read the problem carefully you'll see that > the links do not address our issue. > > With the above in mind, can you please direct us towards a solution? > > On Wednesday, August 26, 2020 at 12:13:31 AM UTC+3 Mary (Google Cloud > Support) wrote: > >> Hello Hadil, >> >> Yes that is correct. Since App Engine Standard the disk is read-only[1] >> with the exception of the /tmp directory, the app.yaml file will be need >> modified for the APP_STORAGE to reflect /tmp as you have mentioned. If >> these are static files that Laravel needs to retrieve you can use a Cloud >> Storage bucket to store them. There is an open source Laravel Google Cloud >> Storage Package[2] which can help you with this. More information can be >> found on how to configure Laravel with the above package[3]. Please be >> aware since this package is open source, it is not supported or endorsed by >> GCP. >> >> [1] >> https://cloud.google.com/appengine/docs/the-appengine-environments#comparing_high-level_features >> [2] https://github.com/Superbalist/laravel-google-cloud-storage >> [3] >> https://stackoverflow.com/questions/57177524/laravel-how-to-access-stored-image-from-google-cloud-storage-using-php-laravel >> >> On Tuesday, August 25, 2020 at 8:02:02 AM UTC-4 hadil.cha...@mmpww.com >> wrote: >> >>> Also, something else worth noting is that we're not sure why Laravel is >>> trying to the write to the folder "/*workspace*/storage/framework/views/" >>> instead of "/*tmp*/storage/framework/views/". Also, in case it's >>> helpful to someone, below is a copy of our app.yaml (it's been configured >>> the same way for over a year): >>> >>> APP_KEY: XXX >>> APP_URL: 'X' >>> APP_NAME: XXX >>> APP_STORAGE: /tmp >>> APP_ENV: production >>> APP_DEBUG: true >>> VIEW_COMPILED_PATH: /tmp >>> >>> On Monday, August 24, 2020 at 7:02:51 PM UTC+3 Hadil Charafeddine wrote: >>> Hello, Our team has been using App Engine Standard for this project (PHP and Laravel 5.8.38) for over a year now without any issues with deployment. However, a couple of weeks ago we started getting a 500 HTTP error every time we deploy (with and without --no-promote option). When we check our logs, we find that the error is caused by: *2020/08/18 06:15:15 [error] 25#25: *2 FastCGI sent in stderr: "PHP message: PHP Fatal error: Uncaught ErrorException: file_put_contents(/workspace/storage/framework/views/c1b305dfa33e081e1fc836f46439e09f829e8770.php): failed to open stream: Read-only file system* It's very weird that we're receiving this message now even though the project has been working fine for more than a year and we've made countless deployments to date without this issue. Furthermore, we did not update the configuration of the project nor did we update the Laravel version.
[google-appengine] Re: Spending LImits Going Away :(
Hello Joshua, I'll try to help the best I can. I've added a question below, along with some answers too. Questions: - Which feature did Google remove? What was this feature called and where was it shown in the GUI? I'm asking so that we can try to maybe do a feature request or see why it was removed. To answer your question, the alerts should be prompt unless there is an outage or some other exceptional circumstance. However, keep in mind that we do not have control over public communication channel delays such as SMS, email, etc... Double-checking in the GUI could tell you if it was already sent out or not. Delays can also happen if alerts have multiple conditions and one of them hasn't been met yet. See full article here[1] for more details about latency possibilities. If you're away from the computer, you have notification options here[2], called "channels". [1] https://cloud.google.com/monitoring/alerts/concepts-indepth#notification-latency [2] https://cloud.google.com/monitoring/support/notification-options#creating_channels On Tuesday, August 25, 2020 at 12:03:33 PM UTC-4 Joshua Smith wrote: > Once again last night, my wallet was saved when a runaway bot chewed up my > site’s whole daily spending limit. I got an email from a user, set up a > firewall rule, and goosed my budget to get things going again. > > I’m *very* concerned about Google’s decision to remove this feature. > Offering a cloud service that bills by usage without having a way to limit > the spend shifts an unreasonable amount of risk onto the subscriber. > > I’ve set up budget alerts, as suggested, but I’m concerned that: > > - What if my bill shoots up really fast? How quickly is this alert going > to go out? > > - What if I am away from the computer (remember when we used to be able to > leave our houses? good times… good times…)? > > I run this particular site as a not-for-profit social good. (It’s a site > that small town governments use to post their meetings.) I make *no* money > on it. > > I’d be perfectly happy to handle this with self-set quotas on something > other than dollars. For example, in my case the budget-buster is always > “Cloud Datastore Read Operations.” If I could set a cap on that one thing, > it’d give me the protection I need. > > -Joshua > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/f77ae31b-61bd-4251-bb5d-4809076b70fan%40googlegroups.com.
Re: [google-appengine] Re: Transactional tasks delayed
Hello Okku, I looked into your case and even though I cannot divulge private information in a public post, there is this[1] article that states that the max is five transactional tasks into the task queues during a single transaction. It also says that the tasks are enqueued, rather than acting in parallel. Obviously, if they happen fast, they may appear as parallel. But in this case, they are slow from your perspective. And I think what you are asking is why they are throttled every second when they take ms to complete (as shown in the picture above), rather than what the ceiling should be. For example, in the picture, the 33rd second of 23:26 has 5 tasks and you should aim to reach that consistently if you want it to be faster (but it will never fully be parallel since they are enqueued). If you have them all in the same run_in_transaction() function, that could be the cause of it. It's hard to say what should and shouldn't be the best pattern due to the semantics of your application, but I think it might be best to look at it in reverse and try to figure why certain tasks can happen 5 times a second and then duplicate that as an appropriate pattern in your application. The bug that was confirmed previously was more of an outage. Engineers can relate those delays to outages. However if you are experiencing this three times with months apart, a secondary problem could be the root cause. I understand that your initial question was if others are having this issue and I will let others answer, but I did some research for that too and didn't find it to be common. If you need further specific assistance, it might best best to do a support ticket. Otherwise, try to isolate why certain tasks are faster than others in comparison with your business logic. [1] https://cloud.google.com/appengine/docs/standard/python/datastore/transactions#transactional_task_enqueuing -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/60e8b3d8-7129-4eb1-a098-74887cdcdcc4o%40googlegroups.com.
[google-appengine] Re: Not able to run dev_appserver with Python 3 only environment
Hello Ritesh, I looked in our internal issues and I did not find this error posted. It could be a new one. However, a quick Google search shows that someone here[1] downgraded to an older version of the SDK and it worked. Older versions of the SDK can be found here[2]. If it does work with an older version, it could mean that there is an issue. And we would need to report that in the issue tracker here[3], under "Cloud SDK issues". However, there are many other possibilities (too many not visible) and it would be good to isolate further without brew, etc.. Just for testing under which environment/config this happens. If you could, please let us know if it works with an older version of Cloud SDK and under which specific environment. Otherwise, this may need to be a support ticket. Thank you in advance. [1] https://www.reddit.com/r/googlecloud/comments/gdndx4/gcloud_sdk_update_from_28900_to_29000_on_mac/ [2] https://cloud.google.com/sdk/docs/downloads-versioned-archives#archive [3] https://cloud.google.com/support/docs/issue-trackers On Wednesday, August 12, 2020 at 6:37:14 AM UTC-4, Ritesh Nadhani wrote: > > Hello > > I am getting back to GAE after many years. Things have definitely changed. > As a new project, I got started with Python 37 runtime. My virtualenv is > created using: python3 -m venv ENV and thus does not have python2. > > When I try to run my app in local mode using dev_appserver.py (i am just > trying to play around with the sample app from the documentation), I get: > > dev_appserver.py app.yaml > ERROR: (dev_appserver) python2: command not found > > I installed google-cloud-sdk using brew with: brew cask install > google-cloud-sdk. > > As of now, my only requirement for my app is GAE and datastore and I am > hoping to be able to use the datastore emulator with the above. > > Ritesh > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/3c414bee-38e8-49fb-bc30-3d6b2796f5dbo%40googlegroups.com.
Re: [google-appengine] Re: Spring Boot + JSP + WAR + Java 11 + GAE Standard Environment = sad
Hello Todd, You are correct for the support of App Engine Standard with JAR and if you prefer WAR it's understandable. There is a possible workaround here[1] but that's for old Java 7 and I think now 8. There is an old discussion about that here[2]. Deploying it in Flex is the better option. However, there is an explanation here[3], under "What are the changes between the Java 8 and Java 11 standard runtimes?", as to why it's not compatible. It's due to the API. To be future-proof, it might be best to stay with the Flex engine. I understand Standard engine is cheaper, but the API is older. [1] https://cloud.google.com/appengine/docs/standard/java/configuration-files#an_example [2] https://github.com/GoogleCloudPlatform/cloud-code-intellij/issues/1912 [3] https://ludoch.github.io/java11.html -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/badd3a69-b476-43ae-8929-144bf7018bdco%40googlegroups.com.
[google-appengine] Re: Currently getting hundreds of 503 errors calling service from task queue
Hello Rob, Without taking other factors into consideration, you may be affected by this[1] issue. It shows that App Engine is affected. It might be best to engage GCP support for a better answer. However, you can also follow the status of the issue here[2]. Hope this helps. [1] https://status.cloud.google.com/incident/zall/20006 [2] https://b.corp.google.com/issues/161347350 On Thursday, July 23, 2020 at 9:17:25 AM UTC-4, Rob Curtis wrote: > > Hi, I'm suddenly getting hundreds of 503 errors when call task handler in > service. > > Is anyone else experiencing this outage? > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/14a1ef47-7834-4bcb-9ab5-e1200d9aa01bo%40googlegroups.com.
[google-appengine] Re: cloudsearch.googleapi is returning drastically wrong result count for a particular
Hello Sudeshna, Thank you for reporting this potential issue. Since you are mentioning that the API call is returning the wrong result count, it might be better to engage our engineers through another system. This[1] issue tracker system will allow you log a ticket with them. Then, we could follow-up with it. Thank you. [1] https://cloud.google.com/support/docs/issue-trackers -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/a62a0557-963c-44e6-8345-9a89f94402aao%40googlegroups.com.
[google-appengine] Re: How to protect for Host Header Injection in Endpoints on AppEngine
Hello Alexandru, Thank you for reporting this potential issue. It would be preferable to post this type of issue directly with our engineering team. You mentioned "As far as I understood from the Endpoints Team"... And I'm wondering if you already did post that with them? I searched and couldn't find such a post in our system. The process on how to do it is listed here[1] and it shows you how to access that ticketing system to submit your issue. Could you let me know if you've already created such an issue please? I don't want to create a duplicate. Then, we could follow-up with it. Thank you. [1] https://cloud.google.com/support/docs/issue-trackers -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/0302f2ac-c9db-4c6a-9e1f-c9f37793e95fo%40googlegroups.com.
Re: [google-appengine] Re: GAE Web app sending mail using SendGrid
Can you please explain how you did this ?
I'm still stuck with an error here : SendGrid sendgrid = new SendGrid(
System.getenv("SENDGRID_SENDER"));
Thanks
Le vendredi 7 septembre 2018 à 18:02:44 UTC+2, rashmi...@gmail.com a écrit :
> I have solved the problem. I had to use Urlfetch configuration in
> appengine-web.xml and use URLFetch class instead of HttpURLConnection. It
> works now. Thanks for the help!
>
>
> On Saturday, August 25, 2018 at 1:08:16 AM UTC+5:30, George (Cloud
> Platform Support) wrote:
>>
>> If you would like us to look more in-depth in the issue, you should write
>> a private message with your project ID. You can do this using the drop-down
>> menu of the reply button, at the top-right of the message window.
>>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/google-appengine/21e0c86a-886a-491b-a57d-cd4d72b4e02cn%40googlegroups.com.
[google-appengine] Strange behavior with my dispatch.yaml
Hi, I just want to understand how to manage correclty the dispatch.yaml. I have 2 services : - Service : default - app.yaml : runtime: php55 service: default api_version: 3 handlers: - url: /help/.* script: help.php - url: .* script: default.php - Service : admin - app.yaml : runtime: php55 service: admin api_version: 3 handlers: - url: .* script: admin.php Each PHP file, print the name of the file. For example, default.php shows "default" *So, my problem, is that I don't understand the following behavior in red.* First : dispatch: # Send all mobile traffic to the mobile frontend. - url: "*/admin/*" service: admin # Default service serves simple hostname request. - url: "*/" service: default result : - X.appspot.com -> default - X.appspot.com/help -> help - X.appspot.com/admin -> admin - admin-dot-X.appspot.com -> default - admin-dot-X.appspot.com/help -> admin - admin-dot-X.appspot.com/admin -> admin Someone, can explain me why I have this behavior ? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/7a8e5fd1-8eb1-4edd-ab6c-007815cf94cb%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Re: Get result of async operation to Google speech-to-text
I have the same problem with the response, it doesn't include any result. Does this mean it didn't understand anything from the audio file? I am sending a 40seconds linear16 raw PCM 16 bit signed file with clear voices. If there is a problem why isn't there an error reported? On Thursday, August 11, 2016 at 2:30:55 AM UTC+2, Bruno Leitão wrote: > > Hi all, > > I perform a async request to google speech-to-text, and now i do not know > how to get the result of operation. > > I already know that operation was successful > > Thanks > BL > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/d728426a-8304-4fb7-9746-31ccac51cbd7%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Re: X-Appengine- flags aren't functioning
Still working fine for me, the number of unresolved location is stable over time for the past month. (about 1 unresolved for 1,000 worldwide-but-mainly-US users) On Friday, December 26, 2014 9:30:38 AM UTC+1, Stuart Langley wrote: No they still work (or should). See the bottom of http://php-minishell.appspot.com/phpinfo for example ($_SERVER[HTTP_X_APPENGINE_CITY]) etc. On Thursday, 25 December 2014 07:05:39 UTC+11, Kaan Soral wrote: I depend on the X-Appengine- header flags to initialize a user's initial location, yet they return ?/0/default values right now, I suspect they mostly return these default values lately Why is this? / Is it expected? (I've tested multiple locations - e.g my location and cloudflare's locations, both are 0.0/0.0 latlon and ? for the other fields) -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Re: Channel API - Channels Created Costs
We've been using the Channel API for a while and since March 2014 pricing changes (http://googlecloudplatform.blogspot.fr/2014/03/google-cloud-platform-live-blending-iaas-and-paas-moores-law-for-the-cloud.html) Channel created are free. They used to be priced at 1 cent for 100 created, which appstats still seems to reflect. On Sunday, October 5, 2014 10:01:20 PM UTC+2, Mihail Russu wrote: You can use appstats https://cloud.google.com/appengine/docs/python/tools/appstats to find out and calculate costs for any of your GAE handlers/functions. A quick test reveals that creating a channel costs 10,000 micro-pennies (1 dollar equals 100 pennies, 1 penny equals 1 million micropennies), which is considered expensive since, if compared, 1 datastore read by key costs only 100 micropennies, so 1 cent USD would let you create 100 channels. Also, I am not sure what is the 90,040 (or 95,040 in my case) limit is about since considering that there are 1,440 minutes per day and according to the like you provided where it says that GAE allows 60 channels created per minute, that calculates to 86,400 MAX channels that can be created, no matter what the budget is (which is not that much if you think about it). It would be great if anyone could clarify this or point to any errors in my calculations. Thanks, Mihail. On Saturday, October 4, 2014 6:33:57 PM UTC-5, cr...@portical.com.au wrote: Hi, I have a paid app and can see the quota is 90,040 per day for channels created. https://console.developers.google.com/project/PROJECTNAME/appengine/quotadetails I have looked at all the pricing pages from cloud services and developer pricing pages and can not see any prices regarding the cost per channel created after the 100 free limit. https://cloud.google.com/appengine/docs/quotas#Channel this page only says the daily limit is Based on your budget but I do not see any costs anywhere on the web. Am I to assume that I get 90,040 free per day for having a paid app? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Re: New Pricing
What about Channel API? It used to be priced $0.01 / channel opened, but I no longer see it mentioned in the pricing page (neither as bundled or paid service): https://cloud.google.com/products/app-engine/ -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Re: How stable is pull queue task tagging for arbitrary tags?
I'm currently using the pull queue with tags, and have experienced TranscientError in the past. Thanks to support cases still being there I can go back in time with precise figures :) Here are my usage figures if it can help: Currently: Daily average 7-8 tasks/second enqueued over 2 tags, with a spike at 37 tasks/second over 5 minutes. One worker per tag that wakes up every 30 minutes to lease them. No transcient error. Oct 2012 (they probably improved the service since): We were using many different tags (basically regrouping events per user with several million users). 1,800 tasks/sec enqueued, can peak at 2,500. Up to 100 requests/seconds that will lease tasks from the same pull task queue - but on different tags. = lots of TranscientError and DeadlineExceededError. To mitigate this the solution was to spread the tasks over multiple task queues: 20 pull task queues instead of one, each pull task queue had 35 tasks added by second, and 4 requests per second were leasing and deleting tasks from that same queue. I did still have regularly DeadlineExceeeded errors mainly about the call to delete tasks. Retrying at once solves it most of the time. I still found a few TranscientError (mostly on tasks delete too) but in such rate that made them acceptable. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Re: How stable is pull queue task tagging for arbitrary tags?
I forgot to mention - but you may realize if you do the math - the figures about when I split to 20 task queues were obtained while testing with traffic split, so it did not get the whole 1,800req/sec traffic but a subset of it. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Re: 1.9.1 Pre-Release SDKs are now available.
You speak about the whole Performance Settings section, but I bet that part of it will remain, like settings for memcache service, and page speed service. Can you give more details just to make sure? On Monday, March 10, 2014 7:19:08 PM UTC+1, Richmond Manzana wrote: We want to inform you that the pre-release SDKs for Python, PHP and Java are now available. As previously announcedhttp://google-opensource.blogspot.com/2013/05/a-change-to-google-code-download-service.html in a Google code site announcement, new App Engine Binaries are no longer available at: http://code.google.com/p/googleappengine/downloads/list Older binaries will remain available at the code.google.com site. 1.9.1 Pre-release SDKs are now available at https://developers.google.com/appengine/downloads#Google_App_Engine_Pre_Release_SDKs Also, please see the pre-release notes below. Python PHP == - The Performance Settings section of the Application settings page in the Admin Console, Backends API and all backends related management tools are now deprecated and will be removed in a future release. Users of Backends are recommended to migrate to the App Engine Modules API, which provides a more flexible implementation of the same functionality. These settings are now all configurable via Modules configuration files. See the Modules documentation for more information: https://developers.google.com/appengine/docs/python/modules/ #Python_Configuration Java = - The Performance Settings section of the Application settings page in the Admin Console, Backends API and all backends related management tools are now deprecated and will be removed in a future release. Users of Backends are recommended to migrate to the App Engine Modules API, which provides a more flexible implementation of the same functionality. These settings are now all configurable via Modules configuration files. See the Modules documentation for more information: https://developers.google.com/appengine/docs/java/modules/#Java_Configuration Cheers, Richmond Manzana Technical Program Manager Google App Engine App Engine SDK - Pre-Release Notes -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Re: Snapchat
Just to emphasize what Kaan Soral said, without going into cheapness evaluation: the initial costs seem to be high, but as the traffic increases, the costs doesn't increase proportionally. So basically, appengine gets cost efficient with increased traffic, seems logical - so basically you have to excuse the initial daily ~10$ costs if you are keeping instances and memcached alive It goes way further than saving costs for keeping instances and memcache alive. When we launched one of our products, we had one engineer working on the back-end side (99% AppEngine). We had a few hundred daily active users. Then our product became very successful and we reached 5 million (!) daily active users. We still only had one engineer working on our back-end. @Tapir I'm sorry I can't really answer your 'Please share your GAE success stories!http://groups.google.com/group/google-appengine/t/945c56ecd47682d4' questions as I'm not allowed to share cost/revenue figures. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] App Engine Version 1.8.8 in control panel ???
They also seem to do so very carefully and progressively. It's been a few days I have some apps with instances in both versions. Usually there is a SDK pre-release around the same time though. Anyway I think many changes they do are not even mentioned in the release notes as it's about things we don't have control over... https://lh3.googleusercontent.com/-MaeObO6gFh4/UoNfAE47toI/G3A/xJ5EP_-iKqM/s1600/Screenshot+-+13-11-13_12-13-17.png On Wednesday, November 13, 2013 6:15:45 AM UTC+1, timh wrote: They always roll out new versions, and you have no control over it. Its a fact of life with appengine. On Wednesday, November 13, 2013 7:50:16 AM UTC+8, o...@haitov.com wrote: Do you know if there is some documentation about it anywhere ??? Can i disable this ?? I provide a paid service to customers, and reliability is a critical issue !!! I must say i'm not feeling too happy about the new undocumented version of GAE being tested on my app, specially without informing me about this prior to applying it ! Can someone at google please provide some information about this issue ? Thank you, Tomer On Wednesday, November 13, 2013 1:32:12 AM UTC+2, barryhunter wrote: Your a canary! Well your app is. You get to test out the new version before it released to everyone. If something is broken, report it. Otherwise enjoy it ;) On 12 November 2013 23:22, o...@haitov.com wrote: Hi logged in to my control panel and this is what i see ??!? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com. To post to this group, send email to google-a...@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Task target - specific Module
It works for me. The target parameter of a cron job definition currently will go to the default version of the specified module or the specified version of the default module. If you both have a URL for the Cron job that points to an specific module and a target parameter that points differently, the URL forwarding rule via dispatch.yaml will prevail here. There is currently no way to target a specific version on a non-default module using cron.yaml or dispatch.yaml. But that may work programmatically (when you enqueue the task you can also specify a target, and maybe version.module works) -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Redirect if login is under wrong domain.
I am trying to create a web app and I have it to where you are required to log in. When I change authentication to Google Apps Domain and put in the domain, if I am not logged into any account it sends me directly to a page with the following message: Error: Server ErrorThe server encountered an error and could not complete your request. If the problem persists, please reporthttp://code.google.com/appengine/community.html your problem and mention this error message and the query that caused it. How can I make it redirect to the Google sign in page and continue to redirect to the log in page if signed in under the wrong domain. My yaml file has the following but it still won't work. - url: .* script: main.app login: required auth_fail_action: redirect Thank you. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: 1.8.1 Pre-release SDKs Available.
Regarding: - The Task Queue async API is now a GA feature. The asynchronous methods improve utilization by allowing your app to add, lease and delete multiple tasks in parallel. Could you also update deferred.defer to expose an asynchronous method as well? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: Trusted Tester program for App Engine Datastore Backup to BigQuery Import
By the way, do you have any program about analyzing an application logs directly in BigQuery? That would be a great feature instead of having to write and manage jobs to read logs to write to Cloud Storage and to then import into BigQuery... On Wednesday, May 29, 2013 12:49:50 AM UTC+2, Takashi Matsuo (Google) wrote: Hi Michael, Sorry for the delay. I've just sent out another set of invitations. Let me know if you haven't received anything. -- Takashi On Tue, May 28, 2013 at 9:40 AM, Michael Chang mic...@firespotter.comjavascript: wrote: Hi Takashi, Is the trusted tester program still running for the Datastore to BigQuery import? I just applied but have not seen any updates regarding when we should hear back. Michael On Monday, October 1, 2012 5:34:21 PM UTC-7, Takashi Matsuo (Google) wrote: *Hi everyone, Do you want to quickly analyze your Datastore data with Google BigQuery? We are looking for a small group of trusted testers who can try out our new feature which allows you to import data from the experimental Datastore backup tool directly into BigQuery for analysis. If you’re interested, please fill out the formhttps://docs.google.com/spreadsheet/viewform?formkey=dHdpeXlmRlZCNWlYSE9BcE5jc2NYOUE6MQ [1] with more details about your use case.* * * *The BigQuery launch also includes support for JSON and its nested/repeated structure as well as significant improvements to the data loading pipeline. Check out their post on the Google Developers Bloghttp://googledevelopers.blogspot.com/2012/10/got-big-json-bigquery-expands-data.html[2] for more details on their latest release.** * *1. The application form for this TT program* https://docs.google.com/a/**google.com/spreadsheet/**viewform?formkey=** dHdpeXlmRlZCNWlYSE9BcE5jc2NYOU**E6MQhttps://docs.google.com/a/google.com/spreadsheet/viewform?formkey=dHdpeXlmRlZCNWlYSE9BcE5jc2NYOUE6MQ 2. The blog post about the new BigQuery release http://googledevelopers.**blogspot.jp/2012/10/got-big-** json-bigquery-expands-data.**htmlhttp://googledevelopers.blogspot.jp/2012/10/got-big-json-bigquery-expands-data.html * Thanks,* -- Takashi Matsuo | Developers Advocate | tma...@google.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com javascript:. To post to this group, send email to google-a...@googlegroups.comjavascript: . Visit this group at http://groups.google.com/group/google-appengine?hl=en . For more options, visit https://groups.google.com/groups/opt_out. -- Takashi Matsuo | Developers Programs Engineer | tma...@google.comjavascript: -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: Accumulated lots of data - deleting is is too expensive
I agree, for some Kinds we accumulated and do no longer use, we just don't delete them as doing so would cost more than keeping them for more than a year (or sometimes several years for light entities with several indexed properties), that is close to the anticipated remaining lifetime of the app. Removing stuff has little added value so paying a big check for it is not appealing. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Does GAE have Support? (other than this forum)
With a Premier account all our apps instances benefit from the reduced pricing ($0.05/hour) as if they were reserved ones. That was a good surprise for us (as we did not see it mentioned anywhere) and we actually made savings switching to Premier while no longer having to perform this annoying optimization with reserved instances. Premier also allowed us to pay our bills later (monthly invoice), which is great when you encounter growth and need more time for the money to come in (delays with Ad providers payments etc.). About the support, you can issue tickets with a priority. Depending on the priority we set, we use to have an answer within 10 minutes to 24 hours. Then the quality of the support we received so far highly depended on who would deal with our tickets (we always have had our tickets resolved but sometimes the support guy is more efficient and gives more insights). -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: What are the pros and cons of using Google App engine for my startup?
@Kaan: The limits can be increased as your app grows as long as you make a decent use of the resources. The GAE team will make sure your app is able to scale and assist you if you become that big (as you will have a Premier Account in this case). For instance, some of our apps have the following limits: URL Fetch 834,159,222 calls daily, 576,000 calls per minute. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/NNlXL_4zTZAJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: mapping Google Apps to Appengine error 1000
I had this issue a week ago: Make sure the domain you want to add is a primary domain or an alias of a primary domain. GAE does not support mapping to a secondary domain. If you map to an alias domain note that you will not be able to send mails with an admin sender from this domain. (Ex: if you want to send emails from no-re...@my-alias-domain.com) -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/GcX65VUghcYJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Undocumented quota: Log Records Received
I just got another undocumented quota... This one is more critical for us as it's not used for debugging like the logs one was. It's called File Call Count and the limit is 8,640,000 daily. I hope it can be increased. I opened a ticket about these undocumented quotas: http://code.google.com/p/googleappengine/issues/detail?id=7594 It'd be nice if we could build the list together until AppEngine team decides to provide it to us... http://stackoverflow.com/questions/10791242/what-are-appengine-hidden-quotas -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/ohAAwdarFo0J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Outage - 503 error requests not returning
I also got spikes of latency and errors for about 20 minutes 4 hours ago, on my four apps in production. Would be nice to have some details -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/_qLA8Jd82ysJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Re: Messages not delivered using Mail API
app-id; fp-spalife I opened a ticket for this: http://code.google.com/p/googleappengine/issues/detail?id=7124 Thanks -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/BoXkLQBfctkJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Messages not delivered using Mail API
Similar issue for us, for one of our apps it works fine, for one other, our test and stage apps works correctly but once in production mails are not delivered, or only a fraction of them. No errors, logging says it worked. Is there some throttling applied? The main difference with our production app is that it sends many more emails, about 17k per day. The other production app on which it's working sends between 5k and 6k per day. Our mails are signed using the new feature (DKIM for Google Apps sender), same sender between our different stage apps. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/nZ0SE_hm9IEJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Time for answer to production issue.
I posted mine one month ago and still no response: http://code.google.com/p/googleappengine/issues/detail?id=6707 I agree it's not critical but still an annoying issue, many production tickets have reported the same kind of issue without response. One month should be enough to have at least a short update from GAE team... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: is app engine down? I'm seeing a huge number of deadline errors and our app failing to server, hr datastore
Also get big issues simultaneously with all our apps 10 min ago. Lot of errors, drop in requests/sec while peak of active instances. http://code.google.com/status/appengine showed an anomaly for Python, but now it's back to Elevated and I guess it'll disappear as usually... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Why Google Groups doesn't send me Abridged Emails of GAE group and GAE Java groups any more?
Same issue for me On 30 jan, 09:32, radomir radomi...@gmail.com wrote: I have exactly the same issue. Didn't get abridged emails for GAE and GAE Java groups for at least one week. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Undocumented quota: Log Records Received
Hi We noticed a few days ago that starting from the middle-end of the day, a new quota line now appears in one of our app dashboard: Log Records Received - 95% - 946,267 of 1,000,000 This line is not there in other apps or at the beginning of the day. So it must appear only because we are going to reach the quota limit. That's fine, but this quota does not appear in the Quota details section of the dashboard, and is not mentioned in the quotas documentation neither in the LogService documentation. In other words I have no clue of what is it and could not anticipate it. We only have an average 15 requests/sec so it's more than 1 million requests a day, the quota can't be that we can't log more that 1M requests right? I think it may be linked to the new LogService API that we are starting to use. Is it because we read too many records using this API? Can someone shed some light on this? Did I miss something? Thanks -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Why are several production issues related to DeadlineExceededErrors being ignored?
I was in the same position in August, with our apps on M/S. The DEE errors began to appear, then were more and more frequent and finally had a high impact on our app availability. We migrated to HRD writing our own mapreduce and then all went fine (except for the costs but hey, was the price for a more reliable solution). But since something like mid-December, exactly the same symptoms began to appear again. Less frequent and not really impacting our availability yet, but still there. And no official acknowledgement of this issue so far... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Why are several production issues related to DeadlineExceededErrors being ignored?
Cezary, I was in the same position in August, with our apps on M/S. The DEE errors began to appear, then were more and more frequent and finally had a high impact on our app availability. Our warmup requests, that only load code, were randomly taking from a few secs to more than 30sec and so triggering the DEE. I could not figure out how it could be linked to the datastore, but Google team only had one word to mouth: move to HRD. No explanation at that time and made so sense for me but I decided to trust them. We migrated to HRD writing our own mapreduce and then all went fine (except for the costs that went higher but hey, was the price for a more reliable solution). But since something like mid-December, exactly the same symptoms begin to appear again. Less frequent and not really impacting our availability yet, but still there. And no official acknowledgement of this issue so far... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Why are several production issues related to DeadlineExceededErrors being ignored?
Cezary, I was in the same position in August, with our Python apps on M/S. The DEE errors began to appear, then were more and more frequent and finally had a high impact on our app availability. Our warmup requests, that only load code, were randomly taking from a few secs to more than 30sec and so triggering the DEE. I could not figure out how it could be linked to the datastore, but Google team only had one word to mouth: move to HRD. No explanation at that time and made so sense for me but I decided to trust them. We migrated to HRD writing our own mapreduce and then all went fine (except for the costs that went higher but hey, was the price for a more reliable solution). But since something like mid-December, exactly the same symptoms begin to appear again. Less frequent and not really impacting our availability yet, but still there. And no official acknowledgement of this issue so far... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Random DeadlineExceed Errors with python27 and HRD
you are not alone, I've got the same issue. It's being discussed here too: http://groups.google.com/group/google-appengine/browse_thread/thread/e176afd47b27a6a9/ On 18 jan, 18:46, sergio.jar...@gmail.com sergio.jar...@gmail.com wrote: Hi, I'm seeing random DeadlineExceeded errors with python27 and HRD since about a week ago, I do have threadsafe False so it's not the threadsafe latency issue. I can provide the app id off-list. Here is a (partial) trace File /base/python27_runtime/python27_lib/versions/1/google/appengine/ ext/db/__init__.py, line 2084, in fetch return list(self.run(limit=limit, offset=offset, **kwargs)) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/ext/db/__init__.py, line 2237, in next return self.__model_class.from_entity(self.__iterator.next()) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/datastore/datastore_query.py, line 2655, in next next_batch = self.__batcher.next() File /base/python27_runtime/python27_lib/versions/1/google/ appengine/datastore/datastore_query.py, line 2525, in next return self.next_batch(self.AT_LEAST_ONE) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/datastore/datastore_query.py, line 2562, in next_batch batch = self.__next_batch.get_result() File /base/python27_runtime/python27_lib/versions/1/google/ appengine/api/apiproxy_stub_map.py, line 592, in get_result return self.__get_result_hook(self) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/datastore/datastore_query.py, line 2317, in __query_result_hook self._conn.check_rpc_success(rpc) File /base/python27_runtime/python27_lib/versions/1/google/ appengine/datastore/datastore_rpc.py, line 1176, in check_rpc_success rpc.wait() File /base/python27_runtime/python27_lib/versions/1/google/ appengine/api/apiproxy_stub_map.py, line 535, in wait assert self.__rpc.state == apiproxy_rpc.RPC.FINISHING, repr(self.state) DeadlineExceededError -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Why are several production issues related to DeadlineExceededErrors being ignored?
Thanks for the shot! Here are some comments: - isn't the limit 60sec instead of 15sec? - We indeed have the pending latency slider set to default, but out of our 12 instances we have 3 resident ones, so the slider will have little effect on our application's performance. - Requests that DEE typically have this signature: ms=63498 cpu_ms=1097 api_cpu_ms=0 cpm_usd=0.030639 loading_request=1 pending_ms=373 exit_code=104 So it did not spent much time waiting to be served... And the DEE is raised during the import phase. - Our warmup requests, that import most of our modules then return a simple ok string, take in average 2100ms to complete (min 860, max 3800). But during DEE spikes they can also raise DEE. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Why are several production issues related to DeadlineExceededErrors being ignored?
Kenneth: I'm talking HRD -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Why are several production issues related to DeadlineExceededErrors being ignored?
+1 On 14 jan, 03:54, GAEfan ken...@gmail.com wrote: +1 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: DeadlineExceeded errors haunting every now and then
Seeing spikes or DEE errors too with my HRD app recently, while it has been right for several months. Reminds me of the symptoms seen when we used M/S: suddenly a burst of DEE and many instances gets killed, with these two logging messages: I 2011-12-08 13:03:35.164 This request caused a new process to be started for your application, and thus caused your application code to be loaded for the first time. This request may thus take longer and use more CPU than a typical request for your application. W 2011-12-08 13:03:35.164 A serious problem was encountered with the process that handled this request, causing it to exit. This is likely to cause a new process to be used for the next request to your application. If you see this message frequently, you may be throwing exceptions during the initialization of your application. (Error code 104) -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Doom day
You can look at this thread if you want more details about instances settings: http://groups.google.com/group/google-appengine/browse_thread/thread/260e7cefc6e5d482/bf9b4a4a804a3091 On 2 déc, 18:04, Gerald Tan woefulwab...@gmail.com wrote: If you set Max Idle Instance to automatic you will always pay for the total number of instances. Looking at your chart I'd set Max Idle Instance to something between 20-25 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Doom day
We are still using Python25 too and don't feel confident moving our production apps to 2.7 when seeing issues like this: http://code.google.com/p/googleappengine/issues/detail?id=6401 or this: http://code.google.com/p/googleappengine/issues/detail?id=6323 or the one Kaan linked to... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.6.0 is now launched
Alfred, I also noticed the split, but I'm talking about something else. What is being written are entities from inside the google.appengine.api.logservice package (_LogRecord, _LogLine, inside a namespace called _Logs) I suspect this may announce some feature about managing logs using some APIs, which would be great, but as this is not available yet and slows down performances of the dev server as I described, I just commented out any writes about these entities in the SDK sources (google.appengine.api.logservice.logservice_stub.py). This fixed my issues. On 19 nov, 04:08, PK p...@gae123.com wrote: Alfred/Alexis, have you gotten anywhere with this issue? Is there a case opened? I agree that what Alexis reports is indeed a very serious issue. Please take a look at issue 6355http://code.google.com/p/googleappengine/issues/detail?id=6355for the way it manifests itself in my environment. Thanks, PK -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.6.0 is now launched
I've got a small issue with this release: the development server is writing many more entities to the local datastore (log entries?). As I have a hook on datastore write ops to log what is put and when, it's polluting my logs but that's not the point: My issue is that the local datastore file is getting bigger and bigger while I run requests on my dev server. Previously, this file size was stable. Now when I restart the dev server, it takes much more time as it have to load a bigger datastore file (several MB), full of things I don't use. Is there a way to prevent the dev server from adding these entities? Thanks On 7 nov, 22:34, Gregory D'alesandre gr...@google.com wrote: There isn't a summary of changes as it was a complete re-write, that being said if you want to see the old terms you can find them here:http://code.google.com/appengine/terms_4.html Thanks, Greg On Mon, Nov 7, 2011 at 1:31 PM, pdknsk pdk...@googlemail.com wrote: Is there a summary of the TOS changes available? I accepted without reading (as many probably have). -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: New Billing: Absolutely make sure you set Max Idle Instances to a fixed value
I have not read in details previous posts but would like to share some figures on this subject. I agree that setting max idle to automatic is very expensive, in my case I adjusted the scheduler to get an acceptable balance between latency and price. In order to do so, I've downloaded logs from the application and studied them. The pending_ms tag being important here. Instances graph: http://dl.dropbox.com/u/497622/instances.png I changed the max idle instances from automatic to 20. Before change: -- Out of 214903 studied requests we have: (from 2011-10-13 09:23:07 to 2011-10-13 09:46:31) Average ms: 303 Pending requests: 2.4 % Global Average pending_ms: 45 Pending Average pending_ms: 1896 Loading requests: 0.0 -- 1h after the change: -- Out of 270115 studied requests we have: (from 2011-10-13 11:25:56 to 2011-10-13 11:55:46) Average ms: 345 Pending requests: 17.3 % Global Average pending_ms: 90 Pending Average pending_ms: 522 Loading requests: 0.3 -- So reducing the max idle instances settings had the following impact: - general latency increase of only 50ms - but 17% of the requests now wait 500ms before being served This appears the be acceptable for our application, and the price reduction is really worth it (from 240 instances down to 60!). Having an even lower max idle instance setting (10? 5?) would further increase latency while the price reduction would not be significant (as we have many instances running). -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Over quota exception even though plenty of quota is available?
I've already got these exceptions too, while running mapreduce. I fixed this by reducing the rate at which workers were executed. Here are the issue's details: http://code.google.com/p/googleappengine/issues/detail?id=5608 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: New Billing: Absolutely make sure you set Max Idle Instances to a fixed value
Hi Will, I used an appcfg.py command, like this: appcfg.py --num_days=1 --append --include_all request_logs path to your app root directory myAppLogs.txt On 14 oct, 17:47, Will vocalster@gmail.com wrote: Alexis, Where can I find and download the said logs? Thanks, Will On Fri, Oct 14, 2011 at 1:30 AM, Alexis alexis.hanico...@gmail.com wrote: I have not read in details previous posts but would like to share some figures on this subject. I agree that setting max idle to automatic is very expensive, in my case I adjusted the scheduler to get an acceptable balance between latency and price. In order to do so, I've downloaded logs from the application and studied them. The pending_ms tag being important here. Instances graph: http://dl.dropbox.com/u/497622/instances.png I changed the max idle instances from automatic to 20. Before change: -- Out of 214903 studied requests we have: (from 2011-10-13 09:23:07 to 2011-10-13 09:46:31) Average ms: 303 Pending requests: 2.4 % Global Average pending_ms: 45 Pending Average pending_ms: 1896 Loading requests: 0.0 -- 1h after the change: -- Out of 270115 studied requests we have: (from 2011-10-13 11:25:56 to 2011-10-13 11:55:46) Average ms: 345 Pending requests: 17.3 % Global Average pending_ms: 90 Pending Average pending_ms: 522 Loading requests: 0.3 -- So reducing the max idle instances settings had the following impact: - general latency increase of only 50ms - but 17% of the requests now wait 500ms before being served This appears the be acceptable for our application, and the price reduction is really worth it (from 240 instances down to 60!). Having an even lower max idle instance setting (10? 5?) would further increase latency while the price reduction would not be significant (as we have many instances running). -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Large number of datastore puts
We also have a large number of Datastore Write ops in our sample bill, and we figured out that it was likely due to properties with indexed=True thanks to the Quota Details admin page that gives more detail: Datastore CPU Time 0% 431.12 of Unlimited CPU hours Datastore Entity Fetch Ops 0% 8,288,728 of Unlimited Datastore Entity Put Ops0% 4,159,838 of Unlimited Datastore Entity Delete Ops 0% 1,072,478 of Unlimited Datastore Index Write Ops 0% 37,251,177 of Unlimited Datastore Query Ops 0% 642,110 of Unlimited Datastore Key Fetch Ops 0% 4,403,176 of Unlimited Here we clearly see that it's Datastore Index Write Ops that is responsible for an expensive bill. We will see soon the impact of having indexed=False on most properties. Hope it helps On 7 sep, 13:23, Stephen sdeasey+gro...@gmail.com wrote: On Tue, Sep 6, 2011 at 10:05 PM, Jason Collins jason.a.coll...@gmail.com wrote: We are seeing a lot more datastore write operations than we can account for (375M / day). Still trying to get to the bottom of it because it makes for a scary line item on the new sample bills. Divide 375M writes by number of request to get the average writes per request. Use code similar to the following to log requests with more than average writes: https://gist.github.com/715284 Should home in on the write-happy code pretty quickly. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Are my QPS and latency typical? (related to new pricing)
Hi, I also have noticed something odd between QPS and latency, although my numbers are not as bad as yours. I generally have stats like this: Total number of instances Average QPS*Average Latency*Average Memory 210 total 0.599 252.3 ms 72.1 MBytes With a latency of 0.25sec, I'd expect to have a QPS close to 4, and hence reducing the number of instances needed. But QPS is still very low... What makes QPS and Latency so different? On 1 sep, 17:04, jon jonni.g...@gmail.com wrote: Hi guys, I may be missing something here but looking at the below screenshot of my instances, it seems like GAE is overly eager to add instances. Each instance seems to be serving a request every 8 seconds approximately, which feels high. Do the numbers look right to you? Thanks in advance, Jonni https://lh3.googleusercontent.com/-R73wUepWYAA/Tl-eYELjwMI/AA... -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Crazy number of Deadine Exceeded exceptions recently
We currently experience an outage longer than previous ones... (about 30min this time) Which is an opportunity to study logs with more detail (as I can get the debug, info levels of recent requests). I've noticed that while the global latency is high, requests that do not write on datastore are successful while the ones that perform some writes raises the hard DeadlineExceeded error. On 3 août, 21:47, Will Reiher wrele...@gmail.com wrote: To further my original comment... While it seems to be back to an almost normal state when it was really bad I was getting DeadlineExceeded errors on Warm ups and other otherwise trivial requests. Most of my issues were not related to any offline task work - in fact for the most part tasks seemed to be the most stable. As I said originally this included deployments that had been stable for 30+ days and were all of a sudden throwing way more Deadline exceptions. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Crazy number of Deadine Exceeded exceptions recently
We are using Python, and we are not using back-ends or taskqueue. But most of our requests fetch entities of the same kind, so a kind- locking can be relevant. I'll setup timeouts on datastore operations to see if this is what is going wrong (and looks good to setup these anyway). Seems logical to have pending_ms values if the app have no instances available instead of the one hundred that used to serve the traffic... however requests should be able to perform in less than 20sec. Nearly all the requests failing with this error, and while the instances get killed off, have this pending_ms value, and don't have the loading_request=1. So I'm not sure whether these DeadlineExceeded errors come first or as a consequence of the instances being killed: they are not loading_request and the warning in the logs says it may kill the instance, but we have these pending_ms showing that we already lack of instances. The traffic is very steady. On 3 août, 05:49, Robert Kluin robert.kl...@gmail.com wrote: Interesting. I've been seen exactly the same strange behavior across several apps as well. Suddenly instances will get killed and restarted in large batches. This happens even with low request latency, small memory usage (similar to yours 50mb), low error rates, and steady traffic. I pretty convinced this is tied to the scheduler changes they've been making over the past few weeks. As a side note, the pending_ms value (9321) indicates that the request sat there waiting to be serviced for quite a long time. That won't leave as much time to respond to the requests. Do you always see bursts of those when your instances get killed off? Are you getting big spikes in traffic when this happens or is it steady? Robert On Tue, Aug 2, 2011 at 05:24, Alexis alexis.hanico...@gmail.com wrote: Hi, I've got a similar issue: lots of DeadlineExceeded errors since a few weeks. I'm on the master-slave datastore too, but what I'm reporting happened again one hour ago. These errors happen in bursts, and I recently realized that it was in fact shutting down ALL instances of the application. (In the logs, I also have this warning: A serious problem was encountered with the process that handled this request, causing it to exit. This is likely to cause a new process to be used for the next request to your application. If you see this message frequently, you may be throwing exceptions during the initialization of your application. (Error code 104)) This does not happen when an instance is spinning up but after several hours. The trace I get along with the DeadlineExceeded errors show that it happens in the second phase: while the app is trying to fallback gracefully because of an other error (that does not appears in logs). Request reported processing time can be like this: ms=100878 cpu_ms=385 api_cpu_ms=58 cpm_usd=0.010945 pending_ms=9321 Here is a screenshot of the admin page, showing that all instances have been shut down about 7 minutes ago, even resident ones: http://dl.dropbox.com/u/497622/spinDown.png The app do work in batches (although not always small ones). But request processing time is usually good enough (see average latency on the screen shot). I'm trying things on my testing applications to see what can be wrong but it's still not clear for me and I'm running short of ideas... Any suggestions? On 2 août, 06:21, Robert Kluin robert.kl...@gmail.com wrote: Hi Will, I assume this is on the master-slave datastore? I think there were a number of large latency spikes in both the datastore and serving last week. Some things to try: - do work in smaller batches. - if you're doing work serially, do it in batches. - use async interfaces to do work in batches, but in parallel using async. http://code.google.com/appengine/docs/python/datastore/async.html Robert On Fri, Jul 29, 2011 at 18:35, Will Reiher wrele...@gmail.com wrote: I'm trying to debug this issue but I keep hitting a wall. I keep trying new things on one of my deployments to see if I an get the number of errors down but nothing seems to help. It all started in the last week or so ago. I also have some existing deployments that I have not changed and are seeing these same errors while the code was never changed and had been stable. 1. This is is happening on existing code that has not changed recently 2. The DeadlineExceededErrors are coming up randomly and at different points in the code. 3. Latency is pretty high and app engine seems to be spawning a lot of new instances beyond my 3 included ones. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/g_C4iPzPeo4J. To post to this group, send email to google-appengine
[google-appengine] Re: Crazy number of Deadine Exceeded exceptions recently
Some more details while searching the logs: The latest burst (we have about one every 1-2 hours) shows this: We had about 90 instances up. We got DeadlineExceeded error during two minutes. During the first 30seconds, 50 successive error requests have no pending_ms. Then during the next 1m30 they nearly all have pending_ms and one third are loading requests. So my guess is that it's not linked to the scheduler: something goes wrong and requests are killing the instances during the first 30 seconds, then killing the remaining ones with an increased latency until what goes wrong is resolved. On 3 août, 10:20, Alexis alexis.hanico...@gmail.com wrote: We are using Python, and we are not using back-ends or taskqueue. But most of our requests fetch entities of the same kind, so a kind- locking can be relevant. I'll setup timeouts on datastore operations to see if this is what is going wrong (and looks good to setup these anyway). Seems logical to have pending_ms values if the app have no instances available instead of the one hundred that used to serve the traffic... however requests should be able to perform in less than 20sec. Nearly all the requests failing with this error, and while the instances get killed off, have this pending_ms value, and don't have the loading_request=1. So I'm not sure whether these DeadlineExceeded errors come first or as a consequence of the instances being killed: they are not loading_request and the warning in the logs says it may kill the instance, but we have these pending_ms showing that we already lack of instances. The traffic is very steady. On 3 août, 05:49, Robert Kluin robert.kl...@gmail.com wrote: Interesting. I've been seen exactly the same strange behavior across several apps as well. Suddenly instances will get killed and restarted in large batches. This happens even with low request latency, small memory usage (similar to yours 50mb), low error rates, and steady traffic. I pretty convinced this is tied to the scheduler changes they've been making over the past few weeks. As a side note, the pending_ms value (9321) indicates that the request sat there waiting to be serviced for quite a long time. That won't leave as much time to respond to the requests. Do you always see bursts of those when your instances get killed off? Are you getting big spikes in traffic when this happens or is it steady? Robert On Tue, Aug 2, 2011 at 05:24, Alexis alexis.hanico...@gmail.com wrote: Hi, I've got a similar issue: lots of DeadlineExceeded errors since a few weeks. I'm on the master-slave datastore too, but what I'm reporting happened again one hour ago. These errors happen in bursts, and I recently realized that it was in fact shutting down ALL instances of the application. (In the logs, I also have this warning: A serious problem was encountered with the process that handled this request, causing it to exit. This is likely to cause a new process to be used for the next request to your application. If you see this message frequently, you may be throwing exceptions during the initialization of your application. (Error code 104)) This does not happen when an instance is spinning up but after several hours. The trace I get along with the DeadlineExceeded errors show that it happens in the second phase: while the app is trying to fallback gracefully because of an other error (that does not appears in logs). Request reported processing time can be like this: ms=100878 cpu_ms=385 api_cpu_ms=58 cpm_usd=0.010945 pending_ms=9321 Here is a screenshot of the admin page, showing that all instances have been shut down about 7 minutes ago, even resident ones: http://dl.dropbox.com/u/497622/spinDown.png The app do work in batches (although not always small ones). But request processing time is usually good enough (see average latency on the screen shot). I'm trying things on my testing applications to see what can be wrong but it's still not clear for me and I'm running short of ideas... Any suggestions? On 2 août, 06:21, Robert Kluin robert.kl...@gmail.com wrote: Hi Will, I assume this is on the master-slave datastore? I think there were a number of large latency spikes in both the datastore and serving last week. Some things to try: - do work in smaller batches. - if you're doing work serially, do it in batches. - use async interfaces to do work in batches, but in parallel using async. http://code.google.com/appengine/docs/python/datastore/async.html Robert On Fri, Jul 29, 2011 at 18:35, Will Reiher wrele...@gmail.com wrote: I'm trying to debug this issue but I keep hitting a wall. I keep trying new things on one of my deployments to see if I an get the number of errors down but nothing seems to help. It all started in the last week or so ago. I
[google-appengine] Re: Crazy number of Deadine Exceeded exceptions recently
Hi, I've got a similar issue: lots of DeadlineExceeded errors since a few weeks. I'm on the master-slave datastore too, but what I'm reporting happened again one hour ago. These errors happen in bursts, and I recently realized that it was in fact shutting down ALL instances of the application. (In the logs, I also have this warning: A serious problem was encountered with the process that handled this request, causing it to exit. This is likely to cause a new process to be used for the next request to your application. If you see this message frequently, you may be throwing exceptions during the initialization of your application. (Error code 104)) This does not happen when an instance is spinning up but after several hours. The trace I get along with the DeadlineExceeded errors show that it happens in the second phase: while the app is trying to fallback gracefully because of an other error (that does not appears in logs). Request reported processing time can be like this: ms=100878 cpu_ms=385 api_cpu_ms=58 cpm_usd=0.010945 pending_ms=9321 Here is a screenshot of the admin page, showing that all instances have been shut down about 7 minutes ago, even resident ones: http://dl.dropbox.com/u/497622/spinDown.png The app do work in batches (although not always small ones). But request processing time is usually good enough (see average latency on the screen shot). I'm trying things on my testing applications to see what can be wrong but it's still not clear for me and I'm running short of ideas... Any suggestions? On 2 août, 06:21, Robert Kluin robert.kl...@gmail.com wrote: Hi Will, I assume this is on the master-slave datastore? I think there were a number of large latency spikes in both the datastore and serving last week. Some things to try: - do work in smaller batches. - if you're doing work serially, do it in batches. - use async interfaces to do work in batches, but in parallel using async. http://code.google.com/appengine/docs/python/datastore/async.html Robert On Fri, Jul 29, 2011 at 18:35, Will Reiher wrele...@gmail.com wrote: I'm trying to debug this issue but I keep hitting a wall. I keep trying new things on one of my deployments to see if I an get the number of errors down but nothing seems to help. It all started in the last week or so ago. I also have some existing deployments that I have not changed and are seeing these same errors while the code was never changed and had been stable. 1. This is is happening on existing code that has not changed recently 2. The DeadlineExceededErrors are coming up randomly and at different points in the code. 3. Latency is pretty high and app engine seems to be spawning a lot of new instances beyond my 3 included ones. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/g_C4iPzPeo4J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Using Model.get_or_insert to create unique values for entities
Hello everybody, I needed to store some information about my users (I
don't need to use Google accounts) and created a model called FbUser,
each entity in the model is a fbuser and each fbuser has a unique user
uid, a field I call 'uid'.
A uid can't appear more than once in the datastore but it could
eventually change for a user, so it's not inmutable. It can't be used
directly as a key_name because it always starts with a number, in fact
is always a ten digit number.
I want to make sure that I only store unique uid's in the datastore so
I thought about Model.get_or_insert:
http://code.google.com/appengine/docs/datastore/modelclass.html#Model_get_or_insert
and found this suggestion by Dado (thanks a lot for it):
http://groups.google.com/group/google-appengine/browse_thread/thread/9f956ebcc39dd80f/bf5bf1edd53a4410?lnk=gstq=Model.get_or_insert#bf5bf1edd53a4410
I implemented it for my case like this:
class FbUser(db.Model):
You can then call FbUser.get_or_insert_by_uid('foo') and get back an
FbUser instance with that unique identifier (it gets created if it
does not yet exists).Model.get_or_insert is automatically wrapped in
a transaction
uid = db.StringProperty(required=True)
@staticmethod
def get_or_insert_by_uid(uid):
# prefix with unique identifier to qualify and avoid beginning
with digits, which datastore does not accept
key_name = 'uid:'+uid
return FbUser.get_or_insert(key_name, uid=uid)
And then I can 'get or create' a fbuser with uid '123' using this:
FbUser.get_or_insert_by_uid('123')
I've tested and it works but I want to be sure if this is the right
way of doing it. What do you think?
Thanks!
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
