[google-appengine] Re: App Engine Standard - app:invalid port 8080

[Jason Collins]

To be clear: I meant the App Engine message was pretty misleading. Your 
message was fine. :)


On Monday, 7 February 2022 at 09:55:13 UTC-8 zhenyang@gmail.com wrote:

> Sorry about the message.
>
> Thank you for the information.
> I will try to bind directly to port 8080.
>
> Thanks Jason
> On Monday, February 7, 2022 at 10:02:54 AM UTC+8 jason.a...@gmail.com 
> wrote:
>
>> That's a pretty misleading message; don't worry about it.
>>
>> If you bind to $PORT, you'll get an nginx instance running in front of 
>> your application.
>> If you don't want the nginx instance, you can bind directly to port 8080.
>>
>>
>> On Sunday, 6 February 2022 at 08:41:31 UTC-8 zhenyang@gmail.com 
>> wrote:
>>
>>> Hello, 
>>>
>>> I'm trying to deploy my react app to app engine using port 8080.
>>>
>>> The logs payloads as below:
>>> 1. textPayload: "[start] 2022/02/06 04:52:28.768200 Waiting for network 
>>> connection open. Subject:"app/invalid" Address:127.0.0.1:8080"
>>> 2.  textPayload: "[start] 2022/02/06 04:52:28.768200 Waiting for 
>>> network connection open. Subject:"app/valid" Address:127.0.0.1:8081"
>>>
>>> NGINX started on port 8080.
>>>
>>> I'm keep getting CANNOT GET/ from gcloud app browse.
>>>
>>> I could not find any resources about app:invalid issue, I do not how to 
>>> approach this issue.
>>>
>>> What should I do now?
>>>
>>>
>>>  
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/dd054037-a191-437d-a879-315bb6b76c5dn%40googlegroups.com.

[google-appengine] Re: App Engine Standard - app:invalid port 8080

[Jason Collins]

That's a pretty misleading message; don't worry about it.

If you bind to $PORT, you'll get an nginx instance running in front of your 
application.
If you don't want the nginx instance, you can bind directly to port 8080.


On Sunday, 6 February 2022 at 08:41:31 UTC-8 zhenyang@gmail.com wrote:

> Hello, 
>
> I'm trying to deploy my react app to app engine using port 8080.
>
> The logs payloads as below:
> 1. textPayload: "[start] 2022/02/06 04:52:28.768200 Waiting for network 
> connection open. Subject:"app/invalid" Address:127.0.0.1:8080"
> 2.  textPayload: "[start] 2022/02/06 04:52:28.768200 Waiting for network 
> connection open. Subject:"app/valid" Address:127.0.0.1:8081"
>
> NGINX started on port 8080.
>
> I'm keep getting CANNOT GET/ from gcloud app browse.
>
> I could not find any resources about app:invalid issue, I do not how to 
> approach this issue.
>
> What should I do now?
>
>
>  
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/1c415c2d-1ea0-43f4-a712-51cee64250fdn%40googlegroups.com.

Re: [google-appengine] Updates from the Google App Engine team (Fall 2021)

[Jason Collins]

Yes, it is the same scheduler (this is all on GAE Standard, right?) Of
course, that scheduler has lots of knobs.

If you can build a repro, can you work with support? If anything, the
concurrency of GAE Python 3 should be better than GAE Python 2.

On Fri, Nov 5, 2021 at 3:36 PM Joshua Smith 
wrote:

> I don't have threadsafe in my 3.7 app's app.yaml. Just runtime and
> instance class. (I do have it in my 2.7 app, of course.)
>
> I'll try playing with max concurrent threads, but if I bump it from 10
> (the default) to 16, that really just changes it from 6 request to start an
> instance, to 9. Hard to see how that would make any difference.
>
> As I said, the 2.7 only starts one more instance when I hammer it, and it
> doesn't end up giving it any work. That's very different behavior from the
> 3.7 app which starts a bunch of instances and spreads the work to all of
> them (resulting in all the requests taking at least 5 seconds to fulfill,
> since the instance takes 5s to start up). This is clearly visible in the
> logs I included in the first message.
>
> You're positive the scheduler is the same?
>
> -Joshua
>
> On Nov 5, 2021, at 5:08 PM, Jason Collins 
> wrote:
>
> The scheduler is the same.
>
> This might have to do with `threadsafe` in app.yaml, which is complicated
> and confusing. I'd suggest removing `threadsafe` if it's present, then try
> setting the max_concurrent_requests to something higher (like 2x your
> number of workers):
> https://cloud.google.com/appengine/docs/standard/python/config/appref#max_concurrent_requests
>
>
> On Fri, Nov 5, 2021 at 1:48 PM Joshua Smith 
> wrote:
>
>> I figured out from the logs (since it prints a message for each worker it
>> starts) that 3.7 is starting 8 workers by default. So adding the entrypoint
>> with defaults has no effect.
>>
>> However, I did some experiments and I think I can characterize the
>> difference between 2.7 and 3.7 when it comes to scaling.
>>
>> When first landing on my app's page it shows a list of thumbnails. These
>> come from the datastore, with each thumbnail making a HTTP request (GET
>> /image?which=1, GET /image?which=2, etc.). If we:
>>
>> 1. Query for all of these at once so the server gets hammered with
>> simultaneous requests:
>> - 2.7 starts one extra instance, and while that's happening the images
>> are all served by the instance that already existed; the new instance never
>> actually serves any of those image requests.
>> - 3.7 starts a lot of instances; it dumps a bunch of requests to each of
>> them (one per worker thread, I presume), all of which are going to take 5
>> seconds because that's how long it takes each instance to start.
>>
>> The net result is the user of the 2.7 app sees all the images load right
>> away, while the 3.7 app gets them over the course of 15 seconds or so, and
>> sometimes there are timeout errors.
>>
>> 2. Query for these with a delay that's long enough to avoid overlapping
>> requests:
>> - Neither starts extra instances.
>>
>> The user sees all the images load pretty quick, but not quite as quick as
>> 2.7 with the first approach.
>>
>> My conclusion is that the change is that the old scheduler wouldn't use
>> an instance until it was running, whereas the new scheduler queues up
>> requests for instances that are still starting up. The old scheduler
>> doesn't start a new instance until the one it started most recently is
>> running, whereas the new scheduler starts a new instance every time the
>> request queue fills up for existing instances.
>>
>> If you can serve a burst of requests in the time it take an instance to
>> start, the old scheduler will get all those served. But the new scheduler
>> will choke and perform very badly.
>>
>> I'm not saying that the old scheduler was better. It just handled this
>> particular case (cold start with a firehose) a lot better. Once the two
>> systems are running at scale, I wouldn't expect there to be much difference.
>>
>> -Joshua
>>
>>
>> On Nov 5, 2021, at 3:29 PM, Joshua Smith 
>> wrote:
>>
>> I think it's telling me that since my P3.7 app has a F4_1G instance, I
>> should be configuring it to run gunicorn with 8 workers. I don't see any
>> information about the default number of workers. I guess I can do an
>> experiment to see what explicitly setting that does...
>>
>> On Nov 5, 2021, at 3:17 PM, Jason Collins 
>> wrote:
>>
>> It sounds like there might be some differences due to concurrency between
>> 2.7 and 3.7/8/9. There are some notes on how to

Re: [google-appengine] Updates from the Google App Engine team (Fall 2021)

[Jason Collins]

The scheduler is the same.

This might have to do with `threadsafe` in app.yaml, which is complicated
and confusing. I'd suggest removing `threadsafe` if it's present, then try
setting the max_concurrent_requests to something higher (like 2x your
number of workers):
https://cloud.google.com/appengine/docs/standard/python/config/appref#max_concurrent_requests


On Fri, Nov 5, 2021 at 1:48 PM Joshua Smith 
wrote:

> I figured out from the logs (since it prints a message for each worker it
> starts) that 3.7 is starting 8 workers by default. So adding the entrypoint
> with defaults has no effect.
>
> However, I did some experiments and I think I can characterize the
> difference between 2.7 and 3.7 when it comes to scaling.
>
> When first landing on my app's page it shows a list of thumbnails. These
> come from the datastore, with each thumbnail making a HTTP request (GET
> /image?which=1, GET /image?which=2, etc.). If we:
>
> 1. Query for all of these at once so the server gets hammered with
> simultaneous requests:
> - 2.7 starts one extra instance, and while that's happening the images are
> all served by the instance that already existed; the new instance never
> actually serves any of those image requests.
> - 3.7 starts a lot of instances; it dumps a bunch of requests to each of
> them (one per worker thread, I presume), all of which are going to take 5
> seconds because that's how long it takes each instance to start.
>
> The net result is the user of the 2.7 app sees all the images load right
> away, while the 3.7 app gets them over the course of 15 seconds or so, and
> sometimes there are timeout errors.
>
> 2. Query for these with a delay that's long enough to avoid overlapping
> requests:
> - Neither starts extra instances.
>
> The user sees all the images load pretty quick, but not quite as quick as
> 2.7 with the first approach.
>
> My conclusion is that the change is that the old scheduler wouldn't use an
> instance until it was running, whereas the new scheduler queues up requests
> for instances that are still starting up. The old scheduler doesn't start a
> new instance until the one it started most recently is running, whereas the
> new scheduler starts a new instance every time the request queue fills up
> for existing instances.
>
> If you can serve a burst of requests in the time it take an instance to
> start, the old scheduler will get all those served. But the new scheduler
> will choke and perform very badly.
>
> I'm not saying that the old scheduler was better. It just handled this
> particular case (cold start with a firehose) a lot better. Once the two
> systems are running at scale, I wouldn't expect there to be much difference.
>
> -Joshua
>
>
> On Nov 5, 2021, at 3:29 PM, Joshua Smith  wrote:
>
> I think it's telling me that since my P3.7 app has a F4_1G instance, I
> should be configuring it to run gunicorn with 8 workers. I don't see any
> information about the default number of workers. I guess I can do an
> experiment to see what explicitly setting that does...
>
> On Nov 5, 2021, at 3:17 PM, Jason Collins 
> wrote:
>
> It sounds like there might be some differences due to concurrency between
> 2.7 and 3.7/8/9. There are some notes on how to tweak the number of worker
> processes started for Python 3 here:
> https://cloud.google.com/appengine/docs/standard/python3/runtime#entrypoint_best_practices
>
>
> On Friday, 5 November 2021 at 11:50:52 UTC-7 George (Cloud Platform
> Support) wrote:
>
>> There is no official description, somewhere, of "how the scaling has
>> changed" or "how it has been implemented differently", as there is no
>> change in scaling behavior, and no different implementation. This is an
>> assumption while we investigate the situation described by Joshua above. I
>> could not find a similar issue in the Public Issue Tracker
>> <https://issuetracker.google.com/>, so the difference in scaling between
>> Python 2 and 3 is likely a one-time occurrence, and caused by idiosyncratic
>> settings in the project, or application-specific factors. Such issue are
>> difficult to approach in a publicly-accessible thread as this one, as
>> investigation needs particulars such as project ID, and access to other
>> private data. To have this issue properly addressed, it's better to open
>> a support case
>> <https://cloud.google.com/support/docs/manage-cases#creating_cases> or
>> an issue in the Public Issue Tracker <https://issuetracker.google.com/>.
>>
>> On Friday, 05 November 2021 at 05:50:42 UTC-4 Nicolas Fonrose (Teevity)
>> wrote:
>>
>>> Hello David,
>>>
>>> >There’s not an official way of

Re: [google-appengine] Updates from the Google App Engine team (Fall 2021)

[Jason Collins]

We'll work to open source the launcher source. In the meantime, the default 
logic looks like this:

workerCount := 8
if memoryMb <= 128 {
workerCount = 1
} else if memoryMb <= 256 {
workerCount = 2
} else if memoryMb <= 512 {
workerCount = 4
}

The number of workers to use is highly dependent on your workload, so these 
are only reasonable general starting points.


On Friday, 5 November 2021 at 12:29:43 UTC-7 Joshua Smith wrote:

> I think it's telling me that since my P3.7 app has a F4_1G instance, I 
> should be configuring it to run gunicorn with 8 workers. I don't see any 
> information about the default number of workers. I guess I can do an 
> experiment to see what explicitly setting that does...
>
> On Nov 5, 2021, at 3:17 PM, Jason Collins  wrote:
>
> It sounds like there might be some differences due to concurrency between 
> 2.7 and 3.7/8/9. There are some notes on how to tweak the number of worker 
> processes started for Python 3 here: 
> https://cloud.google.com/appengine/docs/standard/python3/runtime#entrypoint_best_practices
>
>
> On Friday, 5 November 2021 at 11:50:52 UTC-7 George (Cloud Platform 
> Support) wrote:
>
>> There is no official description, somewhere, of "how the scaling has 
>> changed" or "how it has been implemented differently", as there is no 
>> change in scaling behavior, and no different implementation. This is an 
>> assumption while we investigate the situation described by Joshua above. I 
>> could not find a similar issue in the Public Issue Tracker 
>> <https://issuetracker.google.com/>, so the difference in scaling between 
>> Python 2 and 3 is likely a one-time occurrence, and caused by idiosyncratic 
>> settings in the project, or application-specific factors. Such issue are 
>> difficult to approach in a publicly-accessible thread as this one, as 
>> investigation needs particulars such as project ID, and access to other 
>> private data. To have this issue properly addressed, it's better to open 
>> a support case 
>> <https://cloud.google.com/support/docs/manage-cases#creating_cases> or 
>> an issue in the Public Issue Tracker <https://issuetracker.google.com/>. 
>>
>> On Friday, 05 November 2021 at 05:50:42 UTC-4 Nicolas Fonrose (Teevity) 
>> wrote:
>>
>>> Hello David,
>>>
>>> >There’s not an official way of forcing the scaling to behave like it 
>>> used to be on python 2.7.
>>> Is there an official description, somewhere, of "how the scaling has 
>>> changed" or "how it has been implemented differently" ?
>>> Or this change just a side effect of technical choices that were made in 
>>> the new runtimes?
>>>
>>> Thanks,
>>>
>>> On Thursday, November 4, 2021 at 9:56:43 PM UTC+1 David (Cloud Platform 
>>> Support) wrote:
>>>
>>>> Hello,
>>>>
>>>> Other than applying changes within your own application so it will go 
>>>> easy at startup, you can try using warmup requests 
>>>> <https://cloud.google.com/appengine/docs/standard/python3/configuring-warmup-requests>
>>>>  
>>>> along with tweaking the min_idle_instances element 
>>>> <https://cloud.google.com/appengine/docs/standard/python3/config/appref#scaling_elements>
>>>>  
>>>> in your app.yaml, in order to reduce request and response latency during 
>>>> the time when your app's code is being loaded to a newly created instance.
>>>> There’s not an official way of forcing the scaling to behave like it 
>>>> used to be on python 2.7. However, this type of feedback can be passed to 
>>>> the App Engine engineering team in the form of a feature request. You can 
>>>> create such a feature request here 
>>>> <https://issuetracker.google.com/issues/new?component=187191=1162953>.
>>>>  
>>>> The App Engine engineering team would then evaluate it and decide whether 
>>>> it could be implemented or not.
>>>> On Thursday, November 4, 2021 at 11:38:52 AM UTC-4 Joshua Smith wrote:
>>>>
>>>>> Thanks for sending out this update. I did, indeed, miss most of this 
>>>>> news.
>>>>>
>>>>> This item, in particular, is *awesome*:
>>>>>
>>>>>
>>>>>
>>>>>- 
>>>>>
>>>>>Extending support for App Engine bundled services 
>>>>>
>>>>> <https://cloud.google.com/blog/products/serverless/support-for-app-engine-services-in-

Re: [google-appengine] Updates from the Google App Engine team (Fall 2021)

[Jason Collins]

It sounds like there might be some differences due to concurrency between 
2.7 and 3.7/8/9. There are some notes on how to tweak the number of worker 
processes started for Python 3 
here: 
https://cloud.google.com/appengine/docs/standard/python3/runtime#entrypoint_best_practices


On Friday, 5 November 2021 at 11:50:52 UTC-7 George (Cloud Platform 
Support) wrote:

> There is no official description, somewhere, of "how the scaling has 
> changed" or "how it has been implemented differently", as there is no 
> change in scaling behavior, and no different implementation. This is an 
> assumption while we investigate the situation described by Joshua above. I 
> could not find a similar issue in the Public Issue Tracker 
> , so the difference in scaling between 
> Python 2 and 3 is likely a one-time occurrence, and caused by idiosyncratic 
> settings in the project, or application-specific factors. Such issue are 
> difficult to approach in a publicly-accessible thread as this one, as 
> investigation needs particulars such as project ID, and access to other 
> private data. To have this issue properly addressed, it's better to open 
> a support case 
>  or an 
> issue in the Public Issue Tracker . 
>
> On Friday, 05 November 2021 at 05:50:42 UTC-4 Nicolas Fonrose (Teevity) 
> wrote:
>
>> Hello David,
>>
>> >There’s not an official way of forcing the scaling to behave like it 
>> used to be on python 2.7.
>> Is there an official description, somewhere, of "how the scaling has 
>> changed" or "how it has been implemented differently" ?
>> Or this change just a side effect of technical choices that were made in 
>> the new runtimes?
>>
>> Thanks,
>>
>> On Thursday, November 4, 2021 at 9:56:43 PM UTC+1 David (Cloud Platform 
>> Support) wrote:
>>
>>> Hello,
>>>
>>> Other than applying changes within your own application so it will go 
>>> easy at startup, you can try using warmup requests 
>>> 
>>>  
>>> along with tweaking the min_idle_instances element 
>>> 
>>>  
>>> in your app.yaml, in order to reduce request and response latency during 
>>> the time when your app's code is being loaded to a newly created instance.
>>> There’s not an official way of forcing the scaling to behave like it 
>>> used to be on python 2.7. However, this type of feedback can be passed to 
>>> the App Engine engineering team in the form of a feature request. You can 
>>> create such a feature request here 
>>> .
>>>  
>>> The App Engine engineering team would then evaluate it and decide whether 
>>> it could be implemented or not.
>>> On Thursday, November 4, 2021 at 11:38:52 AM UTC-4 Joshua Smith wrote:
>>>
 Thanks for sending out this update. I did, indeed, miss most of this 
 news.

 This item, in particular, is *awesome*:



- 

Extending support for App Engine bundled services 

 
  
(Sep 2021) 


 This sounds like it will make it so much easier to migrate to Python 
 3.7.

 One thing I've noticed is that my older apps on 2.7 seem to handle peak 
 scaling a lot better than my newer apps on 3.7. For example, if I have a 
 web page that hits a 2.7 app with 100 REST calls at startup (bad design, 
 but it happens), the old app serves them all eventually. But if I do the 
 same thing in a 3.7 app, it's likely to choke and fail a bunch of those 
 requests with these:


 The specific pattern is that after the first couple requests, it spins 
 up a new instance. That takes 5 seconds to serve its first request (simple 
 app, so I guess that's just GAE overhead). Then it spins up a couple more. 
 Then I start getting those errors on some of the requests, because 15s 
 have 
 passed. In this capture the blue ones are new instances spinning up, and 
 the orange ones are timeout errors (note the 15s time on those).


 I've been designing around the issue by making sure my web apps go easy 
 on the server on startup. But it is yet another concern about migrating.

 Is there a way to tune the auto-scaling for 3.7 to behave like the 2.7? 
 Here's a similar set of requests to my 2.7 app, which only spins up one 
 extra instance, and never throws timeout errors, ever:


 -Joshua




-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from 

[google-appengine] Re: Parsing date-time values from JSON fails in standard environment

[Jason Collins]

App Engine uses the standard compiler for Go runtimes.

Can you send a snippet of code that reproduces what you are seeing?

On Monday, 16 November 2020 at 09:10:26 UTC-8 bari...@sprinthubmobile.com 
wrote:

>
> Hi everyone. I am unable to parse time values from JSON when I deploy to 
> app engine standard environment. I'm using the Go 1.14 runtime. The time 
> values are sent in ISO-8601 (or RFC 3339) format. It works in every other 
> environment except on app engine.
> Does the app engine standard environment use a customized Go compiler? I 
> suspect that they replace the encoding/json package for optimization 
> purposes, but their version has a bug.
> I've tested my program on both go113 and go114. If I send a time value 
> like "2018-09-22T12:42:31Z" in JSON, I get the following error.
>
> "invalid character 'T' looking for beginning of value"
>
> If I remove the "T" as in "2018-09-22 12:42:31Z", I get the following 
> error.
>
> "parsing time \"\"2021-01-01 00:00:00.000Z\"\" as 
> \"\"2006-01-02T15:04:05Z07:00\"\": cannot parse \" 00:00:00.000Z\"\" as 
> \"T\""
>
> Clearly it expects the "T", but it chokes on it. Sending only the date 
> part doesn't work either.
> I am certain that the framework I'm using (Buffalo) does not bring its own 
> JSON parser. It uses the standard library, as I can tell from this 
> snippet (copied below for your perusal) 
> 
> :
>
> // BinderFunc returns the Binder for this JSONRequestTypeBinder 
> func (js JSONContentTypeBinder) BinderFunc() Binder { 
>  return func(req *http.Request, value interface{}) error { 
>  return json.NewDecoder(req.Body).Decode(value) 
> } 
> } 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/ac7b843e-6bc3-43dc-a41c-16426005ed3an%40googlegroups.com.

[google-appengine] Re: internal.flushLog: Flush RPC: service bridge HTTP failed ... i/o timeout -> app engine standard environment, go runtime 112 & 113 issue

[Jason Collins]

The appengine package no longer functions in Go 1.12 and newer. Likely that 
package is attempting to make some sort of connection and blocking until 
timeout. Remove the "appengine.Main()" line.


On Sunday, 24 May 2020 11:11:14 UTC-7, Gökhan Akca wrote:
>
>
> When I deploy my go application in GAE standard enviroment using runtime: 
> go112 or go113 loading an html page of my very simple application in my 
> browser takes 10 seconds.
> However, configuring the runtime to go111 in app.yaml leads to immediate 
> page load without any lag or timeout log.
>
> The log contains this entry
>
> {
>   "textPayload": "2020/05/23 20:34:43 internal.flushLog: Flush RPC: 
> service bridge HTTP failed: Post 
> http://appengine.googleapis.internal:10001/rpc_http: dial tcp 
> 169.254.169.253:10001: i/o timeout",
>   "insertId": "5ec988e3000d98cc6b5b9b5e",
>   "resource": {
> "type": "gae_app",
> "labels": {
>   "zone": "europe-west6-3",
>   "version_id": "20200523t223037",
>   "module_id": "default",
>   "project_id": "xxx"
> }
>   },
>   "timestamp": "2020-05-23T20:34:43.891084Z",
>   "labels": {
> "clone_id": 
> "XX117c75b94ee99fb3c024c136a5fce28d7c2a813754222997c13b09XX"
>   },
>   "logName": "projects/xxx/logs/stderr",
>   "receiveTimestamp": "2020-05-23T20:34:43.893803794Z"
> }
>
>
> Since I am learning go, my code is extremely simple:
>
> func main() {
> http.HandleFunc("/home", homepageHandler)
> appengine.Main()
> }
>
> func homepageHandler(w http.ResponseWriter, r *http.Request) {
> fmt.Fprint(w, "home")
> }
>
> Even opening the root URL leading to HTTP 404 takes 10 seconds. 
> Hence, my code seems to be irrelevant.
>
>
> Is there a breaking change using go 1.12+ in GAE standard environment?
>
> Sticking to go version 1.11 is a workaround for now, but I am worried that 
> once it becomes deprecated, I won't be able to run my application in GAE 
> without the timeouts when upgrading the go version.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/2fcc9839-9be7-4761-8ee0-b5cc68298d3c%40googlegroups.com.

[google-appengine] Re: Google Cloud Run Container Status

[Jason Collins]

I believe you can just delay listening on $PORT until you're ready to 
receive traffic.


On Tuesday, 5 May 2020 05:01:33 UTC-7, Rodrigo Silveira wrote:
>
> I realize this is not the Cloud Run group, but that's the closest to it.
>
> I've asked my question on Stackoverflow, but the solution(s) posted there 
> were not helpful.
>
> *My issue*: I have a Flask (Python 3.6) REST API that I'm deploying on 
> Cloud Run. The first thing my server does at startup is to load two large 
> Pandas dataframes into memory. Until those two global objects are loaded, 
> the API is not useful. I have a health check defined in my Dockerfile that 
> helps with this - until the two dataframes are loaded, the health check is 
> not healthy. The problem is that when I deploy a new revision on Cloud Run, 
> it will start delivering traffic to my container before it's actually fully 
> loaded, and thus, the first several requests to the API will take 10-15 
> seconds.
>
> I've optimized this startup step about as much as it can before things 
> become completely ridiculous and unreasonable, and yet, this is the best I 
> can do.
>
> *My question*: is there a way to help Cloud Run know that the container 
> is still loading and shouldn't be taking traffic yet? Something along the 
> lines of: load the container, but don't add it to the load balancer yet, 
> since the status is not yet "healthy"?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/0f382d26-abb4-499f-985e-a71336c49db3%40googlegroups.com.

[google-appengine] Re: Correlating Logs with Requests on Java 11 AppEngine

[Jason Collins]

"If that behavior is desired, manually set the highest severity in the 
top-level entry."

You're correct, this isn't possible. I'll have the docs updated.


"correlated logs don't seems to show up on the first request of the 
instance"

This is a known issue that we are working on fixing (it's due to the 
"embedded" log with the cold start notification; this defeats the 
correlation logic).


On Friday, 28 February 2020 05:31:06 UTC-8, Paolo Conte wrote:
>
> Hi Victor, 
> what I'm asking is not really a how-to, but pointing out 1. what seems to 
> be an error in the documentation, but not clear, and 2. what seems to be a 
> bug in the the log viewer.
>
> Thanks
>
> On Friday, 28 February 2020 13:57:22 UTC+1, Victor (Cloud Support) wrote:
>>
>> Hi there,
>>
>> I can see this thread is more of a technical/how-to question. 
>>
>>
>> Since it is not getting much attention here, I suggest you try to post it 
>> on Stackoverflow , as these questions 
>> usually have greater visibility there.
>>
>> If you decide to post it there, please remember to tag the question with 
>> at least the following tags:
>>
>> *google-app-engine*
>>
>> *stackdriver*
>>
>>
>> Regards,
>>
>> On Wednesday, February 26, 2020 at 6:42:36 PM UTC+1, Paolo Conte wrote:
>>>
>>> Hello,
>>> trying to correlate logs with requests by following the documentation at 
>>> https://cloud.google.com/appengine/docs/standard/java11/writing-application-logs
>>> Here it says: *The highest severity from the "child" log entries does 
>>> not automatically apply to the top-level entry. If that behavior is 
>>> desired, manually set the highest severity in the top-level entry.*
>>> But how to "manually set the highest severity in the top-level entry" 
>>> it they are automatically created? Is this even possible? 
>>>
>>> Also, correlated logs don't seems to show up on the first request of the 
>>> instance, on the web console. Is it a bug of the console?
>>>
>>> Thanks
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/3969f6fe-918d-46aa-a995-da786cd1f7fb%40googlegroups.com.

[google-appengine] Re: Python 3: how to determine the current app's project id?

[Jason Collins]

Ryan, in runtime:python37 there is also a metadata server like the GCE 
metadata server. Unfortunately, you'll have to find the docs for that over 
in the Java8 docs... 
https://cloud.google.com/appengine/docs/standard/java/accessing-instance-metadata

On Wednesday, 6 November 2019 14:10:24 UTC-8, Ryan B wrote:
>
> ah, found it. looks like it's GAE_APPLICATION: 
> https://cloud.google.com/appengine/docs/standard/python3/runtime#environment_variables
>
> might be worth adding to the differences page 
> 
> !
>
> On Wednesday, November 6, 2019 at 7:11:14 AM UTC-8, Ryan B wrote:
>>
>> Hi all! In the Python 2 runtime, I can use the app_identity API 
>>  
>> to determine the current app's project id (ie application id) with 
>> app_identity.get_application_id(). Anyone know how to do this in the 
>> Python 3 runtime? 
>> https://cloud.google.com/appengine/docs/standard/python3/python-differences#app_engine_apis
>>  
>> doesn't mention the app_identity API, and 
>> https://cloud.google.com/appengine/docs/standard/python3/python-differences#modules
>>  
>> doesn't mention project id.
>>
>> Thanks in advance!
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/3f07ef0b-d19c-46f9-ae51-67d982982cc3%40googlegroups.com.

[google-appengine] Re: Active PHP 7.2 sessions lost after gcloud app deploy

[Jason Collins]

In GAE Std, for php72, the default session store is configured. This 
default is `session.save_handler=files` which uses /tmp to store session 
files.

In GAE Std, each instance has its own /tmp and the /tmp is empty on any 
cold start. So this is not really a viable option for session storage 
(e.g., different requests from the same user may land on different 
instances, and thus different sessions).

You can use php.ini to configure an alternative (e.g., MySQL on Cloud SQL, 
or Cloud Memorystore), but you need to do some set up work (e.g., create a 
Cloud SQL instance, configure access).

We need to have better documentation about this; I will create an internal 
bug to have some created.



On Wednesday, 21 August 2019 02:10:53 UTC-7, Andy Whittle wrote:
>
> Hi Jason,
>
> We're using whichever method GAE Standard env provides to store sessions.
>
> On Tuesday, August 20, 2019 at 2:40:24 AM UTC+1, Jason Collins wrote:
>>
>> How are you storing your sessions? Is it the /tmp folder default, or have 
>> you used something like MySQL (Cloud SQL) to store sessions?
>>
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/f4a3b221-d955-472d-8d8c-c3039aeba445%40googlegroups.com.

[google-appengine] Re: Active PHP 7.2 sessions lost after gcloud app deploy

[Jason Collins]

How are you storing your sessions? Is it the /tmp folder default, or have 
you used something like MySQL (Cloud SQL) to store sessions?


On Monday, 19 August 2019 09:58:38 UTC-7, Andy Whittle wrote:
>
> Thanks, George. We already increment the version deployed each time, but 
> this still occurs. 
>
> Can you tell me where I'm going wrong in the deployment process, please? 
>
> Example: 
>
> - Current version with 100% traffic is named "v123" with active users. 
> - gcloud app deploy -v v124 
> - Traffic is automatically sent to v124, at which point the users are 
> kicked out. 
>
> Is there a step I'm missing, please? Thanks

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/ec08eb63-58f3-4fd0-b1f1-af369a597aea%40googlegroups.com.

[google-appengine] Re: Run App Engine standard environment with NodeJs

[Jason Collins]

For GAE Std nodejs8/nodejs10, datastore and task queues are only available 
via Cloud Datastore [1] and Cloud Tasks [2].

There is no access to the classic GAE Std APIs in Second Generation 
runtimes [3].


[1] https://cloud.google.com/datastore/docs/concepts/overview
[2] https://cloud.google.com/tasks/docs/dual-overview
[3] https://cloud.google.com/appengine/docs/standard/runtimes


On Friday, 10 May 2019 04:51:25 UTC-7, Azeem Haider wrote:
>
> I'm going to develop nodeJS application in App Engine Standard Biggest 
> problem I'm facing it how I run app engine standard environment locally?
>
> I already worked in app engine standard locally with java development. We 
> run the app egnine like mvn appengine:run and able to use *datastore* and 
> *Task 
> queues* etc.
>
> How can I use *datastore and task queues* locally in nodeJS?
> In NodeJs Docs Samples 
> 
>  they 
> are using cloud datastore not the local one.
>
> Is it possible to use datastore and task queues locally in nodeJS like 
> JAVA? If it is possible then how I can use it and Document or tutorial?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/95502195-fce6-48f8-b5a3-037bb83fa5f5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: NodeJS AppEngine App error connecting to Google Cloud SQL

[Jason Collins]

The connection technique is different in Standard I believe.

In particular, note the "config.socketPath" in the link I sent.



>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/CAL_dBeZ2DENTq2UG0JhTHSL2G8oRFxeepD1XjzWpEo3cmz2e0A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: NodeJS AppEngine App error connecting to Google Cloud SQL

[Jason Collins]

Your connection string doesn't look right to me. Check out this example:
https://cloud.google.com/appengine/docs/standard/nodejs/using-cloud-sql#setting_connection_strings_and_adding_a_library

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/CAL_dBeaEUBGeUM2KTWC%2BMo3xkiJVD13PFZjE5UVTpfhN304reA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: NodeJS AppEngine App error connecting to Google Cloud SQL

[Jason Collins]

Can you post your repro?

On Monday, 22 April 2019 19:32:48 UTC-7, cristian...@gmail.com wrote:
>
>
> Hi,
>  I'm trying to connect a simple NodeJS App (A REST API) deployed into 
> AppEngine to a Google Cloud SQL instance.
>  Every time I deploy the app using a flex environment, it works 
> perfectly, but when I try to deploy it in a Standard environment it does 
> not work.
>  To isolate the problem I've created a repo with a simple nodeJS app 
> which proves my scenarios. 
>  Is it by design? I mean, the filter in ports blocking the connection 
> from Standard AppEngine to Google Cloud SQL instances?
>  I've tried the same code using Cloud Run and it is not working too.  
>   
>
> Regards
> Cristian
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/54fd5bf0-fef5-416f-bd21-4bcd3a4407b3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Connecting from app engine instance to compute engine instance directly

[Jason Collins]

Check out 
https://cloud.google.com/appengine/docs/standard/python/connecting-vpc


On Thursday, 18 April 2019 06:33:59 UTC-7, Chris Santos wrote:
>
> Still can't believe we're waiting on this. Are there any updates?
>
> On Thursday, May 7, 2015 at 12:52:39 PM UTC-5, Ian Childress wrote:
>>
>> We (our Go dev team) want to connect our app engine apps to our compute 
>> engine apps directly using internal IP address. From my understanding 
>> through documentation and exhaustive searching, the solution is to use a 
>> public IP address. This solution requires whitelisting an entire ip block 
>> (Google's ip block for app engine). This both increases security risk as 
>> well as charges additional bandwidth. The other solution is to use a 
>> messaging service PubSub. This is fine for submitting tasks to be performed 
>> by the back end app, but it prevents the app engine from receiving the 
>> response from the compute engine. 
>>
>> Have I missed the solution somewhere that allows for internal 
>> communication between app engine and compute engine? Using a job queue or 
>> public IP is not a replacement for an internal socket connection.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/6a044186-2f58-413d-8ca7-318b0780be3e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Can't make a new NodeJS App Deploy

[Jason Collins]

This should be fixed as of yesterday afternoon (Pacific Time). Please let 
us know if you are still seeing this issue.


On Tuesday, 22 January 2019 15:58:38 UTC-8, Paulo Freitas wrote:
>
> Hello guys,
>
> I have a nodejs service running in the AppEngine for a while. But today, 
> now, I tried to do a new deploy, `gcloud app deploy` and get a constant 
> *build 
> error* with a link to the log bellow, in itallic.
> Anyone with the same issue? Does anyone has any clue on how to solve it?
>
> att, 
>
> Paulo Freitas
>
>
> Log here
>
> *starting build "3f82f9df-292c-4956-9558-43dcbc1400fa" FETCHSOURCE BUILD 
> Starting Step #0 - "fetcher" Step #0 - "fetcher": Already have image (with 
> digest): gcr.io/cloud-builders/gcs-fetcher 
>  Step #0 - "fetcher": Fetching 
> manifest 
> gs://staging.small-talk-3972f.appspot.com/ae/1081fb4b-8d15-4cda-a21c-e7a42a8a0e3d/manifest.json
>  
> .
>  
> Step #0 - "fetcher": Processing 1785 files. Step #0 - "fetcher": 
> ** Step #0 - "fetcher": 
> Status: SUCCESS Step #0 - "fetcher": Started: 2019-01-22T18:35:38Z Step #0 
> - "fetcher": Completed: 2019-01-22T18:35:41Z Step #0 - "fetcher": Requested 
> workers: 200 Step #0 - "fetcher": Actual workers: 200 Step #0 - "fetcher": 
> Total files: 1785 Step #0 - "fetcher": Total retries: 0 Step #0 - 
> "fetcher": GCS timeouts: 0 Step #0 - "fetcher": MiB downloaded: 5.93 MiB 
> Step #0 - "fetcher": MiB/s throughput: 2.57 MiB/s Step #0 - "fetcher": Time 
> for manifest: 322.05 ms Step #0 - "fetcher": Total time: 2.65 s Step #0 - 
> "fetcher": ** Finished 
> Step #0 - "fetcher" Starting Step #1 - "builder" Step #1 - "builder": 
> Pulling image: 
> gcr.io/gae-runtimes/nodejs8_app_builder:nodejs8_20190118_8_15_0_RC00 
>  
> Step #1 - "builder": Error response from daemon: manifest for 
> gcr.io/gae-runtimes/nodejs8_app_builder:nodejs8_20190118_8_15_0_RC00 
>  
> not found Step #1 - "builder": Error response from daemon: manifest for 
> gcr.io/gae-runtimes/nodejs8_app_builder:nodejs8_20190118_8_15_0_RC00 
>  
> not found Step #1 - "builder": Error response from daemon: manifest for 
> gcr.io/gae-runtimes/nodejs8_app_builder:nodejs8_20190118_8_15_0_RC00 
>  
> not found Step #1 - "builder": Error response from daemon: manifest for 
> gcr.io/gae-runtimes/nodejs8_app_builder:nodejs8_20190118_8_15_0_RC00 
>  
> not found Step #1 - "builder": Error response from daemon: manifest for 
> gcr.io/gae-runtimes/nodejs8_app_builder:nodejs8_20190118_8_15_0_RC00 
>  
> not found Step #1 - "builder": Error response from daemon: manifest for 
> gcr.io/gae-runtimes/nodejs8_app_builder:nodejs8_20190118_8_15_0_RC00 
>  
> not found Step #1 - "builder": Error response from daemon: manifest for 
> gcr.io/gae-runtimes/nodejs8_app_builder:nodejs8_20190118_8_15_0_RC00 
>  
> not found Step #1 - "builder": Error response from daemon: manifest for 
> gcr.io/gae-runtimes/nodejs8_app_builder:nodejs8_20190118_8_15_0_RC00 
>  
> not found Step #1 - "builder": Error response from daemon: manifest for 
> gcr.io/gae-runtimes/nodejs8_app_builder:nodejs8_20190118_8_15_0_RC00 
>  
> not found Step #1 - "builder": Error response from daemon: manifest for 
> gcr.io/gae-runtimes/nodejs8_app_builder:nodejs8_20190118_8_15_0_RC00 
>  
> not found Step #1 - "builder": Error response from daemon: manifest for 
> gcr.io/gae-runtimes/nodejs8_app_builder:nodejs8_20190118_8_15_0_RC00 
>  
> not found ERROR: failed to pull because we ran out of retries. ERROR ERROR: 
> build step 1 
> "gcr.io/gae-runtimes/nodejs8_app_builder:nodejs8_20190118_8_15_0_RC00 
> "
>  
> failed: error pulling build step 1 
> "gcr.io/gae-runtimes/nodejs8_app_builder:nodejs8_20190118_8_15_0_RC00 
> ":
>  
> exit 

[google-appengine] Re: Use larger instance on free tier

[Jason Collins]

Yes, you are correct.


On Tuesday, 18 December 2018 19:07:56 UTC-8, Ng Jun Yen wrote:
>
> According to the pricing page 
> , 
> larger app engine instances such as F2 are billed as 2 F1 instances. Does 
> this mean we can use larger instances on the free tier at the expense of 
> hours available? For example, we get 28 hours of free F1 instances and we 
> can instead opt for a larger F2 instance for 14 hours. Am I understanding 
> this correctly? Thanks!
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/5629c41e-3ebf-485e-a0a0-a40ebbeeceab%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Google Cloud App Engine Standard not support gzip?

[Jason Collins]

Follow https://issuetracker.google.com/issues/117814186 for resolution.

On Thursday, 15 November 2018 16:51:06 UTC-8, Indra Sumawi wrote:
>
> Hi Jason,
>
> I see. Thank you for your answer :)
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/e3b708ca-09ba-4de6-8d68-2206060a56d9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Node 8 Standard (Express) not gzipping content

[Jason Collins]

Follow https://issuetracker.google.com/issues/117814186 for resolution.

On Friday, 16 November 2018 08:50:10 UTC-8, Steve Geer wrote:
>
> Thank you for the update, Jason.  I'll keep an eye on it.
>
> On Monday, October 29, 2018 at 2:03:47 PM UTC-7, Jason Collins wrote:
>>
>> Today, we are not passing through the `Accept-Encoding` header, so it is 
>> not possible for your middleware to decide that it should compress. 
>>
>> We will roll out a fix for this in the new few weeks.
>>
>>
>> On Monday, 29 October 2018 06:12:50 UTC-7, Steve Geer wrote:
>>>
>>> I have run into an issue where app engine standard is not gzipping 
>>> responses.  It's a basic express universal application based on the docs at 
>>> angular.io <https://angular.io/guide/universal>, with the addition of 
>>> the compression middleware recommended by the express documentation.
>>>
>>> It's a very simple Angular 7 universal project, which can been seen here 
>>> <https://ng-universal-test-220902.appspot.com/>.
>>>
>>> If you open the network tab you'll notice none of the assets are gzipped.
>>>
>>> Github <https://github.com/itsthesteve/ng-universal-app-engine>
>>> Stackoverflow 
>>> <https://stackoverflow.com/questions/52752100/deployed-app-engine-isnt-gzipping-my-content>
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/ff75509d-b491-4fb1-9b6a-97dbb8a5c9c4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Google Cloud App Engine Standard not support gzip?

[Jason Collins]

Okay, thx. 

For the "second gen runtimes" (php72, nodejs8, python37, go111) we are 
currently not sending the Accept-Encoding header to your app, and we are 
suppressing the Content-Encoding header on the response. This will change 
in the next 2-3 weeks and both headers will function as expected. We also 
intend to add an nginx webserver to php72 (replacing the existing one) that 
will automatically handle the compression on your behalf.


On Thursday, 15 November 2018 09:34:15 UTC-8, Indra Sumawi wrote:
>
> Hi Jason, 
>
> During the test, I'm using runtime php72 for standard and runtime php for 
> flexible

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/62fa6ef3-0e21-4897-a207-54364441ee1e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Google Cloud App Engine Standard not support gzip?

[Jason Collins]

What runtime are you using?

On Wednesday, 14 November 2018 10:22:27 UTC-8, Indra Sumawi wrote:
>
> Currently I develop REST API using laravel framework and I deploy it to 
> both Google Cloud App Engine Standard and Flexible version.
>
> When I deploy it to flexible version, it works fine. However, since my app 
> still prototype, I want to change it to standard version so I can get the 
> free tier because my app still use very little resources. 
>
> However, when I try to deploy to App Engine standard, everything still 
> works, but the response quite slow, compared to flexible. I already tried 
> to change the instance class to the highest spec machine, but still the 
> same result.
>
> So I try to find out what's going on, I thought App engine standard slower 
> than flexible.
>
> My findings:
>
> App Engine Standard
>
> [image: MdRqA.png]
>
> [image: nx9T3.png]
>
>
>
> App Engine Flexible 
>
> [image: JGBau.png]
>
> [image: lgEpT.png]
>
>
>
>
> As you can see, the standard and flexible request to the same API 
> endpoint, however, in standard version, the response return not in gzip 
> encoding and more bytes were return, while flexible version return in gzip 
> encoding.
>
> I think that might be the problem why every request become slow. In 
> Flexible, mostly the request return under 1000 ms, while in standard most 
> of the time around 2000 ms.
>
> Do you guys have any idea how to enable gzip compression in App Engine 
> Standard, or do you think there is other problem that cause the response to 
> be slow.
>
> Thank You for your help :)
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/97962a5d-5e88-410c-a9dd-ecf103e26552%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Node 8 Standard (Express) not gzipping content

[Jason Collins]

Today, we are not passing through the `Accept-Encoding` header, so it is 
not possible for your middleware to decide that it should compress. 

We will roll out a fix for this in the new few weeks.


On Monday, 29 October 2018 06:12:50 UTC-7, Steve Geer wrote:
>
> I have run into an issue where app engine standard is not gzipping 
> responses.  It's a basic express universal application based on the docs at 
> angular.io , with the addition of the 
> compression middleware recommended by the express documentation.
>
> It's a very simple Angular 7 universal project, which can been seen here 
> .
>
> If you open the network tab you'll notice none of the assets are gzipped.
>
> Github 
> Stackoverflow 
> 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/fb3dcb81-4a5e-45f0-94bc-1f3db0ab625b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Static cache not expiring since yesterdays outage

[Jason Collins]

A common technique is to add a fake version number in the query string for 
your static resources, e.g., 

   http://mysite.com/static/something.css?v=123

You'll need to add something in your templating engine to add the fake 
version number when emitting a link to a static resource. Then, just 
increment your fake v number when you want an "expiration".


On Sunday, 14 October 2018 07:31:31 UTC-7, Sam Edge wrote:
>
> Follow up:
>
> Wasn't anything to do with the outage - just a coincidence. Found this in 
> the docs:
>
> The expiration time will be sent in the Cache-Control and Expires HTTP 
> response headers, and therefore, the files are likely to be cached by the 
> user's browser, as well as by intermediate caching proxy servers such as 
> Internet Service Providers. After a file is transmitted with a given 
> expiration time, there is generally *no way* to clear it out of 
> intermediate caches, even if the user clears their own browser cache. 
> Re-deploying a new version of the app will *not* reset any caches. 
> Therefore, if you ever plan to modify a static file, it should have a short 
> (less than one hour) expiration time. In most cases, the default 10-minute 
> expiration time is appropriate.
>
> So I accidentally set some static files I wanted to redeploy with a long 
> expiration -- not sure to expire until November 10th.
>
> I was planning to release my project this week but this is a blocker. 
> Anyone been in this spot before?
>
> On Saturday, October 13, 2018 at 2:47:59 PM UTC+10:30, Sam Edge wrote:
>>
>> Since yesterdays outage (
>> https://status.cloud.google.com/incident/cloud-networking/18016), the 
>> static files on my AppEngine Standard PHP55 deploys are stale. 
>>
>> For instance:
>>
>>
>> runtime: php55
>> api_version: 1
>>
>>
>> default_expiration: "28d"
>>
>>
>> handlers:
>> - url: /build
>> static_dir: build
>>
>>
>> Usually when this sort of thing happen you can deploy a new version of 
>> your service and it busts any cache issue but I must have tried a dozen 
>> times and nothing happening. And yes I've definitely been cleaning my local 
>> caches ;)
>>
>> I also changed the bucket used when deploying for a clean upload but no 
>> difference.
>>
>> *gcloud app deploy --version next --bucket gs://test-bucket-empty*
>>
>> Overnight my application started serving the right files again but when I 
>> deployed it reverted back to the stale cache it was serving yesterday.
>>
>> I note that in the outage that Google Storage was affected, so I assume 
>> they are still mitigating the issue. 
>>
>> Just putting this out there - is anyone else experiencing this or am I 
>> going mad?
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/dc0c1d95-2841-427f-8d4a-658fedb260aa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: [Error: Server Error] GAE Nodejs8 Standard. Upgrade to Nuxtjs@2.0.0 and deployment fails (could be unrelated)

[Jason Collins]

For Nuxt in the past, I had to set the HOST env variable in app.yaml:

env_variables:
  HOST: 0.0.0.0

Not sure if that would help in this case or not.

On Thursday, 27 September 2018 13:11:46 UTC-7, Steren Giannini wrote:
>
> Hi David,
>
> Error Reporting is supported by the Node.js App Engine standard runtime, 
> we will fix our docs.
> However, Error Reporting does not catch all errors (it only catches 
> exceptions), you should be also looking at your logs.
>
> Can you look at your application output in your logs? (Open Stackdriver 
> Logging from the left nav) 
>
> Can you share here your package.json? I'd like to help you understand what 
> could fix the issue
>
> We have many customers using Nuxtjs on App Engine standard, I personally 
> made sure I was able to deploy a Nuxtjs app in the past
>
> Steren
> Product Manager
> App Engine
>
>
>
> On Thursday, September 27, 2018 at 10:33:56 AM UTC-7, David Davis wrote:
>>
>> [image: Screen Shot 2018-09-26 at 3.35.02 PM.png]
>> I've been building and deploying on Nodejs8 standard and everything was 
>> ok untill today when I upgraded from Nuxtjs 1.4.x to 2.0. Everything works 
>> fine locally but upon deploying am getting this 500 error. According to 
>> these docs 
>> https://cloud.google.com/error-reporting/docs/setup/app-engine-standard-environment
>>  
>> the nodejs8 environement doesn't allow error reporting. I see this in the 
>> console. Honestly confused how Im supposed to debug and proceed without any 
>> sort of error reporting. Please advise if anyone knows what to do.
>>
>> Project ID
>> agency-215614
>> Project number
>> 885488244632
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/e5346396-d44b-46b7-84cc-4cbdb64b7574%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: memory usage of python37 runtime on standard environment

[Jason Collins]

We are rolling out some changes that should clear up this issue. Can you 
try your test again on Wednesday next week to see if the results make more 
sense?



On Thursday, 2 August 2018 04:52:12 UTC-7, Giacomo Graziosi wrote:
>
> Hi,
> I'm doing some tests on the app engine standard enviroment using the 
> python37 runtime (which seems to be available and undocumented).
> This is the relevant part of my app.yaml:
>
> runtime: python37
> env: standard
> instance_class: F2
> entrypoint: gunicorn main:app --workers=1 --bind :$PORT
> #entrypoint: uwsgi --module hubiter_backend.wsgi.production --workers=1 
> --http-socket :$PORT --enable-threads
> default_expiration: "1d"
>
> automatic_scaling:
>   max_instances: 1
>   min_instances: 1
>
> env_variables:
>   PYTHONUNBUFFERED: 1
>
> On my local machine this is using less than 128MB of memory.
>
>   PID USER  PRI  NI  VIRT   RES   SHR S CPU% MEM%TIME+  Command
> 
> 
> 
> 
>   
> 16345 esistgut   20   0  220M 94576 16192 S  0.0  0.6  0:01.03 gunicorn 
> --workers 1 --bind :8000 main:app  
> 
> 
>
> 16342 esistgut   20   0 49804 22484  8412 S  0.0  0.1  0:00.23 gunicorn 
> --workers 1 --bind :8000 main:app
>
> But if I run it on GAE this happens:
>
> 2018-08-02 12:50:58.762 CESTExceeded soft private memory limit of 256 MB 
> with 256 MB after servicing 2 requests total. Consider setting a larger 
> instance class in app.yaml.
>
> The same app is using at least 2x memory.
>
> Using uwsgi on my local env:
>   PID USER  PRI  NI  VIRT   RES   SHR S CPU% MEM%   TIME+  Command
> 
>   
> 18743 esistgut   20   0  217M 85472 18316 S  0.0  0.5  0:01.04 uwsgi 
> --module hubiter_backend.wsgi.development --workers=1 --http-socket :8000 
> --enable-threads 
>
> ~85MB memory usage but on GAE the same problem happens. So I guess the 
> memory usage is up to 3x on GAE in this case.
>
> Am I doing something wrong?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/db65f07f-f212-412c-af78-78715410ac34%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Cannot deploy an example Node8 application to AppEngine standard: An internal error occurred

[Jason Collins]

This issue should be fixed now. We apologize for any inconvenience and will 
make adjustments to avoid this problem in the future.


On Tuesday, 31 July 2018 08:27:39 UTC-7, Steren Giannini wrote:
>
> Thanks for the report. 
> We can reproduce the issue when creating new projects and we are actively 
> investigating the issue.
>
> On Tue, Jul 31, 2018 at 7:44 AM Giacomo Graziosi  > wrote:
>
>> I have the same problem and other people are reporting it on GitHub 
>> https://github.com/GoogleCloudPlatform/nodejs-getting-started/issues/172
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Google App Engine" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to google-appengi...@googlegroups.com .
>> To post to this group, send email to google-a...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/google-appengine.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/google-appengine/7fce4e7b-3ac8-43b3-ad89-7d6fd5337e5b%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/b64db3e6-56a5-4eb2-9a27-890dbe5b8852%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: Node.js standard and urlfetch

[Jason Collins]

Streaming doesn't work for inbound requests (i.e., requests from a client
to your App Engine app), but should work fine for outbound requests
originating from your App Engine app to another server.


On Mon, 9 Jul 2018 at 22:11 Aleksander Efremov  wrote:

> Yes, I use `http`/`https` native libraries and I use streaming of course
> when it's possible. But as I understood from the AppEngine documentation,
> streaming doesn't work and all responses are buffered in any case. Do you
> know why streaming isn't ready yet and when it will possible?
>
>
> On Monday, July 9, 2018 at 6:49:18 PM UTC+4, Steren Giannini wrote:
>
>> Hi,
>>
>> As commented on the bug: It is not clear what problem a Node.js
>> implementation of urlfetch would solve.
>> Today, developers can use idiomatic methods to send asynchronous requests
>> to HTTP endpoints, for example by using the native `http` module. Did you
>> try it?
>>
>> Steren
>> Product Manager
>> App Engine
>>
>> On Sun, Jul 8, 2018 at 10:25 PM Aleksander Efremov 
>> wrote:
>>
> oh, yes, it's important note, thanks.
>>>
>>> On Friday, July 6, 2018 at 6:58:38 PM UTC+4, Jeff Schnitzer wrote:
>>>
 There's one big disadvantage of the URLFetch service, which is that
 it's limited to 32MB inbound and 10MB outbound. Those are pretty small
 numbers in this day and age.

 In Java8-land I've used both Apache HttpClient and OkHttp and both work
 normally.

 Jeff

 On Thu, Jul 5, 2018 at 10:06 PM Aleksander Efremov 
 wrote:

>>> Thanks, then I will star of course that feature request. And it's very
> good there isn't difference.
>
> On Thursday, July 5, 2018 at 11:58:27 PM UTC+4, Yasser Karout (Cloud
> Platform Support) wrote:
>>
>> Currently, there is no implementation of urlfetch for Node.js so you
>> would have to use the standard HTTP library
>>  or other libraries to issue HTTP
>> requests.
>>
>> I created a feature request on the issue tracker website
>>  which will get forwarded
>> to engineering. You can click on the star next to the request title on 
>> the
>> top of the page to mark that you are interested in the feature and 
>> receive
>> email updates.
>>
>> However there are no ETAs or guarantees of implementation for feature
>> requests.
>>
>> Regarding performance, all HTTP requests, whether urlfetch is used or
>> not, are issued using App Engine’s URL Fetch service
>>  for
>> efficiency and scaling purposes. Meaning performance will not differ 
>> while
>> using native libraries or urlfetch.
>>
>> One advantage that urlfetch has is that it provides you with an
>> interface to make asynchronous requests
>> 
>> .
>>
>>
>>
>> On Tuesday, July 3, 2018 at 2:01:26 AM UTC-4, Aleksander Efremov
>> wrote:
>>>
>>> All outbound requests in the "appengine" of "python" and "go" issue
>>> via "urlfetch" service. But for "node.js" it isn't available. There is
>>> improvements on using the "urlfetch" service instead of a native 
>>> libraries
>>> in "python" and "go"? And why it isn't implemented for "node.js"?
>>>
>>> --
> You received this message because you are subscribed to the Google
> Groups "Google App Engine" group.
>
 To unsubscribe from this group and stop receiving emails from it, send
> an email to google-appengi...@googlegroups.com.
> To post to this group, send email to google-a...@googlegroups.com.


> Visit this group at https://groups.google.com/group/google-appengine.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/google-appengine/39c3df33-ceec-4af6-8204-c878b3ab9630%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>
 --
>>> You received this message because you are subscribed to the Google
>>> Groups "Google App Engine" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to google-appengi...@googlegroups.com.
>>> To post to this group, send email to google-a...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/google-appengine.
>>>
>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/google-appengine/10f3f503-8b21-453f-80d4-acbe2d8e7e3b%40googlegroups.com
>>> 
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>

Re: [google-appengine] Re: Node.js standard and urlfetch

[Jason Collins]

For nodejs8, you do not need the Sockets API. In fact, it's not even
possible. Just connect to other servers like you would in a VM environment.

On Mon, Jul 9, 2018 at 12:35 PM Attila-Mihaly Balazs 
wrote:

> ^^^ This.
>
> However, Steren, please correct me if I'm wrong but I believe that to use
> the "idiomatic methods to send asynchronous requests to HTTP endpoints" one
> needs to enable "sockets":
> https://cloud.google.com/appengine/docs/standard/python/sockets/ (and
> also needs to be a paying application).
>
> Regards,
> Attila
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Google App Engine" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/google-appengine/HWSKQIiAmeg/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to
> google-appengine+unsubscr...@googlegroups.com.
> To post to this group, send email to google-appengine@googlegroups.com.
> Visit this group at https://groups.google.com/group/google-appengine.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/google-appengine/f24ddd92-d4ba-4157-bfe2-3de6e3a31949%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/CAL_dBeZwmy%3Dp-NHKF-Ez_9VZ1NtvnwTiskqAhP0%2BMkNgCLdAjA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Regarding Standard Environment For Node Js

[Jason Collins]

Hi Brian - I definitely understand the challenge you're faced with. 
Unfortunately, there is no published range of IP addresses for App Engine 
Standard outbound networking that you can use to configure your firewall. 
This is something we are working to remedy (i.e., allowing access to 
private IP addresses from App Engine Standard).


On Tuesday, 3 July 2018 02:33:24 UTC-6, BRIJQ Brian Oh wrote:
>
> Hello Jason,
>
> I have just manage to resolve the issue as indeed the firewall rules are 
> binded to the mongo port ip addresses are restricted. However that brings 
> us another set of concern is that how can we protect our databases 
> connections as we know now everything is exposed to the public now. On the 
> other hand, is that a possibility that we can find out our standard 
> environment ip addresses and bind it accordingly to the firewall rules just 
> like before?
>
> On Sunday, 1 July 2018 01:58:55 UTC+8, Jason Collins wrote:
>>
>> Hi Brian - if your question is about Node.js on App Engine Standard: 
>> currently you cannot connect to private Compute Engine IP addresses, though 
>> this is something we're working on (sorry, no timeline, I just want to let 
>> you know it's on our radar). You should be able to connect to a public IP 
>> address just fine. I've not done this with Mongo specifically, but I have 
>> with Redis running on Compute Engine.
>>
>> Perhaps your Compute Engine firewall is preventing the connection?
>>
>>
>> On Thursday, 28 June 2018 07:12:14 UTC-6, BRIJQ Brian Oh wrote:
>>>
>>> I am coming from NodeJs App Engine Flexible Environment and we do know 
>>> that the burning issue with this configuration is the price.
>>> As most of us do not really require the amount of minimum size of 
>>> resources and instances CPU power in flexible environment, I believe that 
>>> it does not really suit us.
>>>
>>> As a result, I did the switch over to standard environment as I 
>>> understand that it is still in beta version.
>>>
>>> The following are the two concerns and question I have,
>>>
>>> 1. Networking ports aren really able to be set by us? 
>>> 2. We cannot connect to our other instances for example in compute 
>>> engine via our private Ip-address used within Google Cloud Platform. (For 
>>> example connection with other databases servers)
>>>
>>> A little more information to point 2, the concern is that I have been 
>>> trying to connect to my MongoDB database that is being hosted in Compute 
>>> Engine. All the while, it have been communicating via the internal private 
>>> ip-address across GCP. That is a very good feature and we do know that.
>>> It have been working well with Flexible Environment.
>>>
>>> However, the moment that I launch it over to standard environment, it 
>>> stopped working. As a result, I tried to connect via public ip-address and 
>>> it stills fails to work.
>>>
>>> Please note that the application and same configurations to the same 
>>> mongodb server on compute engine connection have been successfully 
>>> establish using my own development machine with the same nodejs8 version 
>>> and similar environment based on the standard environment.
>>>
>>> 1. Development Machine on my Mac (public ipaddress) 
>>> 2. App Engine Flexible Environment (public and internal IP address)
>>>
>>> All have been successful.
>>>
>>> However both ways failed in standard environment.
>>>
>>> Does anyone have or faces the same issue?
>>>
>>> Please advise
>>>
>>> Thank you in advance.
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/e5d60df0-b3d1-4a3a-a095-58d4037f6a29%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: Connecting Cloud SQL from App Engine with TCP.

[Jason Collins]

Hi Jay - it sounds like you're trying to connect to Cloud SQL _from_ App 
Engine Standard.

There are two ways to do this:

1. Unix socket. 
See https://cloud.google.com/appengine/docs/standard/nodejs/using-cloud-sql
2. TCP connection. To do this, you must connect to a public IP Cloud SQL 
address from Google App Engine.

On Tuesday, 26 June 2018 13:02:12 UTC-6, Jay wrote:
>
> I can’t get the answer from stack exchange because there’s no way to do it 
> as you mentioned. Then, let me change my question.
> Why don’t you support TCP connection proxy for App Engine? Many modern SQL 
> libraries do not support UNIX connection, and it prevents us to use Cloud 
> SQL.
>
> On Mon, Jun 25, 2018 at 1:30 PM 'Jordan (Cloud Platform Support)' via 
> Google App Engine > wrote:
>
>> In-bound socket connections are not supported in App Engine (some 
>> runtimes 
>> support 
>> outbound connections with limitations). Therefore, the correct way to 
>> connect to your SQL instance from App Engine is via a UNIX socket 
>> connection from your application code to the local SQL proxy built into App 
>> Engine. This can be done easily via a supported API in the runtime language 
>> you chose as shown in the documentation 
>> 
>>  (example 
>> for Python).
>>
>> - Note: Google Groups is reserved for general product discussions and is 
>> not for technical support. If you require further technical support for 
>> connecting your App Engine application to Cloud SQL, it is recommended to 
>> post your detailed questions  
>> to Stack Exchang e 
>> using the supported Cloud tags.  
>>
>> -- 
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "Google App Engine" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/google-appengine/qY00qmXpPPo/unsubscribe
>> .
>> To unsubscribe from this group and all its topics, send an email to 
>> google-appengi...@googlegroups.com .
>> To post to this group, send email to google-a...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/google-appengine.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/google-appengine/87acab90-b9ee-4c31-a006-562780df2dcb%40googlegroups.com
>>  
>> 
>> .
>> For more options, visit https://groups.google.com/d/optout.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/66269cf8-cd83-4a51-8ac5-13a3d67c1b03%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Regarding Standard Environment For Node Js

[Jason Collins]

Hi Brian - if your question is about Node.js on App Engine Standard: 
currently you cannot connect to private Compute Engine IP addresses, though 
this is something we're working on (sorry, no timeline, I just want to let 
you know it's on our radar). You should be able to connect to a public IP 
address just fine. I've not done this with Mongo specifically, but I have 
with Redis running on Compute Engine.

Perhaps your Compute Engine firewall is preventing the connection?


On Thursday, 28 June 2018 07:12:14 UTC-6, BRIJQ Brian Oh wrote:
>
> I am coming from NodeJs App Engine Flexible Environment and we do know 
> that the burning issue with this configuration is the price.
> As most of us do not really require the amount of minimum size of 
> resources and instances CPU power in flexible environment, I believe that 
> it does not really suit us.
>
> As a result, I did the switch over to standard environment as I understand 
> that it is still in beta version.
>
> The following are the two concerns and question I have,
>
> 1. Networking ports aren really able to be set by us? 
> 2. We cannot connect to our other instances for example in compute engine 
> via our private Ip-address used within Google Cloud Platform. (For example 
> connection with other databases servers)
>
> A little more information to point 2, the concern is that I have been 
> trying to connect to my MongoDB database that is being hosted in Compute 
> Engine. All the while, it have been communicating via the internal private 
> ip-address across GCP. That is a very good feature and we do know that.
> It have been working well with Flexible Environment.
>
> However, the moment that I launch it over to standard environment, it 
> stopped working. As a result, I tried to connect via public ip-address and 
> it stills fails to work.
>
> Please note that the application and same configurations to the same 
> mongodb server on compute engine connection have been successfully 
> establish using my own development machine with the same nodejs8 version 
> and similar environment based on the standard environment.
>
> 1. Development Machine on my Mac (public ipaddress) 
> 2. App Engine Flexible Environment (public and internal IP address)
>
> All have been successful.
>
> However both ways failed in standard environment.
>
> Does anyone have or faces the same issue?
>
> Please advise
>
> Thank you in advance.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/6db330e8-f1de-4b21-b434-648049ec6e45%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: Best practicies for making GAE microservices idempotent

[Jason Collins]

Sorry, I don't have operational experience with Cloud Spanner. It didn't 
exist when Fantasm was written.


On Monday, 9 October 2017 07:32:13 UTC-7, Nilson Pontello wrote:
>
> Thanks for the clarification, It looks like I will have to keep hitting 
> datastore for my use case.
>
> BTW: Can Fantasm benefit from making lock transactions using cloud spanner 
> <https://cloud.google.com/spanner/> instead of datastore? or  is 
> datastore faster for small transactions like the ones performed by lock.py?
>
> Thanks
>
> On Monday, 9 October 2017 10:28:04 UTC-3, Jason Collins wrote:
>>
>> No. You can't make that assumption about task queue delivery, though it's 
>> very, very good. This is why Fantasm had to go the extra distance with 
>> Datastore transactions. 
>>
>> BTW, Fantasm allows you to turn off the Datastore run-once check and rely 
>> solely on task queue because in practice, task queue does a great job. Just 
>> not a perfect one. So you have to choose between the small performance hit 
>> of Datastore transaction, or very small chance of duplicate task delivery. 
>>
>> BTW I can't recall the named task deduplication ever failing when 
>> enqueuing a task. 
>> On Mon, Oct 9, 2017 at 4:48 AM Nilson Pontello <nilson@gmail.com> 
>> wrote:
>>
>>> Thanks Jason/George,
>>>
>>> My code does a job very similar to lock.py 
>>> <https://github.com/iki/fantasm/blob/master/src/fantasm/lock.py#L150> 
>>> (this is what I am trying to avoid).
>>>
>>> So it looks like the secret is around task queues. Can I assume that 
>>> enqueued tasks will be delivered *just once *(if my endpoint returns 
>>> 200 OK)? 
>>>
>>> If yes then my problem is solved. But if it behaves similar to pubsub 
>>> which guarantees  "At-Least-Once Delivery" then it will be useless for my 
>>> case.
>>>
>>> BTW: If those named tasks are hitting datastore for deduplication then 
>>> lock.py's approach is still valid and its performance will be as good as 
>>> named tasks.
>>>
>>> Thanks
>>>
>>>
>>> On Saturday, 7 October 2017 23:45:25 UTC-3, Jason Collins wrote:
>>>>
>>>> Fantasm! That's a blast from the past!
>>>>
>>>> I'm one of the original authors of that package. It does indeed help to 
>>>> make things idempotent, but at the end of the day, it relies on a 
>>>> datastore 
>>>> transaction (fronted by memcache for some performance gains): 
>>>> https://github.com/iki/fantasm/blob/master/src/fantasm/lock.py#L150
>>>>
>>>> Datastore transactions are one of the few things you can "count on" in 
>>>> a distributed system like App Engine.
>>>>
>>>> Another tool that is often helpful are named tasks, which can help 
>>>> ensure that you don't requeue an already queued task (e.g., in the event 
>>>> of 
>>>> a retry): 
>>>> https://cloud.google.com/appengine/docs/standard/python/taskqueue/push/creating-tasks#naming_a_task
>>>>
>>>> Finally, keep in mind that you can also enqueue a task transactionally 
>>>> with a Datastore transaction: 
>>>> https://cloud.google.com/appengine/docs/standard/python/ndb/transactions#python_Transactional_task_enqueuing
>>>>
>>>> With some creativity, you can get a long way with these basic building 
>>>> blocks.
>>>>  
>>>>
>>>> On Friday, 6 October 2017 16:39:14 UTC-7, George (Cloud Platform 
>>>> Support) wrote:
>>>>>
>>>>> Hello Nilson, 
>>>>>
>>>>> Among other sources, you may benefit from reading the "Implementing 
>>>>> Workflows on Google App Engine with Fantasm" article 
>>>>> <https://cloud.google.com/appengine/articles/fantasm>. Fantasm is an 
>>>>> excellent launch pad for building an understanding of how to decompose a 
>>>>> workflow into the appropriate chunk size as well as gaining a solid 
>>>>> footing 
>>>>> in how to build idempotent states.
>>>>>
>>>>> -- 
>>> You received this message because you are subscribed to a topic in the 
>>> Google Groups "Google App Engine" group.
>>> To unsubscribe from this topic, visit 
>>> https://groups.google.com/d/topic/google-appengine/p8Xcap4eaKU/unsubscribe
>>> .
>>> To unsubscribe from this group and all its topics, 

Re: [google-appengine] Re: Best practicies for making GAE microservices idempotent

[Jason Collins]

No. You can't make that assumption about task queue delivery, though it's
very, very good. This is why Fantasm had to go the extra distance with
Datastore transactions.

BTW, Fantasm allows you to turn off the Datastore run-once check and rely
solely on task queue because in practice, task queue does a great job. Just
not a perfect one. So you have to choose between the small performance hit
of Datastore transaction, or very small chance of duplicate task delivery.

BTW I can't recall the named task deduplication ever failing when enqueuing
a task.
On Mon, Oct 9, 2017 at 4:48 AM Nilson Pontello <nilson.ponte...@gmail.com>
wrote:

> Thanks Jason/George,
>
> My code does a job very similar to lock.py
> <https://github.com/iki/fantasm/blob/master/src/fantasm/lock.py#L150>
> (this is what I am trying to avoid).
>
> So it looks like the secret is around task queues. Can I assume that
> enqueued tasks will be delivered *just once *(if my endpoint returns 200
> OK)?
>
> If yes then my problem is solved. But if it behaves similar to pubsub
> which guarantees  "At-Least-Once Delivery" then it will be useless for my
> case.
>
> BTW: If those named tasks are hitting datastore for deduplication then
> lock.py's approach is still valid and its performance will be as good as
> named tasks.
>
> Thanks
>
>
> On Saturday, 7 October 2017 23:45:25 UTC-3, Jason Collins wrote:
>>
>> Fantasm! That's a blast from the past!
>>
>> I'm one of the original authors of that package. It does indeed help to
>> make things idempotent, but at the end of the day, it relies on a datastore
>> transaction (fronted by memcache for some performance gains):
>> https://github.com/iki/fantasm/blob/master/src/fantasm/lock.py#L150
>>
>> Datastore transactions are one of the few things you can "count on" in a
>> distributed system like App Engine.
>>
>> Another tool that is often helpful are named tasks, which can help ensure
>> that you don't requeue an already queued task (e.g., in the event of a
>> retry):
>> https://cloud.google.com/appengine/docs/standard/python/taskqueue/push/creating-tasks#naming_a_task
>>
>> Finally, keep in mind that you can also enqueue a task transactionally
>> with a Datastore transaction:
>> https://cloud.google.com/appengine/docs/standard/python/ndb/transactions#python_Transactional_task_enqueuing
>>
>> With some creativity, you can get a long way with these basic building
>> blocks.
>>
>>
>> On Friday, 6 October 2017 16:39:14 UTC-7, George (Cloud Platform Support)
>> wrote:
>>>
>>> Hello Nilson,
>>>
>>> Among other sources, you may benefit from reading the "Implementing
>>> Workflows on Google App Engine with Fantasm" article
>>> <https://cloud.google.com/appengine/articles/fantasm>. Fantasm is an
>>> excellent launch pad for building an understanding of how to decompose a
>>> workflow into the appropriate chunk size as well as gaining a solid footing
>>> in how to build idempotent states.
>>>
>>> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Google App Engine" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/google-appengine/p8Xcap4eaKU/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to
> google-appengine+unsubscr...@googlegroups.com.
> To post to this group, send email to google-appengine@googlegroups.com.
> Visit this group at https://groups.google.com/group/google-appengine.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/google-appengine/4ccbccfa-e90a-4ac0-a125-503ba343e2ab%40googlegroups.com
> <https://groups.google.com/d/msgid/google-appengine/4ccbccfa-e90a-4ac0-a125-503ba343e2ab%40googlegroups.com?utm_medium=email_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/CAL_dBeZE9viV%3D2%2BnP2TqRdk9n-1TBYfGWzv3ST%3D0rkkd%2BzcE_g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Best practicies for making GAE microservices idempotent

[Jason Collins]

Fantasm! That's a blast from the past!

I'm one of the original authors of that package. It does indeed help to 
make things idempotent, but at the end of the day, it relies on a datastore 
transaction (fronted by memcache for some performance 
gains): https://github.com/iki/fantasm/blob/master/src/fantasm/lock.py#L150

Datastore transactions are one of the few things you can "count on" in a 
distributed system like App Engine.

Another tool that is often helpful are named tasks, which can help ensure 
that you don't requeue an already queued task (e.g., in the event of a 
retry): 
https://cloud.google.com/appengine/docs/standard/python/taskqueue/push/creating-tasks#naming_a_task

Finally, keep in mind that you can also enqueue a task transactionally with 
a Datastore 
transaction: 
https://cloud.google.com/appengine/docs/standard/python/ndb/transactions#python_Transactional_task_enqueuing

With some creativity, you can get a long way with these basic building 
blocks.
 

On Friday, 6 October 2017 16:39:14 UTC-7, George (Cloud Platform Support) 
wrote:
>
> Hello Nilson, 
>
> Among other sources, you may benefit from reading the "Implementing 
> Workflows on Google App Engine with Fantasm" article 
> . Fantasm is an 
> excellent launch pad for building an understanding of how to decompose a 
> workflow into the appropriate chunk size as well as gaining a solid footing 
> in how to build idempotent states.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/54d1d179-72b4-4add-ba4d-289928d7d832%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: What's up with Blobstore?

[Jason Collins]

It depends on when you created your project. Projects created since late
2016 use GCS by default.

Projects created before that date used an internal storage by default where
the blobs do not appear in GCS, though you can choose to use GCS instead by
explicitly specifying a bucket.

On Mon, 29 May 2017 at 04:52 Attila-Mihaly Balazs 
wrote:

> I'm confused a little bit. Isn't Blobstore backed by GCS? Why would one
> need to "migrate"? I only use Blobstore for the "create_upload_url"
> functionality and all the post-processing of the data is done trough GCS.
>
> Attila
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Google App Engine" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/google-appengine/2puYf1l7vVs/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to
> google-appengine+unsubscr...@googlegroups.com.
> To post to this group, send email to google-appengine@googlegroups.com.
> Visit this group at https://groups.google.com/group/google-appengine.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/google-appengine/81a16c21-a62c-4763-a245-cd602fe210f6%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/CAL_dBeYyd96okbBEOK38%2Bc9MsWXnk66KANQP7JMj0UeJ6mTnDg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: What's up with Blobstore?

[Jason Collins]

This (unsupported, open source) tool might help 
too: https://github.com/GoogleCloudPlatform/appengine-blobstoremigrator-python


On Wednesday, 17 May 2017 11:51:37 UTC-7, Nicholas (Google Cloud Support) 
wrote:
>
> Here's a link to the above mentioned public issue: 
> https://issuetracker.google.com/35888043
>
> On Wednesday, May 17, 2017 at 2:37:49 PM UTC-4, George (Cloud Platform 
> Support) wrote:
>>
>> Hello Joshua, 
>>
>> There is work in progress on this issue, as one can gather from the "Add 
>> full blobstore support to bulkloader" issue 
>> . 
>>
>> Meanwhile, similar functionality can be set up on user side. As an 
>> example, the "Backup GAE Data/Files from Blob Store to Google Cloud Storage 
>> / other space" page 
>> 
>>  
>> provides useful information. 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/f246b4b2-6f27-4ac7-8ee8-9f9e925e697a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: Plans for Node.js on App Engine Standard?

[Jason Collins]

"ability to transactionally enqueue tasks" <-- probably my favourite
feature.

On Sat, 25 Mar 2017 at 11:06 Jeff Schnitzer <j...@infohazard.org> wrote:

> On Sat, Mar 25, 2017 at 8:39 AM, Jason Collins <jason.a.coll...@gmail.com>
> wrote:
>
> "Not only for some of the API's that are unique to standard"
>
> Wilfred, which APIs specifically?
>
>
> The most notable are the Task Queue API (with the ability to
> transactionally enqueue tasks) and the Search API.
>
> Jeff
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Google App Engine" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/google-appengine/68fg3RwWdFQ/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to
> google-appengine+unsubscr...@googlegroups.com.
> To post to this group, send email to google-appengine@googlegroups.com.
> Visit this group at https://groups.google.com/group/google-appengine.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/google-appengine/CADK-0ujt91bCQOL0Auf1CVMai-gZtjPfZhzxKjCohrki%3Doikdg%40mail.gmail.com
> <https://groups.google.com/d/msgid/google-appengine/CADK-0ujt91bCQOL0Auf1CVMai-gZtjPfZhzxKjCohrki%3Doikdg%40mail.gmail.com?utm_medium=email_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/CAL_dBeZoVOmTNc7kK11ZVxoCVgDNB7kYuC9eM%2BN1yHoVHxZ59w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Plans for Node.js on App Engine Standard?

[Jason Collins]

"Not only for some of the API's that are unique to standard"

Wilfred, which APIs specifically?


On Tuesday, 21 March 2017 06:10:45 UTC-7, Wilfred van der Deijl wrote:
>
> Hi,
>
> Does anyone know if there are any plans for Node.js on App Engine 
> Standard? I do see some notes (and alpha request form) for Java8 on 
> Standard, but I see no mention of Node.js.
> I am aware of the Node.js support on App Engine Flex, but we would prefer 
> to use Standard. Not only for some of the API's that are unique to 
> standard, but mainly for the scaling down to 0 instances. This makes it 
> ideal for APIs that aren't used frequently and for dev/test scenarios.
>
> Thanks
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/b1ab8f78-13cd-43e0-8564-65eb2f4b4147%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: In queries and cursors limitation

[Jason Collins]

Add a new field called, e.g, queryableStatus that is a list of strings. 
Then pre-compute your query logic so that you only need an equality filter.

E.g, when saving the entity (warning: pseudocode ahead):

  if status in ['PAID', 'PARTIALLY_PAID']:
invoice.queryableStatus.append('PAID_OR_PARTIALLY_PAID')
  # you can have all sorts of logic combinations and tags appended, if you 
need them
  invoice.save()


Then. when you query, you can use an equality:

  ofy.query(Invoice.class).filter("status =", 
['PAID_OR_PARTIALLY_PAID']).limit(100).list()


This will let the cursor work as expected.


Of course, you'll have to one-time update all your existing invoices. And 
you'll need to one-time update all your existing invoices whenever your 
QueryableStatus logic changes.



On Tuesday, 17 January 2017 20:38:53 UTC-8, Rajesh Gupta wrote:
>
> I have the following 
>
> Public class Invoice {
>   @Id Long id
>Key partyKey;
>Status status
> }
>
> enum Status {
>   PAID, PARTIAL_PAID, PENDING, DRAFT
> }
>
> I want to find all invoices with Status = PAID, PARTIAL_PAID, so I do the 
> IN query
>
> ofy.query(Invoice.class).filter("status in", [PAID, 
> PARTIAL_PAID]).limit(100).list()
>
> However, then, if I want the next 100, the cursor will not work because of 
> IN query
>
> What are the alternate solutions or schemas'.
>
> -- 
> Regards,
> Rajesh
> *www.VeersoftSolutions.com *
> *www.GainERP.com *
> *Accounting/Inventory/Orders/Sales/Purchase on Google Cloud Platform and 
> Mobile*
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/c194cda0-2fdb-4f2e-8b11-1b641cd7de82%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Can't write to local GCS file (python)

[Jason Collins]

App Engine accesses the Cloud Storage bucket using a service account, so 
likely that service account doesn't have write access to that bucket.

If you are accessing the default bucket (it appears you are, though the 
error message doesn't look like it), then the service account usually has 
access unless someone has revoked/changed that permission. You can repair 
your configuration using the Admin API repair 
method: 
https://cloud.google.com/appengine/docs/admin-api/reference/rest/v1/apps/repair 
  You can use the form at the bottom of the page to invoke it.


On Saturday, 3 December 2016 09:34:39 UTC-8, Mike Lucente wrote:
>
> I'm getting "Access Denied" trying to open a GCS file for writing. I'm 
> using these instructions: 
> https://cloud.google.com/appengine/docs/python/googlecloudstorageclient/read-write-to-cloud-storage.
>  
> Googling not helping much. Thanks in advance.
>  
> Here's my code and the error:
>
> bucket_name = os.environ.get('BUCKET_NAME', 
> app_identity.get_default_gcs_bucket_name())
> filename = '/' + bucket_name + '/' + 'myfile'
> print filename
> gcs_file = gcs.open(filename,
> 'w',
> content_type='text/plain')
>
> ForbiddenError: Expect status [201] from Google Storage. But got status 
> 403.
> Path: '/app_default_bucket/myfile'.
> Request headers: {'accept-encoding': 'gzip, *', 'content-type': 
> 'text/plain', 'x-goog-api-version': '2', 'x-goog-resumable': 'start'}.
> Response headers: {'server': 'UploadServer', 'vary': 'Origin', 
> 'content-length': '210', 'x-guploader-uploadid': 
> 'AEnB2UrEb9QG7lLMhN-Fu_kjPLbVwSuKfufgxIufy37ENZq0bK97IMaPdIuxA0ao4pTfuGAhZyi5hWzG5eqvBKEFk0nwheghJbkH4MlUSgFdKVzkFgt6rO8',
>  
> 'content-type': 'application/xml; charset=UTF-8', 'alt-svc': 'quic=":443"; 
> ma=2592000; v="36,35,34"', 'date': 'Sat, 03 Dec 2016 17:33:41 GMT'}.
> Body: " encoding='UTF-8'?>AccessDeniedAccess 
> denied.Caller does not have storage.objects.create 
> access to bucket app_default_bucket.".
> Extra info: None.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/ff26fc16-b261-49f6-b461-f3eb480a2670%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: REST DELETE API with request body

[Jason Collins]

The public issue tracker for this is 
here: https://code.google.com/p/googleappengine/issues/detail?id=601

Unfortunately, as per the RFC, you cannot supply a body with a DELETE 
request.

When I've hit this in the past, I've been lucky enough that the API 
provided also supported an equivalent POST endpoint (in my case I had to 
specify the query argument ?method=DELETE).


On Sunday, 28 August 2016 23:22:29 UTC-7, Vinod Kumar wrote:
>
> Hi,
>
> I am trying to delete multiple records through  REST DELETE API. For some 
> reason the request body is not reaching the API. I know that it is not 
> recommended to send a request body in REST DELETE. How can i achieve 
> deletion of multiple records?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/12fd1cc4-e210-4aac-a1a7-cfe91d793c12%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: microservice messaging

[Jason Collins]

You might want to check out this 
series: 
https://cloud.google.com/appengine/docs/python/microservices-on-app-engine

I would suggest Task Queue for messages between modules/services of the 
same app, and Cloud PubSub for messages between apps on different projects. 
Cloud PubSub could be used uniformly for both cases, but Task Queue has 
more features for point-to-point deliveries (e.g., countdown, finer grained 
retry semantics, better queue visibility for DevOps).



On Monday, 11 July 2016 03:49:21 UTC-7, wge vwgolf wrote:
>
> Hi
>
> I've used app engine for a little while now, and need to break up my 
> monolith into separate micro-services. 
>
> *The key decision is how the microservices communicate *- the criteria is:
>
>- It must be asynchronously, which then implies messaging. 
>- The messages ideally need to be "fire and forget". 
>- They also need to be pretty good in terms of performance, and packet 
>size, but these two are smaller considerations.
>
>
> I can think of  two ways I could do this in the GAE world
>
> 1) *Use one GAE, split it into services*, and have the services talk to 
> each other via taskQueues (which I believe are async)
> Disadvantage: Not completely separate in terms of database. So the same db 
> meta data is used for both read and write.
> i.e The services still share the same database (I believe). My requirement 
> is to have a separate write db, separate read db. Each of these have 
> different *representations* of the data. The readDB is *eventually 
> consistent* with the write db.
>
> 2) 
> *Use multiple GAE projects.* 
> This solves the above issue, but, what is the best way to get messaging 
> working between them ? I've looked everywhere but cannot find a decent 
> solution.
> a) XMPP is too high level, and it also requires a response to come back.
> b) Can I use GCM ? If so, how ?
> c) Is it possible to use RabbitMQ / ZeroMQ somehow ? if so how ?
>
>
> Would really appreciate help and advice with this.
>
> Regards
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/4b9e544d-bd30-4696-940f-52e8de9d5389%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: How do I list all of the files I've uploaded to the datastore?

[Jason Collins]

[... and now for the exciting conclusion to that last post ]


class UploadFile(blobstore_handlers.BlobstoreUploadHandler):
   def post(self):
   if self.get_uploads():
upload = self.get_uploads()[0]  # if you only have one file 
input on your form
entity = DatastoreFile(
data=upload.key(),  # this is the BlobKey
filename=upload.filename,
mimetype=upload.content_type
)
entity.put()
...

NOTE that you do not get the file bytes themselves. They are "peeled off" 
the POST request and stored into Blobstore - the key() is the reference to 
the bytes.

If you want to store bytes directly into Datastore yourself, then you don't 
want the Blobstore API at all and you need to perform the POST arg 
processing yourself.


On Monday, 16 May 2016 14:57:55 UTC-7, Jason Collins wrote:
>
> The __BlobInfo__ will be written regardless.
>
> So, given that your DatastoreFile entities are not being created, look to 
> the post() method of UploadFile.
>
> The parent class BlobstoreUploadHandler gives you a method called 
> get_uploads() - use this instead of looking at the POST args.
>
> So you have something like:
>
> class UploadFile(blobstore_handlers.BlobstoreUploadHandler):
> def post(self):
>
>
>
>
> On Monday, 16 May 2016 14:51:15 UTC-7, Peter S wrote:
>>
>> Thanks for that! Seems that it's still not displaying my list though. 
>> After some investigation I see that my files are being uploaded to the 
>> datastore as __BlobInfo__ objects, and not as my 'DatastoreFile' objects 
>> that I had expected. Any ideas why that is happening? Is there something I 
>> can do to ensure that they're uploaded a 'DatastoreFile' objects, or 
>> failing that, how would I go about listing the __BlobInfo objects which 
>> have been uploaded?
>>
>>
>> <https://lh3.googleusercontent.com/-c7w5tyjV1rw/VzpAmWCLdPI/AAAAAAg/nc76zRP7wnoSsCuE64E_OHAQENMxdarVACLcB/s1600/Capture.JPG>
>>
>>
>> On Sunday, May 15, 2016 at 4:51:42 PM UTC+1, Jason Collins wrote:
>>>
>>> I think your  and  tags are out of order (try doing a View 
>>> Source on your page to see if the  filenames are there). And you should 
>>> include a 
>>>
>>> Just reorder things a bit, something like:
>>>
>>> class MainHandler(webapp2.RequestHandler):
>>> def get(self):
>>> upload_url = blobstore.create_upload_url('/upload_file')
>>>
>>> *self.response.write("""*
>>> **
>>> **
>>> *""")*
>>>
>>> files = DatastoreFile.query().order(-DatastoreFile.timestamp)
>>> for file in files:
>>> self.response.write(' %s' % file.filename)
>>>
>>> self.response.write("""
>>> **
>>> 
>>>  Upload File: 
>>> 
>>> 
>>> """.format(upload_url))
>>>
>>>
>>> Also, look into using templates like Jinja as it will make it much 
>>> easier to write and debug stuff like this.
>>>
>>>
>>>
>>> On Friday, 13 May 2016 18:43:30 UTC-7, Peter S wrote:
>>>>
>>>>
>>>> I'm completely new to GAE and trying to figure some things out. I'm 
>>>> trying to write a small app which will allow a user to upload files, and 
>>>> display a list of those files already uploaded. I am having trouble 
>>>> getting 
>>>> my app to list the files already uploaded. 
>>>>
>>>> The code works in so far as it allows me to upload a file, but it does 
>>>> not print the list of files already uploaded. Any ideas where I'm going 
>>>> wrong?
>>>>
>>>> My Code:
>>>>
>>>> from google.appengine.ext import blobstore
>>>> from google.appengine.ext import ndb
>>>> from google.appengine.ext.webapp import blobstore_handlers
>>>> import webapp2
>>>>
>>>> class DatastoreFile(ndb.Model):
>>>> data = ndb.BlobProperty(required=True)
>>>> mimetype = ndb.StringProperty(required=True)
>>>> filename = ndb.StringProperty(required=True)
>>>> timestamp = ndb.DateTimeProperty(required=True,auto_now_add=True)
>>>>
>>>> class MainHandler(webapp2.RequestHandler):
>>>> def get(self):
>>>> upload_url = blobstore.create_upload_url('/upload_file')
>>>>
>>>> files = DatastoreFile.query().order(-Datastor

[google-appengine] Re: How do I list all of the files I've uploaded to the datastore?

[Jason Collins]

The __BlobInfo__ will be written regardless.

So, given that your DatastoreFile entities are not being created, look to 
the post() method of UploadFile.

The parent class BlobstoreUploadHandler gives you a method called 
get_uploads() - use this instead of looking at the POST args.

So you have something like:

class UploadFile(blobstore_handlers.BlobstoreUploadHandler):
def post(self):




On Monday, 16 May 2016 14:51:15 UTC-7, Peter S wrote:
>
> Thanks for that! Seems that it's still not displaying my list though. 
> After some investigation I see that my files are being uploaded to the 
> datastore as __BlobInfo__ objects, and not as my 'DatastoreFile' objects 
> that I had expected. Any ideas why that is happening? Is there something I 
> can do to ensure that they're uploaded a 'DatastoreFile' objects, or 
> failing that, how would I go about listing the __BlobInfo objects which 
> have been uploaded?
>
>
> <https://lh3.googleusercontent.com/-c7w5tyjV1rw/VzpAmWCLdPI/AAg/nc76zRP7wnoSsCuE64E_OHAQENMxdarVACLcB/s1600/Capture.JPG>
>
>
> On Sunday, May 15, 2016 at 4:51:42 PM UTC+1, Jason Collins wrote:
>>
>> I think your  and  tags are out of order (try doing a View 
>> Source on your page to see if the  filenames are there). And you should 
>> include a 
>>
>> Just reorder things a bit, something like:
>>
>> class MainHandler(webapp2.RequestHandler):
>> def get(self):
>> upload_url = blobstore.create_upload_url('/upload_file')
>>
>> *self.response.write("""*
>> **
>> **
>> *""")*
>>
>> files = DatastoreFile.query().order(-DatastoreFile.timestamp)
>> for file in files:
>> self.response.write(' %s' % file.filename)
>>
>> self.response.write("""
>> **
>> 
>>  Upload File: 
>> 
>> 
>> """.format(upload_url))
>>
>>
>> Also, look into using templates like Jinja as it will make it much easier 
>> to write and debug stuff like this.
>>
>>
>>
>> On Friday, 13 May 2016 18:43:30 UTC-7, Peter S wrote:
>>>
>>>
>>> I'm completely new to GAE and trying to figure some things out. I'm 
>>> trying to write a small app which will allow a user to upload files, and 
>>> display a list of those files already uploaded. I am having trouble getting 
>>> my app to list the files already uploaded. 
>>>
>>> The code works in so far as it allows me to upload a file, but it does 
>>> not print the list of files already uploaded. Any ideas where I'm going 
>>> wrong?
>>>
>>> My Code:
>>>
>>> from google.appengine.ext import blobstore
>>> from google.appengine.ext import ndb
>>> from google.appengine.ext.webapp import blobstore_handlers
>>> import webapp2
>>>
>>> class DatastoreFile(ndb.Model):
>>> data = ndb.BlobProperty(required=True)
>>> mimetype = ndb.StringProperty(required=True)
>>> filename = ndb.StringProperty(required=True)
>>> timestamp = ndb.DateTimeProperty(required=True,auto_now_add=True)
>>>
>>> class MainHandler(webapp2.RequestHandler):
>>> def get(self):
>>> upload_url = blobstore.create_upload_url('/upload_file')
>>>
>>> files = DatastoreFile.query().order(-DatastoreFile.timestamp)
>>> for file in files:
>>> self.response.write(' %s' % file.filename)
>>>
>>> self.response.write("""
>>> 
>>> 
>>> Upload File: 
>>> 
>>> 
>>> """.format(upload_url))
>>>
>>> class UploadFile(blobstore_handlers.BlobstoreUploadHandler):
>>> def post(self):
>>> file_data = self.request.POST.get('file_data', None)
>>>
>>> if file_data is not None:
>>> entity = DatastoreFile(
>>> data=file_data.value,
>>> filename=file_data.filename,
>>> mimetype=file_data.type
>>> )
>>> entity.put()
>>> response = {
>>> 'filename': entity.filename,
>>> 'timestamp': str(entity.timestamp),
>>> 'key': entity.key.urlsafe()
>>> }
>>>
>>>
>>> else: # no files were received
>>> response = {'error': 'file not received'}
>>> self.redirect('/')
>>>
>>> app = webapp2.WSGIApplication([
>>> ('/', MainHandler),
>>> ('/upload_file', UploadFile),
>>> ], debug=True)
>>>
>>>
>>>
>>>
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/3c4f1121-33e8-48c5-ba71-cd656e9cb6dd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: How do I list all of the files I've uploaded to the datastore?

[Jason Collins]

I think your  and  tags are out of order (try doing a View 
Source on your page to see if the  filenames are there). And you should 
include a 

Just reorder things a bit, something like:

class MainHandler(webapp2.RequestHandler):
def get(self):
upload_url = blobstore.create_upload_url('/upload_file')

*self.response.write("""*
**
**
*""")*

files = DatastoreFile.query().order(-DatastoreFile.timestamp)
for file in files:
self.response.write(' %s' % file.filename)

self.response.write("""
**

 Upload File: 


""".format(upload_url))


Also, look into using templates like Jinja as it will make it much easier 
to write and debug stuff like this.



On Friday, 13 May 2016 18:43:30 UTC-7, Peter S wrote:
>
>
> I'm completely new to GAE and trying to figure some things out. I'm trying 
> to write a small app which will allow a user to upload files, and display a 
> list of those files already uploaded. I am having trouble getting my app to 
> list the files already uploaded. 
>
> The code works in so far as it allows me to upload a file, but it does not 
> print the list of files already uploaded. Any ideas where I'm going wrong?
>
> My Code:
>
> from google.appengine.ext import blobstore
> from google.appengine.ext import ndb
> from google.appengine.ext.webapp import blobstore_handlers
> import webapp2
>
> class DatastoreFile(ndb.Model):
> data = ndb.BlobProperty(required=True)
> mimetype = ndb.StringProperty(required=True)
> filename = ndb.StringProperty(required=True)
> timestamp = ndb.DateTimeProperty(required=True,auto_now_add=True)
>
> class MainHandler(webapp2.RequestHandler):
> def get(self):
> upload_url = blobstore.create_upload_url('/upload_file')
>
> files = DatastoreFile.query().order(-DatastoreFile.timestamp)
> for file in files:
> self.response.write(' %s' % file.filename)
>
> self.response.write("""
> 
> 
> Upload File: 
> 
> 
> """.format(upload_url))
>
> class UploadFile(blobstore_handlers.BlobstoreUploadHandler):
> def post(self):
> file_data = self.request.POST.get('file_data', None)
>
> if file_data is not None:
> entity = DatastoreFile(
> data=file_data.value,
> filename=file_data.filename,
> mimetype=file_data.type
> )
> entity.put()
> response = {
> 'filename': entity.filename,
> 'timestamp': str(entity.timestamp),
> 'key': entity.key.urlsafe()
> }
>
>
> else: # no files were received
> response = {'error': 'file not received'}
> self.redirect('/')
>
> app = webapp2.WSGIApplication([
> ('/', MainHandler),
> ('/upload_file', UploadFile),
> ], debug=True)
>
>
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/e3196267-57eb-477f-a806-99cbd620fa31%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: Are wildcard subdomains and thousands of domains pointing to my domain supported?

[Jason Collins]

Here's my obligatory plug for an API to setup and manage custom domain 
mappings - please star it!

  https://code.google.com/p/googleappengine/issues/detail?id=8528



On Saturday, 12 December 2015 15:35:04 UTC-8, Jeff Schnitzer wrote:
>
> I run a whitelabeled ecommerce system on GAE with hundreds (and growing) 
> of custom domains. It can be done but there are a couple issues.
>
> 1) The onboarding process for each custom domain involves an extra "verify 
> your ownership of the domain to google" step. There does not appear to be 
> an API for this so it will involve handholding your partners through the 
> DNS configuration process.
>
> 2) The data on the "domains associated with your app" screen for your 
> project is wholly dependent on your *user account* and not the project 
> itself. This is batshit crazy insane. The only solution to preserve your 
> sanity is to create a fake user account and only ever touch the Custom 
> Domains screen using that fake user. 
>
> If you can live with a high-touch onboarding process, the benefits of GAE 
> outweigh the shortcomings WRT custom domains. If you're trying to run a 
> mass market solution like a public blog platform, GAE can't do it.
>
> Jeff
>
> On Fri, Dec 11, 2015 at 7:26 AM, Gitted  
> wrote:
>
>> Hello,
>>
>> The application will run in my GAE account 100%.  The only thing I will 
>> have to ask the client to do if they want to use their own custom domain 
>> (or subdomain) is to point it to my GAE server via domain or ip-address.
>>
>> So to summarize:
>>
>> 1. My company will code the application and deploy to my own GAE account 
>> for the domain www.exampe.com
>> 2. The customers will choose their subdomain and then they can access the 
>> application at:  customer1.example.com
>> 3. If the customer wants to use their own domain, they will have to point 
>> their domain to my GAE domain/ip address.
>>They customer may point the domain e.g. www.customer1.com or they 
>> might create a subdomain like:  example.customer1.com
>>
>> SSL has to be supported for both cases #2 and #3, but for #3 they will 
>> get their own SSL certification I would imagine.
>>
>>
>> So my question is, is the above possible?  
>> If yes, do I have to make a API call to your DNS service or something for 
>> a subdomain or their own custom domain to allow the request to go through 
>> your load-balancers?
>> Is there a limit to the # of subdomains or domains?  Say I want 10,000 of 
>> them?
>>
>>
>> Thanks.
>>
>>
>>
>>
>>
>>
>> On Tuesday, December 8, 2015 at 12:00:10 PM UTC-5, Patrice (Cloud 
>> Platform Support) wrote:
>>>
>>> Hi,
>>>
>>> It is definitely a possibility, but it depends on what you're using to 
>>> host. Do you code applications for your users and then deploy on App 
>>> Engine, or do you simply host their data in a GCS bucket (or a GCE 
>>> instance)? 
>>>
>>> We'd need more information into what you are doing precisely. I think in 
>>> the meantime, this article 
>>> 
>>>  will 
>>> help out a bit.
>>>
>>> Cheers! 
>>>
>>> On Monday, December 7, 2015 at 11:47:04 AM UTC-5, Gitted wrote:


 Hi,

 If I want to build a SaaS application using appengine, all of my 
 customers would either be using a sub-domain or their own custom domain.

 Would this be something that app-engine could support?

 Say my service uses the domain example.com.

 My customers would then get their own subdomain like:

 https://customer1.example.com
 https://customer2.example.com


 Or they would use their own domain, and then point their domain to my 
 service like:

 http://example.customer-domain.com   points to http://example.com or 
 its ipaddress.


 Hope this makes sense and someone can answer this for me.

>>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Google App Engine" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to google-appengi...@googlegroups.com .
>> To post to this group, send email to google-a...@googlegroups.com 
>> .
>> Visit this group at http://groups.google.com/group/google-appengine.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/google-appengine/6a29d048-0750-4538-8aa5-5f6b0b82a0cf%40googlegroups.com
>>  
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this 

[google-appengine] Re: bucket size for task.

[Jason Collins]

I think of it this way:

- a task will execute if there is a token in the bucket (by consuming the 
token)
- the bucket size can be defined to hold a particular number of tokens
- the bucket will refill tokens at the queue rate

So, when the bucket is "empty", you are effectively processing at your 
queue rate (i.e., the rate at which the bucket is being refilled). The max 
bucket size is 100, so this allows you to have a small burst in processing 
if desired. 

This is the clearest description from the docs:


*The task queue uses token buckets 
 to control the rate of task 
execution. Each named queue has a token bucket that holds a certain number 
of tokens, defined by the bucket_size directive. Each time your application 
executes a task, it uses a token. Your app continues processing tasks in 
the queue until the queue's bucket runs out of tokens. App Engine refills 
the bucket with new tokens continuously based on the rate that you 
specified for the queue.*



On Tuesday, 17 November 2015 11:10:26 UTC-8, Patrice (Cloud Platform 
Support) wrote:
>
> Hi,
>
> Since you have a bucket size of 5, the system will immediately start the 5 
> tasks you've asked. Then, because of the rate of your queue, over time 
> (each tenth of a second) your queue should get a new "token" that can be 
> used by your tasks. Depending on a lot of factor, the system can decide to 
> "hold" onto your token to only give you a significant amount (so instead of 
> a new token every tenth of a second, you may get a group of 5 tokens each 
> half-second). 
>
> For more information, I would suggest reading the official doc 
> 
>  (I 
> linked the section about processing rates, but the whole doc is helpful 
> here)
>
> Cheers!
>
> On Tuesday, November 17, 2015 at 12:09:36 PM UTC-5, Trez Ertzzer wrote:
>>
>> Hello.
>> I have been reading a lot of things about tasks and rate and bucket size 
>> but I cannot figure out how it works.
>>
>> suppose the rate is 10/s and the bucket size is 5.
>> suppose that I never added any task yet.
>>
>> now I add 20 tasks.
>> can you describe how the tasks will be executed?
>>
>> I understood, that 5 tasks will execute immediatly.
>> then what will happen...?
>> After how long another task will start? (and how many will start...)
>>
>>
>> thank you.
>>
>>
>> PS: I read this:
>>
>> https://groups.google.com/forum/#!searchin/google-appengine/bucket$20task/google-appengine/A1vC7raLfuY/h0Bi82Y5HCcJ
>>  
>> and this
>>
>> https://groups.google.com/forum/#!searchin/google-appengine/bucket$20size/google-appengine/4qBjwhvp_kU/wUlZAOJmOK8J
>>
>> but the information are really not clear
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/a3d98838-24a2-4669-bb8b-1e7e919d1342%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: "Request was aborted after waiting too long to attempt to service your request." sprees

[Jason Collins]

More min_idle_instances (at an increased cost of course) will help service 
bursts and avoid these pending queue timeouts.


On Monday, 2 November 2015 18:05:30 UTC-8, Kaan Soral wrote:
>
> Yes, AppEngine can't handle bursts well, so the solution is to slow things 
> down
>
> Actually, there are speed limits on a lot of things that are usually not 
> advertised well, if you are getting these kind of errors, try to slow 
> things down and see how it goes
>
> Slowing things down is also pretty challenging, much harder than building 
> stuff that will scale
>
> On Monday, November 2, 2015 at 6:48:57 PM UTC+3, Alexandra wrote:
>>
>> Hi,
>>
>> Did you find the reason of these errors?
>>
>> I also see a lot of them. The time which is elapsed is about 10 min, so 
>> it seems that there are tasks that are stuck for some reason in the task 
>> queue.
>>
>> Any insights on that?
>>
>> Thanks
>> Alexandra
>>
>> On Saturday, April 11, 2015 at 8:02:34 PM UTC+3, Kaan Soral wrote:
>>>
>>> "Request was aborted after waiting too long to attempt to service your 
>>> request."
>>>
>>> I've been seeing these messages a lot lately, momentarily many requests 
>>> log these errors, it floods the error logs
>>>
>>> Anyone else?
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/7b7bca79-e617-4660-bae3-dc0970c6c9e1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Masking using Google Domain

[Jason Collins]

Domain "masking" isn't achieved within the DNS proto itself. To do this, 
basically, you need to stand-up a webserver that serves out a frameset to 
the actual domain. (GoDaddy does this for you under the hood.)

You could do this on App Engine by creating a separate module that serves 
the frameset and then using dispatch.yaml to route requests for abc.com to 
this module that serves the iframe for xyz.com.

That said, domain "masking" is super-yuchy and totally breaks bookmarking. 
Don't do it. ;)



On Tuesday, 27 October 2015 10:32:56 UTC-7, Gaurav Nanda wrote:
>
> Hi,
>
> This may not be the right forum to ask, but someone might have some info.
>
> I recently bought a domain name from "Google domains" say "abc.com". I 
> want to forward "abc.com" to some "xyz.com" *without* changing the URL 
> address. I know I could do that in GoDaddy, there was simple option to 
> mask. However, I don't see it in Google Domain. 
>
> I am sure, im missing something. Could someone tell how could we do it.
>
> Thanks,
> Gaurav Nanda
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/847c6b39-c884-4841-8013-d2daa2c3ceda%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Old versions of Managed VMs

[Jason Collins]

Jeff, there is a tiny switch in the legacy console Versions tab that allows 
you to "stop" a managed VM. I believe this will delete the underlying VMs 
while still keeping your version and logs around. Big gotcha: you need to 
manually re-"start" the version to serve any traffic (regardless of your 
scaling configuration) - this has tripped me up a number of times.





I have not found the equivalent button in the new console, so I don't know 
how you'd achieve same there.

Also, this is still a rapidly evolving feature, so this advice is seriously 
time-limited! ;)

j


On Wednesday, 16 September 2015 14:05:58 UTC-7, Jeff Schnitzer wrote:
>
> Interesting. Maybe I just didn't wait long enough after deleting the 
> nondefault versions.
>
> Still, it's a small surprise that I have to prune old versions. With 
> normal GAE, I leave those versions around so I can go back through the logs 
> when necessary.
>
> Jeff
>
> On Wed, Sep 16, 2015 at 1:03 PM, Michael Spainhower  > wrote:
>
>> I am using managed vms and also have the issue of non-default versions 
>> running residual instances.  However, I do not experience the issue you 
>> have described where the instances remain even after deleting the 
>> non-default versions.
>>
>> Are you using a standard runtime or custom runtime (
>> https://cloud.google.com/appengine/docs/managed-vms/#runtimes)?  The 
>> lifecycle (shutdown) request handling seems like a potential culprit, so 
>> that may be worth examining if you are using a custom runtime.  I am using 
>> a node custom Dockerfile and my app just returns a 200 on requests to `
>> /_ah/stop` (
>> https://github.com/home-buddy/express-appengine-handlers/blob/master/index.js#L18).
>>   
>> You can probably check your logs for requests to that URL to see if there 
>> are non-200 responses.
>>
>> What version of the gcloud cli tool are you running?  While potentially 
>> unrelated to this issue, you may want to update it because I do not think 
>> `--remote` is an option anymore (implying you may need an update).
>>
>> As an aside, I would love guidance on fixing the issue where every 
>> non-default version has instances running.  Perhaps this is the expected 
>> behavior since managed VMs is labeled as beta?
>>
>>
>> On Tuesday, September 15, 2015 at 7:15:10 PM UTC-4, Jeff Schnitzer wrote:
>>>
>>> I have a python app on a Managed VM which I deploy with:
>>>
>>> gcloud preview app deploy app.yaml --remote --set-default
>>>
>>> It's set to manual scaling, instances 1.
>>>
>>> It appears that every time I deploy it, the old instance stick around 
>>> (and get billed for). Even deleting the old versions from the appengine 
>>> console didn't shut down the compute engine instances. I had to figure out 
>>> which compute engine instance was my "live" one and delete all the others 
>>> from the Compute Engine part of the console.
>>>
>>> Is this supposed to work this way or is there a better workflow? I just 
>>> want to deploy the new version and have the old version go away... like 
>>> regular App Engine.
>>>
>>> Jeff
>>>
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/e69f4b39-b1f2-495f-9aa2-6b862b2a8faa%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: How do I use Federated Login with OpenID Connect?

[Jason Collins]

And make sure you link the feature request here; I'll gladly star it.

On Monday, 10 August 2015 16:29:06 UTC-7, Nick (Cloud Platform Support) 
wrote:

 Hi NP,

 They do indeed appear to be presently parallel systems. You can explore 
 the App Engine Users service and Google+ (OpenID Connect) authentication at 
 the example app gae-login-explainer.appspot.com, which goes into some 
 detail.

 I think you've got the makings of a great feature request here, and 
 encourage you to summarize the results into a Feature Request issue report 
 in the public issue tracker 
 http://code.google.com/p/google-appengine/issues/list, to request 
 tighter integration of login: required / admin in app.yaml and OpenID 
 Connect (Federated Login).

 Best wishes,

 Nick

 On Monday, August 10, 2015 at 2:47:39 AM UTC-4, NP wrote:

 Hi Nick,

 Thanks for your response.

 On further research, I discovered that the Users API from 
 google.appengine.api.users has no relation to the new Google OpenID 
 Connect i.e. even after I've successfully logged in via Google OpenID 
 Connect, calling Users.get_current_user() will return None. It seems Google 
 has not yet provided a way for the Users API to work with the Google OpenID 
 Connect. From what I understand if your application uses Federated Login, 
 then there is currently no way to support urls restricted to admin or login 
 required via app.yaml.



 On Thursday, August 6, 2015 at 3:58:28 PM UTC-7, Nick (Cloud Platform 
 Support) wrote:

 Hey NP,

 This may very well be a side-effect of the decommissioning of Google's 
 OpenID provider service and the moving to OpenID Connect. 

 Could you link which published Google Libraries for OpenID Connect 
 you've used, and if possible some of the code relevant to your use-case?

 As noted in the docs 
 https://cloud.google.com/appengine/docs/python/users/functions#create_login_url,
  
 support for OpenID was experimental, probably since the landscape of Auth 
 has been changing quite a bit, as new improvements are made through the 
 experience of successive paradigms, however all I really need to comment on 
 is that it was experimental. Despite this, I'm committed to helping you 
 find a solution that works.

 There are several docs in our Developers resources which discuss OpenID 
 Connect [1] 
 https://developers.google.com/identity/protocols/OpenID2Migration?hl=en, 
 [2] https://developers.google.com/+/web/api/rest/openidconnect/, [3] 
 https://developers.google.com/identity/protocols/OpenIDConnect?hl=en, 
 and you might want to give those a read. If the login: admin feature turns 
 out to have been specific to Google OpenID accounts, you can rest assured 
 that:

 A) A feature request in the public issue tracker 
 http://code.google.com/p/google-appengine/issues/list to get this 
 function updated to work with OpenID Connect is possible and awesome

 B) you can implement your own admin feature using an admin check in your 
 request handlers and implementing an OpenID Connect auth scheme which 
 allows accounts from OpenID Connect services (you would add the various 
 sign-in buttons to the sign-in page of your (web)app).

 Best wishes,

 Nick

 On Tuesday, August 4, 2015 at 1:08:50 PM UTC-4, NP wrote:

 Hello all,

 I have an application (python) on GAE which uses Federated Login. In 
 app.yaml, I have restricted some urls to be accessible only to admins 
 (i.e. 
 login; admin). According to GAE documentation, any url restricted to admin 
 for applications using Federated Login will trigger a call to 
 /_ah/loginrequired and I'm supposed to handle such calls by getting user 
 to 
 login using* users.create_login_url(dest_url, federated_identity).*

 Documentation for users.create_login_url says if no federated_identity 
 is specified (when your site is set to use federated identity), the system 
 will default to Google as the OpenID provider. So the application 
 currently 
 defaults to  'https://www.google.com/accounts/o8/id' which is no 
 longer supported by Google and I believe this is the reason why it gives 
 me 
 a 500 server error.

 I have tried using the published Google Libraries for OpenID Connect. 
 If I use the library outside of the create_login_url, I'm able to log in 
 and get the user email but calling users.get_current_user() returns None 
 and so the application doesn't see me as being logged in which means I 
 don't get access to the url that I had restricted to admins. If I apply 
 the 
 OpenID Connect url to create_login_url, I get a 500 server error.

 Does anybody know how to handle requests to urls restricted to 
 logged-in users/admins using the new OpenID Connect?




-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at 

[google-appengine] Re: Problem authenticating to GAE app using GoogleCredential OAuth2

[Jason Collins]

Julian, can you post your link to your SO question?


On Thursday, 6 August 2015 12:20:28 UTC-7, Julian Bunn wrote:

 Hi Nick,

 Many thanks - I had already posted on stackoverflow with no luck, so came 
 here :-) I do have one reply now over there, which suggests using client 
 secrets, so that is a good lead. Also your comments on the use of service 
 account are well taken - it looks like that may be inappropriate.

 Thanks for the pointers to the documentation, which I'd already visited 
 and read but ended up being confused - as is no doubt evident from my 
 question :-)

 Julian

 On Wednesday, August 5, 2015 at 4:57:26 PM UTC-7, Nick (Cloud Platform 
 Support) wrote:

 Hi Julian,

 You've produced an excellent post which would belong on stackoverflow.com. 
 Google Groups isn't the place to post specific technical issues, as this 
 forum is meant more for general discussion of the platform and services. 

 I'll give you the advice before you post there that it seems you've 
 combined examples from different kinds of OAuth flow and this might be the 
 cause of your issues. I see that there's a variable emailScope - this is 
 a scope which a user would actually grant to your application, not one 
 which a service account could grant. 

 The service account and its credentials are used to call APIs on behalf 
 of your application, although I don't think I've seen this pattern before, 
 where you want to call an endpoint on your own app using a service account. 
 As far as I know, service accounts have only been used to authenticate with 
 Google APIs, although I suppose it might be possible to write an endpoint 
 which correctly authenticates it.

 You could do some more reading on OAuth2 
 https://developers.google.com/identity/protocols/OAuth2, OpenID Connect 
 https://developers.google.com/identity/protocols/OpenIDConnect?hl=en, 
 Service 
 Accounts 
 https://developers.google.com/identity/protocols/OAuth2ServiceAccount, 
 and the Google Identity Platform 
 https://developers.google.com/identity/, and try to repost your 
 question to stackoverflow.com. That would be the best action as there 
 are many more users there ready to help with a technical question.

 If you would like to open a thread in this forum discussing the platform 
 or services in more broad terms, starting a discussion that would be useful 
 for other users to join in to, feel free to do so.

 Have a great day!

 [1] http://www.stackoverflow.com/
 [2] http://www.serverfault.com/
 [3] http://code.google.com/p/google-appengine/issues/list

 On Wednesday, August 5, 2015 at 1:32:41 AM UTC-4, Julian Bunn wrote:

 I have a GAE application with an endpoint that requires authentication, 
 which I need to call from an application (rather than from in a browser). I 
 was using ClientLogin, but that is now obsolete, so I have set up a Service 
 Account in the Google Console, and stored its keypair .p12 file so that I 
 can use the OAuth methods as described in the documentation.

 Although the GoogleCredential builder successfully returns an 
 authorization token, if I then use that token in an HTTP get call to the 
 endpoint, the response is always the Google Login page.

 Why, if I use the token, does GAE not take my application call as 
 authorized? Am I doing this all wrong or missing a step? 

 Here is the code:

 String emailAddress = ...@developer.gserviceaccount.com;
 JsonFactory JSON_FACTORY = JacksonFactory.getDefaultInstance();
 String emailScope = https://www.googleapis.com/auth/userinfo.email;;
 String keyFileName = Y.p12;
 String baseURL = http://Z.appspot.com;;
 HttpTransport httpTransport;
 try {
 httpTransport = GoogleNetHttpTransport.newTrustedTransport();

 File keyFile = new File(keyFileName);
 if(!keyFile.exists()) {
 System.err.println(Key file +keyFileName+ missing);
 System.exit(0);
 }

 GoogleCredential credential = new GoogleCredential.Builder()
 .setTransport(httpTransport)
 .setJsonFactory(JSON_FACTORY)
 .setServiceAccountId(emailAddress)
 .setServiceAccountScopes(Collections.singleton(emailScope))
 .setServiceAccountPrivateKeyFromP12File(keyFile)
 .build();

 boolean success = credential.refreshToken();
 System.out.println(Access token refresh + success);

 String token = credential.getAccessToken();

 System.out.println(Token +token);

 String uri = http://Z.appspot.com/gcm/home;;

 System.out.println(uri:  + uri);

 HttpGet get = new HttpGet(uri);
 get.setHeader(Cookie, token);

 HttpClient client = new DefaultHttpClient();
 HttpResponse response = client.execute(get);
 response.getEntity().writeTo(System.out);

 Typical output:

Access token refresh true
Token ya29.xQGG1kxxx
uri: http://Z.appspot.com/gcm/home

!DOCTYPE html
html 

Re: [google-appengine] Re: appstats analysis - Same request - Too much difference in serving time.

[Jason Collins]

Rajesh - if you're performing *exactly* the same execution and are simply 
asking about the variability - that's just sort of the nature of the 
distributed RPCs. There can sometimes be pretty wide range of response 
times in RPCs.

Depending on your frameworks, there are ways to execute queries in parallel 
which may help. Also, your framework may be trying to help by 
automatically using memcache (e.g., NDB does this, though not for query 
results); if you memcache hit rate is low, then all the memcache overhead 
is just waste.


On Tuesday, 4 August 2015 09:21:10 UTC-7, Alex Martelli wrote:



 On Mon, Aug 3, 2015 at 4:43 AM, Rajesh Gupta 
 rajesh...@veersoftsolutions.com javascript: wrote:

 Hi,
 I have posted this post, with attachments twice.  But it did not make 
 into the group.  Any limitation on posting with attachments.


 Sorry Rajesh, it was a temporary glitch with the group -- I have now 
 unblocked your messages, and future ones from you should just go through 
 (sorry I don't know enough to actually help w/your question, but we can 
 hope somebody else on the group can).

 Alex
  

   Any settings to be enabled.
 I have problem with serving the same request with big difference in the 
 datastore and memcache timings.  Please see the below email.

 On Sat, Aug 1, 2015 at 10:12 AM, Rajesh Gupta 
 rajesh...@veersoftsolutions.com javascript: wrote:

 resending... as it did not post earlier.


 Attached are the two screen shots showing the appstats
 - same request
 - same appengine instance
 - same user

 The first request took 2914 ms, and the second request takes only 1032ms
 As you can see in the attached images, 

 3rd line datastore_v3.RunQuery, difference of 9ms 
 5th line memcache.get, difference of 73ms
 6th line memcache.get, differnce 73ms

 Through out the request, there is lot of time difference in the 
 datastore and memcache calls, between the two requests.

 What can it done to make consistent.

 -- 
 Rajesh
 *www.VeersoftSolutions.com http://www.VeersoftSolutions.com*
 *www.AccountingGuru.net http://www.AccountingGuru.net*
 *Accounting/Inventory/Orders on Google Cloud Platform and Mobile*




 -- 
 Rajesh
 *www.VeersoftSolutions.com http://www.VeersoftSolutions.com*
 *www.AccountingGuru.net http://www.AccountingGuru.net*
 *Accounting/Inventory/Orders on Google Cloud Platform and Mobile*




 -- 
 Rajesh
 *www.VeersoftSolutions.com http://www.VeersoftSolutions.com*
 *www.AccountingGuru.net http://www.AccountingGuru.net*
 *Accounting/Inventory/Orders on Google Cloud Platform and Mobile*

 -- 
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to google-appengi...@googlegroups.com javascript:.
 To post to this group, send email to google-a...@googlegroups.com 
 javascript:.
 Visit this group at http://groups.google.com/group/google-appengine.
 To view this discussion on the web visit 
 https://groups.google.com/d/msgid/google-appengine/CA%2BS7ijYGLJvcPK8Cfi7Vmv%2B%3Dekx5BqyVxPeMdKt-o9K9Hdc1vw%40mail.gmail.com
  
 https://groups.google.com/d/msgid/google-appengine/CA%2BS7ijYGLJvcPK8Cfi7Vmv%2B%3Dekx5BqyVxPeMdKt-o9K9Hdc1vw%40mail.gmail.com?utm_medium=emailutm_source=footer
 .

 For more options, visit https://groups.google.com/d/optout.




-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/6767dfbf-4505-42ed-8549-f0d1796bddf3%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Have you had success with Traffic Splitting? How do you use it to roll out new versions?

[Jason Collins]

Traffic-splitting / canary releases on App Engine are definitely a thing.

Traffic-splits on non-default modules are now available via API:

  
https://cloud.google.com/appengine/docs/admin-api/quickstart/#splitting_traffic


On Friday, 24 July 2015 11:50:33 UTC-7, Michael Spainhower wrote:

 @David, I started following this thread because I have the exact same 
 question and agree the lack of response is worrisome.

 We are in the Cloud Startup Program and I plan to ask about canary testing 
 during my next engineering 1-on-1.  I will reply to this thread with what I 
 learn from their engineer.



 On Friday, July 24, 2015 at 9:14:02 AM UTC-4, David Hardwick wrote:

 Oh boy, the lack of response here is not encouraging

 On Tuesday, July 21, 2015 at 1:53:17 PM UTC-4, David Hardwick wrote:

 Hello,

 We haven't used Traffic Splitting yet but it has been an available 
 feature for a while and rumor has it that traffic splitting for non-default 
 modules could be coming in as soon as a month.  

 Any who, if you have experience use it, then I would like to hear how 
 you are using it to roll out new features or versions.  I've heard the term 
 'canary' testing where you roll out a new version to 10% of folks...you 
 measure the results and then either rollback and fully roll it out.  So if 
 anyone is doing 'canary' testing and deployments as I've described it, then 
 I've like to hear from you.

 Thanks in advance,
   Hardwick



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/21cfeabb-9334-4af9-b143-82b42d97a199%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Odd console behavior WRT custom domains

[Jason Collins]

*It sounds like, for now, the only way to get the truth from the 
CustomDomains page is to always have all domains added/removed by a single 
Googleuser account, and only look at that page from that account. Right? It 
seemscrazy but ok. *

This is right. And yes it seems crazy.


On Thursday, 9 July 2015 21:57:45 UTC-7, Jeff Schnitzer wrote:

 Now you have me even more confused ;-)

 Is there anything in the TOS that I should pay particular attention to? I 
 just re-read it and don't see anything unusual. My application is an 
 ecommerce site serving a particular niche; partners set up shops in our 
 application using their own domain. Something like Shopify.

 Our onboarding process is fairly high-touch so going through the domain 
 verification process manually is not prohibitive. However, not having a 
 coherent view of what domains are mapped to our application could be a 
 serious problem. The domains will not be Google Apps domains.

 It sounds like, for now, the only way to get the truth from the Custom 
 Domains page is to always have all domains added/removed by a single Google 
 user account, and only look at that page from that account. Right? It seems 
 crazy but ok. Would it help if I opened a public issue to track this, or is 
 the internal issue enough?

 I read the documentation for the Google+ Domains API and I am only more 
 confused - it does not seem relevant to my application. Am I missing 
 something?

 Yep, had that issue starred for years... long enough to see the major 
 progress you guys have made separating GAE from Google Apps. I know you get 
 a lot of flak for the slow progress, but I do appreciate that the current 
 situation is vastly better than it was in 2012. Fortunately I wasn't trying 
 to run a whitelabel service in 2012 :)

 Thanks,
 Jeff



 On Thu, Jul 9, 2015 at 4:42 PM, Nick (Cloud Platform Support) 
 pay...@google.com javascript: wrote:

 Hey Jeff,

 I'd recommend that you get in touch with sales 
 https://cloud.google.com/contact/ and check the terms of service 
 https://cloud.google.com/terms/, to ensure that everything's good. 
 There's also a public issue tracker feature request 
 https://code.google.com/p/googleappengine/issues/detail?id=8528q=custom%20domains%20apicolspec=ID%20Type%20Component%20Status%20Stars%20Summary%20Language%20Priority%20Owner%20Log
  open 
 for a Custom Domains API, which you should star to register interest and 
 lend weight to. Finally, I'm not sure if you've already checked this out, 
 but there's a Domains API https://developers.google.com/+/domains/ for 
 Google+ that might fill some needs for you.

 Best wishes,

 Nick


 On Thursday, July 9, 2015 at 1:16:50 PM UTC-4, Jeff Schnitzer wrote:

 Yes, please make this a single place where all administrators can manage 
 the domains pointing at an app. We're running a whitelabel service which 
 will eventually have thousands of entries and management of domains can't 
 be tied to specific user accounts. This is going to be really confusing.

 Thanks,
 Jeff

 On Wed, Jul 8, 2015 at 3:02 PM, Nick (Cloud Platform Support) 
 pay...@google.com javascript: wrote:

 Hey Jeff,

 As of now, the Developers Console only shows domains verified by you. 
 We've an internal request to make this information available to all 
 owners/developers of app. In the meantime, other owners can use Google 
 Apps 
 control panel to view this information, if these domains are managed by 
 Google Apps. You can also add other owners of this project as owners in 
 Webmaster tools for the domain. This will enable them to view custom 
 domain 
 mappings in the Developers Console. 

 Regards,

 Nick


 On Wednesday, July 8, 2015 at 2:12:47 PM UTC-4, Jeff Schnitzer wrote:

 This looks like some sort of bug:

 OwnerA added customdomain1 to TheProject
 OwnerB added customdomain2 to TheProject

 When OwnerA looks at the Custom Domains page for TheProject in the 
 console, he sees customdomain1 only.

 When OwnerB looks at the Custom Domains page for TheProject in the 
 console, he sees customdomain2 only.

 Oddly, both custom domains seem to work.

 I would expect that all app owners see all custom domains for 
 TheProject in the console. Am I missing something?

 Thanks,
 Jeff





-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/3753dc94-157e-493c-9c8e-c26b55375edb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: [Python] NDB Update taking long time

[Jason Collins]

Hi Michael, the flag you are referencing is for the dev_appserver, which 
allows the simulation of eventually consistent query behavior on the local 
development environment.

On production, you always have to contend with eventual consistency when 
querying (exception: ancestor queries).

Saturnino, are you seeing that a key.get() is returning stale data, or a 
query result? If query result, there could be latency, but 1 day is a 
_huge_ amount of time and you shouldn't be seeing this.


On Friday, 19 June 2015 10:41:53 UTC-7, Michael (Cloud Platform Support) 
wrote:

 Hi Saturnino!

 If you mean you are experiencing a delay in Datastore consistency, I 
 recommend including the flag --datastore_consistency_policy=consistent when 
 deploying your application (within the Application Settings in the Google 
 App Engine Launcher).

 Cheers!

 On Thursday, June 18, 2015 at 4:05:56 PM UTC-4, Saturnino Mateus wrote:

 Hello;

 I have an entity with some attributes is a list of objects, but i noticed 
 that when i update the object it takes about 1 day to update. Why this 
 happens? What can i do?

 Regards



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/c917d2d9-9f96-4add-b5a9-c535433d5f9c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Google Cloud Datastore Incremental Backups

[Jason Collins]

I have an old feature request here if you want to star it:

  https://code.google.com/p/googleappengine/issues/detail?id=7428


On Tuesday, 16 June 2015 08:37:46 UTC-7, Ryan (Cloud Platform Support) 
wrote:

 Salutations Gubbi,

 This Feature Request would best be posted in the Public Issue Tracker 
 https://code.google.com/p/googleappengine/issues/entry?template=Feature%20request.
  
 When you fill out the report let me know on this thread and I will process 
 it for you.

 On Tuesday, June 16, 2015 at 10:53:47 AM UTC-4, Gubbi wrote:

 Hi,

 The official backup method only supports full backup as of now. I don't 
 know when, if at all, it'll support incremental backup.  It'll be wasteful 
 to take full backups every time.

 What strategies are you using for taking incremental backups of your 
 datastore? 


 Thanks,
 Vinuth.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/bc5fd96d-bd07-48e9-8c33-55b5d77b4c20%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: serve ssl requests on both custom domain and appspot.com

[Jason Collins]

Yes, the existing https://myapp.appspot.com will continue to work after you 
set up SNI (or VIP) SSL on a custom domain.


On Tuesday, 2 June 2015 07:27:43 UTC-7, Kehinde Ayanniyi wrote:

 Am planning to develop an app that is free to the end users. The little 
 cost of running the server will be taken care of by me, so cost saving is 
 important in this context. 

  The app requires that users access via SSL/TLS, so I will be using google 
 apps for domain to setup an SSL custom domain. Because of limited budget, 
 using VIP with the SSL certificate is out of the option, so am left with 
 using SNI which I guess is free for the first 5 slots. Ofcourse the 
 down-side of using SNI SSL is that old platforms like IE on XP and Android 
 2.X.X will have issues with my app. As a work around to this SNI 
 limitation, I consider to detect and redirect users on old platform where 
 SNI will fail to the https://myapp.appspot.com instead of the default 
 https://www.myapp.com. 

 So I will like to know whether after I set up custom domain with SSL for 
 my GAE app, will I still be able to use https://myapp.appspot.com to 
 access my app and at the same time still have the https://www.myapp.com 
 working correctly.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/4f6df432-16e7-4972-b574-dd634f522e8d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Appengine Logs and BigQuery

[Jason Collins]

Even easier: use table wildcards.

 https://cloud.google.com/bigquery/query-reference#tablewildcardfunctions

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/a56d1d53-5612-4951-bb9b-4ccb5e8b0e26%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: New instance spinning for user facing apps

[Jason Collins]

Similar to this? 
https://code.google.com/p/googleappengine/issues/detail?id=7865


On Monday, 4 May 2015 03:17:24 UTC-7, aswath wrote:

 A new instance is started for a user facing request.  In the logs, you can 
 see that for the following endpoint request, a new instance is started, and 
 it takes 21718ms to serve the request. 


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/0ee05fba-f06e-40c6-a6ce-f70ea1795d1c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: How to alleviate 200gb/day - 20$+ daily outgoing bandwidth image serving costs

[Jason Collins]

Hi Kaan,

Out of curiosity, have you tried resampling the PNG (while leaving them as 
PNG) with something like https://imageoptim.com/? I've seen huge amounts of 
size come out of images, without any noticeable degredation to the image 
quality.

j

On Saturday, 2 May 2015 15:41:18 UTC-7, Kaan Soral wrote:

 Thanks a lot for all your input

 I've cranked up the default_expiration to 60d, yet I doubt it will have 
 any effect, unless there is a cut-off structure and 1h is below that 
 cut-off, the majority of these images are consumed by an API which is 
 pretty much a black-box

 My monthly usage is around 6TB's, so cloud storage isn't worth the effort, 
 app engine is 0.12/gb, while cloud storage becomes 0.11/gb as far as I see

 If all else fails, I will probably move the serving to AWS, as the 33% 
 saves stack up to $2k+ in a year, or just degrade the images and reduce the 
 user satisfaction, but the AWS option is obviously better

 -

 On the other hand, Cloudflare might reduce this cost to 0, but I'm not 
 sure about the reliability, they don't advertise limits for their free CDN, 
 there were also some App Engine incompatibilities in the past, their system 
 might also block the API whose access is crucial


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/167b15f1-6cfc-436e-96e7-6f02945761ae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Google Cloud Platform wants to hear from you

[Jason Collins]

 we need to process logs before sending to bigquery

 
Aleem, can you tell me more about what processing you do on your logs 
before sending them to BigQuery?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/91564e30-0a50-460d-8e7f-680871272c34%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: [SEVERE] App Engine can't consistently execute tasks

[Jason Collins]

Hi Kaan,

Just lending support to this:

engineer told me that internal/silent/invisible execution failures also 
 count as task retries


I have also been told this. We have an open-source framework for doing 
workflows based on task queue and one of the features we tried to build in 
to it was an alerting system when the final task retry failed (and thus the 
task would go away permanently). We had to give up on the feature because 
it was possible (and we saw it) for tasks to fail before ever hitting our 
code, and this counts as a retry, and if it happened to be on the last 
retry, then our alert code wouldn't get a chance to operate. 

In short, it would have been an unreliable feature. Note, this was a long 
time ago though (3+ years).

j

On Monday, 23 February 2015 23:49:44 UTC-6, Kaan Soral wrote:

 If there was any logs, it wouldn't be an issue, I would likely solved the 
 issue on the logs, the issue is there are no logs

 With this recent issue, the only error logs are some instances dying 
 critically, from memory overflows's (after going over 256mb's - my theory 
 is a ndb memory leak, but it's offtopic)
 (There are 5-6 error logs like these, all instance overflows, since there 
 are tens of thousands of other operations, the error ratio is low)

 In this specific case, the critical death of instances might be related to 
 the issue, those instances might be taking some tasks with them, and leave 
 the tasks unexecuted (theory)

 I will dig in and find the discussion



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/db137884-3510-4042-909a-7781f5283d9d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: [SEVERE] App Engine can't consistently execute tasks

[Jason Collins]

 basic_scaling made the issue worse somehow


So I suspect that the silent failures occur when the dispatcher is 
attempting to service a dequeued task, but cannot find an available 
instance (nor spin one up) in some unknown period of time.

Just a guess though.

j

On Tuesday, 24 February 2015 12:45:18 UTC-6, Kaan Soral wrote:

 You can't see these issues from the logs, yet you can see them as 5xx 
 spikes from the error *graphs*/dashboard (most of the time)


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/39b982b5-9e9d-4c73-8775-7642933e78ad%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: [SEVERE] App Engine can't consistently execute tasks

[Jason Collins]

If you're looking for guaranteed task execution, you probably don't want to 
specify max retries or max age.

But regardless, the fact (if it is still a fact) that a task with max 
retries might never hit your code before disappearing is still a pretty 
unfortunate thing.

j

On Tuesday, 24 February 2015 11:20:55 UTC-6, husayt wrote:

 Hi Kaan,
 I had exactly same issue (and maybe still have, as there is no easy way to 
 find it). This caused us a huge problem as we were migrating our namespaces 
 to new dataschema and we only found out after some namespaces stopped 
 working after migration.
 Upon examination we found out that that tasks didn't run for these 
 namespaces, there were no error logs and I don't know how many of these 
 missed tasks we had before and after.

 This is scary. It happens during bursts when we run many tasks at once. 
 The major problem is the absence of logs.
 It is difficult to reproduce and even more difficult to catch them.

 Task queues are offered as a guaranteed task execution solution, but this 
 shows they are not.

 Best,
 HG


 On Tuesday, February 24, 2015 at 2:48:34 PM UTC, Jason Collins wrote:

 Hi Kaan,

 Just lending support to this:

 engineer told me that internal/silent/invisible execution failures also 
 count as task retries


 I have also been told this. We have an open-source framework for doing 
 workflows based on task queue and one of the features we tried to build in 
 to it was an alerting system when the final task retry failed (and thus the 
 task would go away permanently). We had to give up on the feature because 
 it was possible (and we saw it) for tasks to fail before ever hitting our 
 code, and this counts as a retry, and if it happened to be on the last 
 retry, then our alert code wouldn't get a chance to operate. 

 In short, it would have been an unreliable feature. Note, this was a long 
 time ago though (3+ years).

 j

 On Monday, 23 February 2015 23:49:44 UTC-6, Kaan Soral wrote:

 If there was any logs, it wouldn't be an issue, I would likely solved 
 the issue on the logs, the issue is there are no logs

 With this recent issue, the only error logs are some instances dying 
 critically, from memory overflows's (after going over 256mb's - my theory 
 is a ndb memory leak, but it's offtopic)
 (There are 5-6 error logs like these, all instance overflows, since 
 there are tens of thousands of other operations, the error ratio is low)

 In this specific case, the critical death of instances might be related 
 to the issue, those instances might be taking some tasks with them, and 
 leave the tasks unexecuted (theory)

 I will dig in and find the discussion



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/dad9e53b-c5e5-461e-8ca9-82acff11ed50%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: New Datastore query tool

[Jason Collins]

Well, Google I/O is approaching, and I'm sure they'd like to have stuff to 
announce...
j

On Friday, 13 February 2015 17:53:31 UTC-6, husayt wrote:

 Hi Mario. 

 this is strange considering tool was already much better than current 
 broken one.

 Considering that there was no update to 1.9.17 for last two months, this 
 makes me worry. I remember GAE team was very proud by how lean they were 
 and aiming to do monthly release cycles.

 Yes there were improvements to Google cloud in general, but no single 
 exciting new feature has been added to Appengine SDK for a whole year now.

 See for yourselves:
 https://code.google.com/p/googleappengine/wiki/SdkForJavaReleaseNotes

 only minor bug fixes and that's it. 

 Unless i miss something, or things are not announced release notes 
 properly, but this really worries me and i am sure many others here.

 I am very keen to find out about perspectives of developing with GAE.

 Thanks.
 HG


 On Thursday, February 12, 2015 at 12:34:41 PM UTC, Mario wrote:

 Hi Huseyn,

 I'm glad that you liked it.

 As I said in my previous message we don't have yet any information about 
 the release date of that feature.

 Regards

 Mario C.
 Google Cloud Platform Support

 On Wednesday, February 11, 2015 at 3:16:46 PM UTC+1, husayt wrote:

 Yes I use that one, which is extremely slow and buggy.

 For couple of days in january there was updated version which was much 
 better, but unfortunately that was replaced with old one again.

 On Wednesday, February 11, 2015 at 10:51:38 AM UTC, camaram wrote:

 Hi again,

 I'd like to confirm that you're using the Query Tool in the Developers' 
 Console [1] and not the one in the old App Engine Console? I was talking 
 about the former one.

 Regards,

 [1] https://console.developers.google.com/project/*YOUR_PROJECT_ID*
 /datastore/query

 On Tuesday, February 10, 2015 at 9:54:48 AM UTC+1, camaram wrote:

 Hello H.,

 Currently we lack information on when that new Datastore Query tool 
 will be available for public use.

 Please, follow the official Google Cloud Platform blog 
 http://googlecloudplatform.blogspot.comfor updates on new products 
 and features.

 Mario C.
 Google Cloud Platform Support.

 On Thursday, February 5, 2015 at 2:07:25 PM UTC+1, husayt wrote:

 Hi,

 we had this new uber cool datastore query tool enabled couple of 
 weeks ago. That was great, but sadly it was switched back to the old one 
 next day.

 I am in anticipation of getting the new query tool back.

 Can someone from Google please advise?

 Thanks,
 Huseyn Guliyev



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/e6316f5a-ec07-420c-9560-f50da37b625c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Distributed Promises library for python on appengine

[Jason Collins]

Maybe I can help? I've packaged a few App Engine related libraries. The 
largest, though pretty dated now is Fantasm 
https://code.google.com/p/fantasm/ - in fact, it sought to solve a similar 
problem as your lib in a way, i.e., leveraging taskqueue retries and 
asynchronous operations while abstracting away the overhead of taskqueue 
itself.

Let me know,
j

On Thursday, 20 November 2014 21:43:49 UTC-6, Emlyn wrote:

 Hi AppEngine Pythonistas, 

 I've written a useful library for distributed processing in AppEngine, 
 implementing distributed promises. They're described in detail in this 
 post: 

 https://point7.wordpress.com/2014/11/21/distributed-promises-for-appengine/ 

 I want to open source the library, can anyone help me with best 
 practices for creating an open source python package, including how to 
 make that most easily accessible to appengine developers? 

 -- 
 Emlyn 

 http://point7.wordpress.com - My blog 
 https://plus.google.com/u/0/100281903174934656260 - Google+ 


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Modules 101

[Jason Collins]

Note that you can't directly call code between modules. Effectively, you 
call from one module to the other with urlfetch (e.g., taskqueue).

App Engine is a distributed computing platform. Every time you flip 
versions, you have multiple versions in flight for some unknown period of 
time. This is business as usual and your app needs to allow for that. 
Modules is no different.

j

On Sunday, 12 October 2014 00:48:05 UTC-6, Kaan Soral wrote:

 The conversion was very smooth, I only added one dispatch entry and 
 modified my defer wrapper to set target's to the new background module

 *One very serious issue, correct me if I'm wrong*
 It seems there is no way to upload modules in a transaction, they are 
 uploaded one after another, this creates a temporary code discrepancy, 
 which might cause serious and confusing issues, once appengine rolled back 
 to previous deployments as a bug, and I remember how confusing it was at 
 first, this might creates issues like that if one isn't always on the watch

 If you are uploading module1 and module2, which depend on each other and 
 share the codebase, if you deploy something that will be used in the first 
 30 seconds, your app will probably fail, data/states/requests might be lost 
 or malformed during this stage
 It could be much much worse if module2 fails to deploy, and the gap 
 increases to minutes instead

 One workaround is to always increase versions and bump up to the new 
 versions after deployment, which would be a huge burden (even in that 
 method, there will be seconds/milliseconds of gaps)

 If I'm not mistaken, I think it might be a good idea to create an issue 
 about this and urge the deployment routine to be improved


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Modules 101

[Jason Collins]

The biggest hurdle was realizing that dispatch.yaml, queue.yaml, cron.yaml, 
index.yaml all update separately from the app(s)/modules themselves. 
(update_dispatch, update_queue, etc.)

Other than that, it is really easy and you should totally do it. If your 
url namespace is well structured, you can use dispatch.yaml to move traffic 
to different modules very, very easily.

j

On Friday, 10 October 2014 17:33:09 UTC-6, Kaan Soral wrote:

 I've been interested in Modules, mainly to:
 1) Seperate the logs / components of the app, to ease the debugging and 
 monitoring
 2) Hope to increase the performance, leave the frontend instances to 
 user-facing requests only (?)

 The documentation is very adequate, however, as all services, you have to 
 dive into it to actually learn it

 What I learned:
 1) You have to use a manual appcfg command, manually cd the directory, 
 appcfg update app.yaml module.yaml's
  (Which is something everyone should do anyway, imo, I only use the 
 gui/launcher for deployment, I will probably not do that anymore)
 2) Each module seems to be like a separate app, that shares the appengine 
 services like datastore etc. (Great!)
 3) Each module gets uploaded separately (the command does this), 
 multiplying the deployment time/risk
  (It's kind of obvious, as module's are separate components, however the 
 risks are unknown)
  (Does the rollback command work the same way? - for example this is one 
 risk, unknown)
 4) The UI/AppEngine Console improves with module tabs at each tab like 
 Dashboard/Instances/Logs/Versions (Great!)
 5) Each module .yaml should include everything, similar to app.yaml, like 
 libraries etc
 6) I'm guessing the costs will increase a bit, however, it might also 
 decrease the costs of some high traffic apps as it will allow throttling at 
 the backend level

 I have some questions to those who are experienced with Modules:
 1) Are there any side-effects, not obvious thing that are major road 
 blocks? ( Are the GCS/Datastore/Memcached all shared etc? - They should be )
 2) How does the SDK handle things? (I've yet to test the SDK)
 3) Any advice?


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] HTTPS as a ranking signal

[Jason Collins]

Re: 
http://googlewebmastercentral.blogspot.ca/2014/08/https-as-ranking-signal.html

Ok Google, now we most definitely need an API to manage custom domains, 
mount certificates, and configure HTTPS hosting using an API without a 
Google account requirement.

Our Cloud Platform hosted products are whitelabelled, meaning people can 
serve them on their own domain. As the world moves towards HTTPS-always, we 
need a scaleable way for our customers to upload and configure their SSL 
certificates fronting our products without our involvement - and without 
requiring our customers to use a Google account.

That is, we would like APIs that we can build our own user interfaces over 
top to help our users through this typically frustrating setup experience.

BTW, free SNI-based HTTPS termination was a huge step along the way towards 
this vision - thanks for that!

Thanks,
j

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Is it still possible to signup for a premier account or is this only for Fortune 500 companies?

[Jason Collins]

Isn't it just this form? https://cloud.google.com/support/signup
j

On Friday, 11 July 2014 13:03:29 UTC-6, Kaan Soral wrote:

 Yeah, I should have read the link before :)

 Why don't you open a ticket from the google cloud support programs then, 
 it seems they would be obliged to respond


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: Appengine ecosystem?

[Jason Collins]

This remains my favourite go-to spot for interesting discussions about 
appropriate ways to leverage the overall Cloud Platform and interesting 
architecture discussions.

Maybe I'm just a dinosaur

j

On Sunday, 6 July 2014 20:57:26 UTC-6, Jeff Schnitzer wrote:

 On Thu, Jul 3, 2014 at 11:15 PM, Richard Watson richard...@gmail.com 
 javascript: wrote:

 I suppose if we get really uppity we could create an unofficial forum 
 that isn't as restricted.


 What's wrong with this one, the one we are posting to right now?

 Jeff


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Will ignoring the google recommended setting for a cname (www) come back and bite me?

[Jason Collins]

App Engine now supports naked domains - without using Google Apps.

Go here to set it up:

  https://console.developers.google.com/project/apps~*[your-app-id]*
/appengine/settings/domains

You'll get a set of actual A and  IPs to use on your domain's root 
record.

On Tuesday, 17 June 2014 15:53:16 UTC-6, david...@gmail.com wrote:

 Hi, 
 I've pointed my naked domain, .com, to my appspot address - I am not 
 using Google Apps.
 I then went on to add www..com and the google wizard told me to:







 * 1. Create a resource record for the .com http://.com domain 
 using values as shown:- Type: CNAME- Alias: www- Canonical name: 
 ghs.googlehosted.com http://ghs.googlehosted.comNow the problem was that 
 my registrar dns console gave an undescribed error when I tried to use 
 ghs.googlehosted.com http://ghs.googlehosted.com.So, being completely 
 ignorant about these things I created a CNAME that just points www..com 
 http://www..com to .com http://.com and it seems to have 
 worked - I can now access my app via my custom domain and via the www 
 subdomain.But now I am worried - what problems have I created for the 
 future by not using ghs.googlehosted.com 
 http://ghs.googlehosted.com?David*


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Least Expensive Way to Delete Entities

[Jason Collins]

+1, this feature is really needed.

For us, truncation by namespace works best, though truncation of an entire 
Kind would also be a useful dimension. 

We're happy to soft-truncate the data - i.e., simply have it marked for 
deletion so that we can piggy-back on some other background process that is 
inexpensive for Google and have it finally be deleted days/weeks later.

j

On Thursday, 5 June 2014 05:10:37 UTC-6, SP wrote:

 Hi Ikai, 

 In traditional RDBMS like Oracle, MySQL, there is truncate command to 
 truncate a table. It is very light in term of resource usage.

 Do we have similar feature to truncate a kind in datastore? Such a feature 
 would make housekeeping old data a lot more efficient. The lack of such 
 feature would be a major bottleneck that prevents us from moving our 
 serious business apps to App Engine. 

 Regrards, 
 SP

 On Wednesday, 21 December 2011 02:39:37 UTC+8, Ikai Lan (Google) wrote:

 Guess I forgot about the EntitiesByKind index:

 http://code.google.com/appengine/articles/storage_breakdown.html

 Yes, the correct number should be 22 datastore writes.

 --
 Ikai Lan 
 Developer Programs Engineer, Google App Engine
 plus.ikailan.com | twitter.com/ikai



 On Tue, Dec 20, 2011 at 4:58 AM, Brian Quinlan bqui...@google.com 
 wrote:

 I think that Ikai made an off-by-one-error: it requires 22 writes
 assuming that every property is indexed :-)

 Check out Entity Delete (per entity) at:

 http://code.google.com/appengine/docs/billing.html#Billable_Resource_Unit_Cost

 Cheers,
 Brian

 On Tue, Dec 20, 2011 at 11:47 PM, supercobra super...@gmail.com wrote:
  Could you explain (again?) how a deletion of an entity w/ 10- property 
 is 21
  writes?
 
 
  On Mon, Dec 19, 2011 at 3:18 PM, Ikai Lan (Google) ika...@google.com
  wrote:
 
  One thing you can do is making this really slow. The free quota 
 actually
  gives you quite a bit of quota (if your app is relatively small) so 
 if you
  can stretch out your job you can purge a relatively small datastore 
 at low
  cost.
 
  I suspect what makes this expensive is datastore ops, not instance 
 hours.
  Deleting an entity with 10 properties is 21 writes! There's not a lot 
 we can
  do there ...
 
  --
  Ikai Lan
  Developer Programs Engineer, Google App Engine
  plus.ikailan.com | twitter.com/ikai
 
 
 
  On Mon, Dec 19, 2011 at 9:50 AM, Patrick Twohig
  pat...@namazustudios.com wrote:
 
  Hi,
 
  I have a bit of old data that I want to clean out.  I tried to write 
 a
  fanout operation to clean out some of the old data, but it's 
 actually manged
  to run out my budget in a few hours.  I'm curious if there would be 
 a way to
  delete huge amounts of old data that isn't needed anymore.  Is there 
 a tool
  or something that i can use to efficiently purge entity groups from 
 my
  application?  Any help would be greatly appreciated.
 
  Thanks,
  Patrick.
 
  --
  Patrick H. Twohig.
 
  Namazu Studios
  P.O. Box 34161
  San Diego, CA 92163-4161
 
  --
  You received this message because you are subscribed to the Google 
 Groups
  Google App Engine group.
  To post to this group, send email to google-a...@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengi...@googlegroups.com.
  For more options, visit this group at
  http://groups.google.com/group/google-appengine?hl=en.
 
 
  --
  You received this message because you are subscribed to the Google 
 Groups
  Google App Engine group.
  To post to this group, send email to google-a...@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengi...@googlegroups.com.
  For more options, visit this group at
  http://groups.google.com/group/google-appengine?hl=en.
 
 
  --
  You received this message because you are subscribed to the Google 
 Groups
  Google App Engine group.
  To post to this group, send email to google-a...@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengi...@googlegroups.com.
  For more options, visit this group at
  http://groups.google.com/group/google-appengine?hl=en.

 --
 You received this message because you are subscribed to the Google 
 Groups Google App Engine group.
 To post to this group, send email to google-a...@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengi...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.




-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Cloud SQL vs Datastore NDB

[Jason Collins]

You could check out Battle of the 
Backends: https://www.youtube.com/watch?v=rRoy6I4gKWU

j

On Monday, 21 April 2014 13:11:56 UTC-6, coto wrote:

 Despite both have amazing features, these two database systems have 
 different strengths. I would like to hear your opinion about, when is 
 recommended to use Cloud SQL and when Datastore through NDB.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] How do I wind-down my Google Apps that are setup solely for Custom Domains?

[Jason Collins]

With the new Custom Domain hookup tool in the Cloud Console (thank you, 
thank you!), I would like to wind-down all the various Google Apps that we 
have set up over the years (and are paying for).

However, I still want the App Engine mappings to continue.

Is there a right way to do this? Do I request for the Apps accounts to be 
cancelled? Do I remove my credit card from the Apps account and just let 
them go stale/inactive?

I definitely cannot have interruption of service, so the mappings need to 
remain regardless of approach.

Thanks for any info,
j

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] New Custom Domain hookup feature - what if our customer owns/verifies the domain?

[Jason Collins]

We sell an App Engine based product that we allow customers to point their 
own domain to (i.e., our product is served on their domain).

With the new Custom Domain mapper tool in Cloud Console, I'm trying to get 
my head around how this will work.

To kick off the process, we need to first start in our own Cloud Console 
and enter the domain. This kicks over to Webmaster Tools for domain 
verification. However, we don't want to grab control of the customer's 
domain (and/or they may already have Webmaster Tools setup for their 
domain).

So, is there anyway to allow the customer to verify the domain such that we 
can use the new Cloud Console tool to associate a host of that domain to 
our App Engine product?

Thanks,
j

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: Announcing a credit for App Engine applications with new custom domains

[Jason Collins]

One of our book keepers was just through my office showing me dozens of 
$4.16 charges on our credit card - one for each of the custom domains that 
we have mapped on behalf of our customers - all single-user Google Apps 
accounts _solely_ for the purpose of mapping a domain to an App Engine 
application.

I guess now I can get her to fill out dozens of forms to get a $50 rebate - 
I'm not even sure how that flows to us, and presumably we need to fill 
these out annually.

Oh no, wait - we don't pass these criteria: No person may request this 
credit for more than 3 domains, Only one credit may be granted per Google 
Apps for Business domain account, or Google App Engine application

What a massive PITA.
j

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Adding domain to google app engine

[Jason Collins]

Nope - that's the way it works.

Crazy, eh?

j

On Sunday, 9 March 2014 19:18:51 UTC-6, Matthew Cohen wrote:

 I'm writing a little app on Google App Engine.  I would like to point a 
 domain I own to this application.  I have owned the domain for years and 
 it's been parked on another host.

 I think I am missing something,though.

 When I go to add the domain to my google app engine account, it forces me 
 to create a Google Apps account?  As in gmail, google drive, etc?  A bunch 
 of stuff I already have and don't need another one.

 I dont know if I've done something wrong or I'm not understanding, but I 
 don't see why it's forcing me to create a Google Apps account when all I 
 want to do is point a domain I already own to my Google App Engine 
 application.

 If I'm doing something incorrect, please let me know.  If not, what is the 
 reason for forcing me to create a Google Apps account when I am not 
 interested in getting another gmail or google drive account?

 It feels like this should be a simple process and I am just missing 
 something completely obvious.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Add My own domain to appengine application

[Jason Collins]

Oh, and when you get fed up with that, please go star 
this: https://code.google.com/p/googleappengine/issues/detail?id=8528
j

On Thursday, 30 January 2014 02:21:22 UTC-6, Vinny P wrote:

 You need to map your Apps account to the App Engine application ID. Go to 
 the Applications Settings page on the admin console, scroll down to Domain 
 Setup ( http://imgur.com/y4jxxlv ) and click the Add Domain button. From 
 there, type in the domain you want to use and follow the instructions that 
 follow. 
   
 -
 -Vinny P
 Technology  Media Advisor
 Chicago, IL

 App Engine Code Samples: http://www.learntogoogleit.com
   
  


 On Wed, Jan 29, 2014 at 8:22 AM, Dinakar Sandeep 
 sunde...@gmail.comjavascript:
  wrote:

 I am trying to add my own domain www.example.com . In Appengine 
 www.example.appspot.com is the domain . I want to change to 
 www.example.com . I signed up for google apps after that I added my 
 domain to apps but still I am not getting the www.example.com




  



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Request was aborted after waiting too long to attempt to service your request

[Jason Collins]

Requests that hit your application are put into a queue waiting for an 
instance to become available. If those requests become 10-seconds old, they 
are cancelled (error to client).

The App Engine Scheduler may or may not attempt to spin up a new instance 
under this pressure (rules are black box and change from time to time). If 
it is able to spin up an instance, it will send the request to the cold 
(no warm up request) instance, though there is much debate (e.g., 
https://code.google.com/p/googleappengine/issues/detail?id=7865) about 
whether or not this should occur, especially for Java apps.

The only way to avoid these errors are to allocate Min Idle Instances, 
which keeps resident instances around to help in this specific case. These 
resident instances themselves can be somewhat confusing because they are 
really only used to serve when a dynamic instance cannot be found - i.e., 
basically under this pressure situation you've outlined. You'll find, in a 
more smoothly loaded case, that the resident instances can be very 
under-utilized as they are basically idle waiting for spikes. This too is 
the subject of much debate.

j

On Saturday, 25 January 2014 15:42:07 UTC-6, o...@haitov.com wrote:

 Hi,
 I'm experiencing a very strange issue with my app, App engine suddenly 
 can't deal with incoming requests and shows this message in the logs 
 without any error code and at information level ! 
 Sometimes all the incoming requests during a few minutes can return this 
 message and then everything goes back to normal !
 This is really bad as it is a front end that serves a mobile app and that 
 means that during this time the app is fully non functional :-(((

 @Google 
 Why isn't this message at error or warning level ? 


 Attached is a screenshot of the log message





-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Have to type in www before custom domain name

[Jason Collins]

You won't be able to access your application on the naked domain (at 
least not currently). You'll always need to redirect the client browser 
from the naked domain to a particular hostname (e.g., www), and the 
hostname is used to route requests to your App Engine application (note 
that you can use a wildcard to route all hostnames to your app). Google 
Apps does have a built-in mechanism to perform this redirect.

It's hard to find stuff in the Apps console and they keep moving things 
around. App Engine things (domains, SSL, host mappings) tend to get buried. 
Try going to the following url to see the naked domain redirection 
configuration (after auth'ing to the appropriate domain at 
http://www.google.com/a/[your-domain.com]):

  https://admin.google.com/AdminHome?pli=1fral=1#Domains:

(I just clicked the Domains button to get to the above link in case that 
direct link doesn't work.)

j

On Thursday, 5 December 2013 20:33:38 UTC-6, br...@lazytex.com wrote:

 Hello!  I'm a complete beginner to Google App Engine, so this might be a 
 very stupid question.  

 I just created my first app on GAE (Python).  I went to the Dashboard for 
 this app and went to Add New Domain.  I signed up for Google Apps, 
 registering for a custom domain name through Google.  Then I went to Add 
 New URL for the app and put in www before the custom domain name (you had 
 to put something before the domain name).  Now I can access my app at 
 www.customdomain.com, but not at customdomain.com.  

 I'm thinking that the problem might be related to steps 6-9 on this page: 
 https://developers.google.com/appengine/docs/domain.  But in step 6, I 
 can't find the Service Settings button.  This guide seems to correspond 
 to an older version of GAE.

 Any help is greatly appreciated!  Thank you very much!


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Is there any reason that urlfetch DELETE method does not support a payload?

[Jason Collins]

When attempting to perform a urlfetch DELETE call, it seems the payload is 
explicitly NOT added. From urlfetch.py:

  if payload and method in (POST, PUT, PATCH):
request.set_payload(payload)

This makes it impossible to work with an API provided by our vendor.

Is there any particular reason that a payload cannot be added to a DELETE 
urlfetch?

j

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Is there any reason that urlfetch DELETE method does not support a payload?

[Jason Collins]

Turns out, happily, that our vendor has an alternate technique based on 
POST.

So this can be ignored. 

Took me a while to figure out what was going on; a logging.warning sure 
would have helped...

j

On Wednesday, 27 November 2013 14:55:14 UTC-6, Jason Collins wrote:

 When attempting to perform a urlfetch DELETE call, it seems the payload is 
 explicitly NOT added. From urlfetch.py:

   if payload and method in (POST, PUT, PATCH):
 request.set_payload(payload)

 This makes it impossible to work with an API provided by our vendor.

 Is there any particular reason that a payload cannot be added to a DELETE 
 urlfetch?

 j


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] Re: Google-branded 503 error screens?

[Jason Collins]

https://code.google.com/p/googleappengine/issues/detail?id=10216


On Thursday, 31 October 2013 10:07:17 UTC-6, Vinny P wrote:

 On Wed, Oct 30, 2013 at 7:40 PM, Jason Collins 
 jason.a...@gmail.comjavascript:
  wrote:

 I don't want _any_ branding. I realize that I won't be able to project my 
 application branding through the layers, but I want Google to replace their 
 branded error screen with a brandless, black-on-white-text error screen.



 If you file an issue, I'd star it as well. This issue hit me too. With 
 that said, I doubt Google will do anything - as Tim said, the request might 
 error out before even hitting GAE servers.

   
 -
 -Vinny P
 Technology  Media Advisor
 Chicago, IL

 App Engine Code Samples: http://www.learntogoogleit.com
   


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Google-branded 503 error screens?

[Jason Collins]

We are getting reports of our customers seeing Google-branded 503 error 
screens (attached).

This is a bad thing and needs to be corrected. Usually there is a 
non-branded black text on white background error screen for truly 
catastrophic issues in the serving infrastructure. Where is the 
Google-branding coming from?

Is anyone else seeing this?

j

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.
attachment: image001.png

Re: [google-appengine] Google-branded 503 error screens?

[Jason Collins]

Good to know it's resolved, but the Google-branding is disconcerting. We 
whitelabel our products (i.e., our branding cannot show, only our 
customers') and go to great lengths for our customers to remove any of our 
branding from escaping down the pipe. 

Google needs to do the same.

j

On Wednesday, 30 October 2013 11:29:37 UTC-6, barryhunter wrote:

 This was posted in the Google Page Speed Service forum. 

 Apprently a temporally connectivity issue. 

 Should now be resolved. 


 On 30 October 2013 17:23, Jason Collins jason.a...@gmail.comjavascript:
  wrote:

 We are getting reports of our customers seeing Google-branded 503 error 
 screens (attached).

 This is a bad thing and needs to be corrected. Usually there is a 
 non-branded black text on white background error screen for truly 
 catastrophic issues in the serving infrastructure. Where is the 
 Google-branding coming from?

 Is anyone else seeing this?

 j

 -- 
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to google-appengi...@googlegroups.com javascript:.
 To post to this group, send email to 
 google-a...@googlegroups.comjavascript:
 .
 Visit this group at http://groups.google.com/group/google-appengine.
 For more options, visit https://groups.google.com/groups/opt_out.




-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Google-branded 503 error screens?

[Jason Collins]

TIm, the point is that I don't want _any_ branding. I realize that I won't 
be able to project my application branding through the layers, but I want 
Google to replace their branded error screen with a brandless, 
black-on-white-text error screen.

j

On Wednesday, 30 October 2013 18:02:23 UTC-6, timh wrote:

 You need to consider all the layers involved here.

 If the errors occur in infrastructure before it even gets to appengine 
 (and custom static error pages) then how can you brand the error page,
 and something has to be shown.

 T

 On Thursday, October 31, 2013 1:23:16 AM UTC+8, Jason Collins wrote:

 We are getting reports of our customers seeing Google-branded 503 error 
 screens (attached).

 This is a bad thing and needs to be corrected. Usually there is a 
 non-branded black text on white background error screen for truly 
 catastrophic issues in the serving infrastructure. Where is the 
 Google-branding coming from?

 Is anyone else seeing this?

 j



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Google-branded 503 error screens?

[Jason Collins]

(and I realize now that you might have been replying to Kaan.)


On Wednesday, 30 October 2013 18:40:08 UTC-6, Jason Collins wrote:

 TIm, the point is that I don't want _any_ branding. I realize that I won't 
 be able to project my application branding through the layers, but I want 
 Google to replace their branded error screen with a brandless, 
 black-on-white-text error screen.

 j

 On Wednesday, 30 October 2013 18:02:23 UTC-6, timh wrote:

 You need to consider all the layers involved here.

 If the errors occur in infrastructure before it even gets to appengine 
 (and custom static error pages) then how can you brand the error page,
 and something has to be shown.

 T

 On Thursday, October 31, 2013 1:23:16 AM UTC+8, Jason Collins wrote:

 We are getting reports of our customers seeing Google-branded 503 error 
 screens (attached).

 This is a bad thing and needs to be corrected. Usually there is a 
 non-branded black text on white background error screen for truly 
 catastrophic issues in the serving infrastructure. Where is the 
 Google-branding coming from?

 Is anyone else seeing this?

 j



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Any plan to allow a quick way to drop an entire search index?

[Jason Collins]

We have a bunch of search indexes full of orphaned documents. Rather than 
deleting the orphaned documents one at a time, is there any plan to provide 
a function to drop an entire search index (and associated documents)?

E.g., 

  from google.appengine.api import search

  search.*delete_index*('my-dead-index', namespace='')


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] how reduce memory usage on mapreduce controller callback

[Jason Collins]

You guys should 
star https://code.google.com/p/googleappengine/issues/detail?id=9610

In the past, I've found surprising amount of memory use when working with 
NDB, especially when using repeated properties. In my experiments, it was 
very easy to blow up instances with a seemingly small number of entities. I 
think it's wrapped up in protobuf deserialization which is low enough down 
the stack that no one seems to have much appetite to touch.

j


On Monday, 14 October 2013 04:14:05 UTC-6, Jason Galea wrote:

 Hi Moises,

 you can find all the details here.. 
 https://developers.google.com/appengine/docs/python/ndb/cache

 The In-Context Cache

 The in-context cache persists only for the duration of a single incoming 
 HTTP request and is visible only to the code that handles that request. 
 It's fast; this cache lives in memory. When an NDB function writes to the 
 Datastore, it also writes to the in-context cache. When an NDB function 
 reads an entity, it checks the in-context cache first. If the entity is 
 found there, no Datastore interaction takes place.

 My take-away is that the in-context cache is handy when different parts of 
 your code are calling get() on the same entities and would certainly make 
 things faster, but comes with the trade-off that all entities you touch are 
 staying in memory until the request completes, even if you don't need them 
 any more.

 With queries you're going to be loading all entities regardless so 
 disabling the in-context cache alone likely won't help much. If you do a 
 keys_only query, though, and get() each entity in turn, then disabling the 
 in-context cache should reduce memory usage (assuming that some/all of the 
 memory used by previous entities is able to be re-used..). Once again, 
 though, you'll likely be sacrificing speed for less memory usage.

 This is mostly based on how I believe the different parts would/should 
 work, I have no hard evidence..

 Jason





 On Mon, Oct 14, 2013 at 5:36 PM, Moises Belchin 
 moises...@gmail.comjavascript:
  wrote:

 Hi Jason,

 Thanks for the detailed answer. I'm very surprised that no one else is 
 talking about these issues.

 I'm using ndb and my appstats are off. I could see a incredible great 
 improvement in my app when I turned off the stats. So, I recommend people 
 only use stats for testing or debug.

 As you mentioned I'll try projection queries.

 Could you explain something more about NDB in-memory cache?

 Thanks again.




 Saludos.
 Moisés Belchín.


 2013/10/12 Jason Galea ja...@lecstor.com javascript:

 Hi Moises,

 we're currently trying to deal with this issue too. Not in mapreduce, 
 just regular handlers.

 same here - Python 2.7 and F2 instances with 256MB

 Early on I found that fetching 1000 entities and looping through them to 
 update a property would blow the instance. Reducing this to say 100 fixed 
 the issue. (than when create another task do do the next 100)

 After handling this request - as far as I understand this isn't so bad 
 unless you're blowing it on every request and starting a new instance is 
 detrimental.

 While handling this request - this concerns us most as the request 
 does not complete and breaks stuff.. and we see far too many of them.

 I've spent more than a little time trying to work out what is causing 
 the blowouts but as far as I've been able to work out, memory usage and 
 what causes it is near impossible (or just very, very hard).

 Are you using NDB? if so..
 - you could try disabling the in-memory cache. As I see it, even though 
 you only access one entity at a time, NDB's in memory cache will store them 
 all until the request is completed.
 - you could try projection queries if you don't need the complete object 
 (or possibly even if you do). Projection queries get their data from an 
 index and the entities returned cannot be put() so I assume they are not 
 cached at all. We're trialling some fixes with these atm.

 ** If anyone knows any of this is incorrect, please let me know.. 

 I'm actually surprised there is not more discussion of these issues from 
 what we have experienced so maybe we're doing something fundamentally 
 wrong, but I don't believe so.

 oh, is appstats turned on? I believe the most noticeable improvement 
 we've seen was when we turned it off..

 regards,

 Jason






 On Fri, Oct 11, 2013 at 5:51 PM, Moises Belchin 
 moises...@gmail.comjavascript:
  wrote:

 Hi Vinny,

 Thanks for the tips, but actually I'm not loading a file. I'm only 
 using mapreduce lib for read all the entities for one of my kinds, work 
 with them (I only read some properties to compose the csv line format) and 
 then I write to CSV file on cloud storage using mapreduce FileOutputWriter.

 Any idea why I'm getting this Criticals memory errors?

 Thanks all again.


 Saludos.
 Moisés Belchín.
  

 2013/10/10 Vinny P vinn...@gmail.com javascript:

 On Thu, Oct 10, 2013 at 5:47 AM, Moises Belchin 
 moises...@gmail.comjavascript:
  wrote:

 

Re: [google-appengine] how reduce memory usage on mapreduce controller callback

[Jason Collins]

I dug up some old research I had done which found the entities that should 
be smaller than 10kB were lugging around almost 100kB in-memory due to the 
in-memory protobuf representation.

On Monday, 14 October 2013 07:58:46 UTC-6, Jason Collins wrote:

 You guys should star 
 https://code.google.com/p/googleappengine/issues/detail?id=9610

 In the past, I've found surprising amount of memory use when working with 
 NDB, especially when using repeated properties. In my experiments, it was 
 very easy to blow up instances with a seemingly small number of entities. I 
 think it's wrapped up in protobuf deserialization which is low enough down 
 the stack that no one seems to have much appetite to touch.

 j



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: Unable to map domain to my appengine application

[Jason Collins]

Please star: https://code.google.com/p/googleappengine/issues/detail?id=8528


On Tuesday, 24 September 2013 07:44:52 UTC-6, Gubbi wrote:

 Hi,

 I already have a couple of domains and alias domains mapped to my app. I 
 wanted to map a new alias-domain to my 
 application today, but I keep getting the following error message: 
 We are unable to process your request at this time. Please try again 
 later. (Error #1000)

 I have been trying this for several hours now. What does this error 
 message mean? How do I resolve it?

 Any pointers would help.

 Thanks,
 Vinuth.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] datastore stats not updated for 4 days

[Jason Collins]

Still flying blind. No update since Sep 18.

j

On Tuesday, 24 September 2013 17:10:11 UTC-6, PK wrote:

 +1

 6 days for my key app, similar situation with Usage History, last time 
 of a daily charge entry in on 9/18...

 PK
 http://www.gae123.com

 On September 24, 2013 at 4:04:07 PM, Nicolas G 
 (nicol...@gmail.comjavascript:) 
 wrote:

 Same for me

 Nicolas



 On Mon, Sep 23, 2013 at 1:55 PM, timh zute...@gmail.com javascript:wrote:

 Howdy

 Normally stats are only a couple of days old, but I am up to 4+ days and 
 no updated datastore stats.
 Anyone else seeing this ?

 T

 -- 
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to google-appengi...@googlegroups.com javascript:.
 To post to this group, send email to 
 google-a...@googlegroups.comjavascript:
 .
 Visit this group at http://groups.google.com/group/google-appengine.
 For more options, visit https://groups.google.com/groups/opt_out.


  -- 
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To unsubscribe from this group and stop receiving emails from it, send an 
 email to google-appengi...@googlegroups.com javascript:.
 To post to this group, send email to google-a...@googlegroups.comjavascript:
 .
 Visit this group at http://groups.google.com/group/google-appengine.
 For more options, visit https://groups.google.com/groups/opt_out.



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [google-appengine] Re: Lots of warmup requests

[Jason Collins]

I get a 403 when trying to access Issue 8004 - anyone else?
j

On Monday, 23 September 2013 10:41:05 UTC-6, Mos wrote:

 This is the bug that always pops up like in these days:
 https://code.google.com/p/googleappengine/issues/detail?id=8004 


 On Mon, Sep 23, 2013 at 6:36 PM, Mos mos...@googlemail.com 
 javascript:wrote:

 Same with my application: AppID: krisen-talk

 I happens from time to time and makes GAE for a lot of low traffic sites 
 unusable. 
 In past it could take days to weeks until Google resolves the problem.
 I hope they can fix this issue 


 On Mon, Sep 23, 2013 at 5:49 PM, timh zute...@gmail.com javascript:wrote:

 I am seeing odd things with instances at the moment.  


 Again low traffic site - (which has been fine for ages) at the moment 
 after a request 6 instances pop up, then get shut down immediately 
 then a couple more are started  , then they shut down within a minute 
 and a few more instacnes are started

 This isn't something I have been experiencing at all over the last few 
 weeks - and earlier today.


  -- 
 You received this message because you are subscribed to the Google 
 Groups Google App Engine group.
 To unsubscribe from this group and stop receiving emails from it, send 
 an email to google-appengi...@googlegroups.com javascript:.
 To post to this group, send email to 
 google-a...@googlegroups.comjavascript:
 .
 Visit this group at http://groups.google.com/group/google-appengine.
 For more options, visit https://groups.google.com/groups/opt_out.





-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: GAE with major problems currently

[Jason Collins]

A problem was encountered with the process that handled this request
causing it to exit. This is likely to cause a new process to be used
for the next request to your application. (Error code 202)

We've been seeing tens of thousands of these for a few days now. Don't know 
what's causing it.

j

On Monday, 23 September 2013 13:17:28 UTC-6, Cesium wrote:

 Pear shaped for sure.
 David

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.

[google-appengine] Re: datastore stats not updated for 4 days

[Jason Collins]

Yup  - 5 days ago for us. Our Billing (Usage History) is also 5 days 
behind. Probably related to the 1.8.5 rollout.

j

On Monday, 23 September 2013 05:55:19 UTC-6, timh wrote:

 Howdy

 Normally stats are only a couple of days old, but I am up to 4+ days and 
 no updated datastore stats.
 Anyone else seeing this ?

 T


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/groups/opt_out.