Re: [google-appengine] Re: Cron tasks not uploaded using Google Cloud Tools for Eclipse

[Brian de Alwis]

Jason, 

This will be fixed in the next release of Cloud Tools for Eclipse.

In the meantime you should be able to use the `appcfg.sh` command [1] to upload 
your cron.xml (`appcfg.sh help update_cron`) and `queue.xml` (`appcfg.sh help 
update_queues`).  You'll need to use the `-A` argument to specify your Project 
ID if you don't have an `` element in your appengine-web.xml.

Brian.

[1] appcfg.sh is from the App Engine SDK for Java 
  and not 
included in the Google Cloud SDK.  But it's a simple wrapper around a jar that 
is included in the Google Cloud SDK:

$ java -cp 
$GOOGLE_CLOUD_SDK_HOME/platform/google_appengine/google/appengine/tools/java/lib/appengine-tools-api.jar
 com.google.appengine.tools.admin.AppCfg [args]


> On 31-Mar-2017, at 9:18 PM, Jason  wrote:
> 
> What if you are not using a Maven-based project? I created a non-maven 
> project and it is not uploading my queue.xml or cron.xml either.
> 
> On Wednesday, March 15, 2017 at 3:23:46 AM UTC-7, Nicola Spreafico wrote:
> If you're using a Maven-based project, by default implementation of the Maven 
> Plugin only the app.yaml file (the application itself) is deployed.
> 
> If you need to deploy the cron, queue and index as well, you need to 
> configure the deployables configuration with all the files
> Please see this issue 
>  where I 
> posted a configuration example.
> 
> With the default configuration of the Maven plugin, your configuration will 
> be something like this:
> 
>  target/appengine-staging/app.yaml
>  target/appengine-staging/cron.yaml
>  target/appengine-staging/queue.yaml
>  target/appengine-staging/index.yaml
> 
> 
> Il giorno martedì 14 marzo 2017 22:35:19 UTC+1, Daniel Garrido ha scritto:
> Hi all,
> 
> I have tried to create a cron task using cron.xml file:
> 
> 
> 
>   
> /save
> daily summary job
> every 2 minutes
>   
> 
> 
> When I deploy my app to app engine using Google Cloud Tools for Eclipse, the 
> cron task is not being created (it doesn't appear in the console).
> 
> I tested the same example using the old Eclipse plugin and it worked.
> 
> It seems that the cron.xml file is not being uploaded. Documentation 
> (https://cloud.google.com/appengine/docs/standard/java/config/cron 
> ) says 
> that I have to upload cron tasks using appcfg. This is automatically 
> performed by the eclipse plugin.
> 
> Finally, I used appcfg and the cron task was created.
> 
> I am wondering if I can do the same using the new Google Cloud Tools in an 
> automatic way.
> 
> Best regards,
> Daniel.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to google-appengine+unsubscr...@googlegroups.com 
> .
> To post to this group, send email to google-appengine@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/google-appengine 
> .
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/google-appengine/735dac15-28f2-4b86-893f-6c9f103768d1%40googlegroups.com
>  
> .
> For more options, visit https://groups.google.com/d/optout 
> .

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/50C97C9D-CC66-4274-8A59-AD3C048979C7%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Cron tasks not uploaded using Google Cloud Tools for Eclipse

[Jason]

What if you are not using a Maven-based project? I created a non-maven 
project and it is not uploading my queue.xml or cron.xml either.

On Wednesday, March 15, 2017 at 3:23:46 AM UTC-7, Nicola Spreafico wrote:
>
> If you're using a Maven-based project, by default implementation of the 
> Maven Plugin only the app.yaml file (the application itself) is deployed.
>
> If you need to deploy the *cron*, *queue *and *index *as well, you need 
> to configure the deployables configuration with all the files
> Please see this issue 
>  where 
> I posted a configuration example.
>
> With the default configuration of the Maven plugin, your configuration 
> will be something like this:
>
> 
>  target/appengine-staging/app.yaml
>  target/appengine-staging/cron.yaml
>  target/appengine-staging/queue.yaml
>  target/appengine-staging/index.yaml
> 
>
>
> Il giorno martedì 14 marzo 2017 22:35:19 UTC+1, Daniel Garrido ha scritto:
>>
>> Hi all,
>>
>> I have tried to create a cron task using cron.xml file:
>>
>> 
>> 
>>   
>> /save
>> daily summary job
>> every 2 minutes
>>   
>> 
>>
>> When I deploy my app to app engine using Google Cloud Tools for Eclipse, 
>> the cron task is not being created (it doesn't appear in the console).
>>
>> I tested the same example using the old Eclipse plugin and it worked.
>>
>> It seems that the cron.xml file is not being uploaded. Documentation (
>> https://cloud.google.com/appengine/docs/standard/java/config/cron) says 
>> that I have to upload cron tasks using appcfg. This is automatically 
>> performed by the eclipse plugin.
>>
>> Finally, I used appcfg and the cron task was created.
>>
>> I am wondering if I can do the same using the new Google Cloud Tools in 
>> an automatic way.
>>
>> Best regards,
>> Daniel.
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/735dac15-28f2-4b86-893f-6c9f103768d1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Google App Engine NodeJS log severity

['Adam (Cloud Platform Support)' via Google App Engine]

I've also been unable to get winston-gae working. It hasn't been updated in 
over 2 years and is not an official Google library, so for now it seems 
it's in need of a maintainer.

The standard winston  logger does 
work, but only outputs to stdout and as such you don't get proper severity 
in the Logs Viewer.

On Friday, March 31, 2017 at 11:03:20 AM UTC-4, Marco Galassi wrote:
>
>
> I am using nodejs on App Engine and I would like to have the ability to 
> log my application in an easy manner.
> Possibily, I would like to be able to log as easy as in python standard 
> environment 
> :
>
> import logging
>> import webapp2
>> class MainPage(webapp2.RequestHandler):
>>   def get(self):
>> logging.debug('This is a debug message')
>> logging.info('This is an info message')
>> logging.warning('This is a warning message')
>> logging.error('This is an error message')
>> logging.critical('This is a critical message')
>> try:
>>   raise ValueError('This is a sample value error.')
>> except ValueError:
>>   logging.exception('A example exception log.')
>> self.response.out.write('Logging example.') 
>
>  
>
> app = webapp2.WSGIApplication([ 
>
> ('/', MainPage)
>> ], debug=True)
>
>
> This is very useful since it shows the logs with the corresponding logging 
> severity in the Stackdriver Logging console.
> Unfortunately, it doesn't look that something like this is possible on 
> NodeJS.
>
> I have tried to follow what the documentation says 
> 
>  
> on logging on NodeJS, which says that I can use winston-gae 
> , but it doesn't work at all.
>
> Any help is very appreciated.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/9189dabe-b09a-4e24-bc7a-be68f0fb1142%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Mail API Quotas

[P K]

(End of quarter, and revisiting all our favorite topics today :-) )

1. I remind everybody that I have a use case that justifies why this needs 
to be a native GCP offering (please read and star this 
, glad it made it through 
the tracker transition.)
2. As I have mentioned before, Amazon in AWS has a native e-mail service 
since 2015 called SES ---I use it with another 
AWS customer I consult with. At this point you cannot be top tier Cloud 
Platform without such an offering.
3. Google has proven with gmail and the early GAE offering that they do 
know how to do e-mail at scale. My highest volume GAE app has been 
grandfathered with the old quotas and I am super happy with its track 
record on the e-mail front.

Between, the three points above I just cannot see how Google will be able 
to justify for much longer not including a better SES like service in their 
offering. It is just unfortunate that they could be leading instead of 
following in that space given their early head start with GAE e-mail APIs 
and experience.



On Friday, March 31, 2017 at 4:24:04 AM UTC-7, Bookingforce CTO wrote:
>
> Hello there,
>  
>   being CTO of a company that is betting on Google Infrastructure when I 
> found out about the limitation to 100 recipients, at first I thought "ah, I 
> have to increase that quota", then, after asking (twice) to support team, I 
> felt like "what's wrong with GAE ?" 
>
> The only fact that more than 100 emails is referred as "high volume" is 
> almost disturbing... we are using GAE to build cloud applications that 
> scale up using multiple instances but still we can't contact our customers 
> via email if they are more than 100 ? If you want to prevent spam, bill 
> each email, we may be willing to pay as much as we pay for other services. 
>
> If you just don't want to offer a serious service about email, you should 
> think to discontinue the Mail API, like this it makes no sense.
>
> Cheers,
>   Luca 
>
> On Wednesday, October 26, 2016 at 7:49:05 PM UTC+2, Nick (Cloud Platform 
> Support) wrote:
>>
>> The Public Issue Tracker thread brought our attention has gotten tracked 
>> now, thanks to PK. We'll update it with any progress.
>>
>> Now, as much as I'm sure we all appreciate Jeff's kind offer, I don't 
>> think it will be necessary to pay him $1000, since Cloud Platform Community 
>> Support are here to do a similar job at no cost. As said in other threads 
>> on this topic, if anybody at any time has issues migrating from any service 
>> to any other, has questions about design patterns, has questions about the 
>> use of our APIs or how to refactor their code, etc., this forum is the 
>> perfect place for such higher-level discussion and we would be happy to 
>> discuss in depth. 
>>
>> We never want users to feel left alone, and asking a question once has 
>> the benefit that all other users in a similar situation can read and learn 
>> from the thread. This doesn't fit under the umbrella of "platform issues / 
>> feature requests" (Public Issue Tracker) or "a specific error / stack 
>> trace" (Stack Overflow), so it would be fine to post such threads in this 
>> forum.
>>
>> As an additional, final comment, I'll say that we do pay close attention 
>> to the requests for services / service improvements that users bring 
>> forward, whether here or in the Public Issue Tracker. We've taken the 
>> feedback from many users on the Mail API and bulk sending and while we 
>> don't promise any concrete action given this feedback, know that we have 
>> taken it into account and we want to thank you for bringing up how you view 
>> things, what you'd like to see, etc.
>>
>> Cheers,
>>
>> Nick
>> Cloud Platform Community Support
>>
>> On Tuesday, October 18, 2016 at 10:42:26 PM UTC-4, Jeff Schnitzer wrote:
>>>
>>> Since this thread keeps coming up… I’ll make you all an offer: For $1k 
>>> I’ll migrate your GAE app to Sendgrid, Mailgun, or whatever other email 
>>> service you want. Assuming your code isn’t spaghetti, it will probably take 
>>> me an hour. It should take you about the same or less.
>>>
>>> This is sooo much kvetching about something that is so trivial to fix.
>>>
>>> Jeff
>>>
>>> On Tue, Oct 18, 2016 at 9:27 AM, PK  wrote:
>>>
 +1 for Joshua’s comment. 

 Nick, wrt "We always encourage users to make use of the Public Issue 
 Tracker “ the issue is issue 10560 
  stuck 
 in state “New” for 2.5 years like 100s maybe thousands other issues in the 
 public tracker.

 PK
 p...@gae123.com




 On Oct 18, 2016, at 8:50 AM, Joshua Smith  wrote:

 Nick,

 I know you are stuck in the apologist position, and there’s nothing you 
 can do about it, but come on…

 It is positively absurd that GAE doesn’t include any bulk 

[google-appengine] Re: Ongoing App Engine Commitment?

[P K]

This question/concern keeps popping up every few months for years now. 
Being a very early GAE user, I have my opinions and history based analysis 
that I have posted before. Recent signals, having attended Next as well, 
make me more optimistic than usual.

Today I want to add a new perspective: The question really is how much risk 
do you want to take? Recently the US stock market listed Snap. When you get 
listed in the US you are required to provide the investors an S-1 that 
contains all the risks of investing in this offering. Open the Snap S-1 here 

 
and search for Google and read these paragraphs. Bottom line, Snap says "we 
are alive as long as Google Cloud is alive". Then look at this Quora 
thread:  https://www.quora.com/How-does-Snapchat-use-Google-App-Engine Bottom 
line between those two, for now Snap, a US public company, is alive as long 
as GAE Standard is alive.

So, I conclude that Google will be committed to GAE Std. for a while. These 
are the kind of signals to look for; "formal statements" are good for few 
days, and we have seen many of those over the years, but can come and go as 
fast as an e-mail can be written. Does anybody else know of other such "too 
big to fail" successes built on GAE Std? The more of those the more 
confident I would be... 

Another signal of course is where Google puts its investment dollars. In 
the early 2010s timeframe Google puts all its (few at the time) eggs in 
GAE. The marketplace did not reward this approach so Google increased its 
Cloud investment and shifted all of it to build a better AWS because this 
is what the volume enterprise and startup market seemed to want. However 
GAE survived and remains a competitive advantage and differentiator for 
GCP, so I hope Google delivers and puts back more investment to GAE now.

PK
www.gae123.com

PS If the two GAEs merge downstream I am fine as long as I do not have to 
revisit (ie modify/test etc) code in production that has been working for 
years. The more of that I would have to do the more negative I would become 
of such a development.

On Friday, March 31, 2017 at 5:39:29 AM UTC-7, Tim Consolazio wrote:
>
> I've been noticing some things:
>
> - App Engine support shunted over to Stack Overflow (so...a non-Google 
> community resource).
> - Mail API pretty much deprecated (no more quota expansions, 100 email 
> limit). 
> - Deprecation of Channel API (sure we have Firebase, but still...what's 
> the commitment to that?)
> - No Java 8 (and 7 was a long time in coming). 
> - Gradual deprecation of local SDK in favor of remote/virtual dev (which 
> to me is unacceptable, I depend on that local SDK). 
>
> I frequently recommend App Engine (and I've built quite a few apps on it). 
> I've been a rogue evangelist of the platform for years (since it was 
> released in fact), and have been contracted by Google three times to build 
> applications using it. So I'm not a hater in any way. 
>
> But I can't look at these events (particularly the Stack Overflow thing) 
> and ignore the impression that App Engine is beginning to feel more like a 
> side project rather than a commercial offering with full resource 
> commitment by the vendor. 
>
> Curious about a formal statement of commitment here, and a complete 
> roadmap of what we can reasonably expect over say, the next two years. 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/49fdae08-8fcf-4f38-a9fc-d013b8fcd439%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Sudden import errors in unmodified Python 2.7 applications since the past few hours.

['Nick (Cloud Platform Support)' via Google App Engine]

@Richard specifically,

Do you think you could post the details of this patch you apply after each 
upgrade? This sounds like a feature request in the making.

On Thursday, March 30, 2017 at 3:56:03 AM UTC-4, Richard Cheesmar wrote:
>
> Hi, Nick, I've been got the same evening/morning 29th 30th March. Some 
> continuous errors all of the the ImportError: No module named pwd kind.
>
> I get this with the dev environment everytime I upgrade to a new version 
> of the cloud software, but that is solved by white listing pwd in one of 
> the local Google Cloud setting files, can't remember which at the moment.
>
> Never had this on the live environment though.
>
> On Tuesday, March 28, 2017 at 12:45:22 PM UTC+3, rrk wrote:
>>
>> Hi,
>>
>> Anyone else facing sudden ImportError issues with previously working 
>> applications in Python 2.7 runtime of Google App Engine?
>>
>> One such error is given below, which happens when *requests *package is 
>> trying to import *netrc*. This code was working fine until today. 
>>
>> Traceback (most recent call last):
>>
>>   File 
>> "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/runtime/wsgi.py",
>>  
>> line 240, in Handle
>>
>> handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
>>
>>   File 
>> "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/runtime/wsgi.py",
>>  
>> line 299, in _LoadHandler
>>
>> handler, path, err = LoadObject(self._handler)
>>
>>   File 
>> "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/runtime/wsgi.py",
>>  
>> line 85, in LoadObject
>>
>> obj = __import__(path[0])
>>
>>   File 
>> "/base/data/home/apps/s~app-id-removed/p31-b.391085029481971487/x.py",
>>  
>> line 36, in 
>>
>> from yy import *
>>
>>   File "/base/data/home/apps/s~app-id-removed/p31-b.391085029481971487/
>> yy.py", line 32, in 
>>
>> import requests
>>
>>   File 
>> "/base/data/home/apps/s~app-id-removed/p31-b.391085029481971487/requests/__init__.py",
>>  
>> line 52, in 
>>
>> from . import utils
>>
>>   File 
>> "/base/data/home/apps/s~app-id-removed/p31-b.391085029481971487/requests/utils.py",
>>  
>> line 19, in 
>>
>> from netrc import netrc, NetrcParseError
>>
>>   File 
>> "/base/data/home/runtimes/python27_experiment/python27_dist/lib/python2.7/netrc.py",
>>  
>> line 7, in 
>>
>> import pwd
>>
>> ImportError: No module named pwd
>>
>>
>> I have the same application deployed for other customers within their 
>> accounts and also in my account, but this error seems to have occurred only 
>> for one deployment.
>>
>> Why is this happening? Does this have something to do with the 
>> python27_experiment version, which I believe has been in use since over 
>> a month (based on error traces on the internet with this version), but this 
>> ImportError seems to have got triggered only today? May be my client's 
>> application has been included within the experimental version?
>>
>> Thanks and regards,
>> Raj
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/8c0ffdbc-84e6-4b86-ab29-c816f44b884d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Sudden import errors in unmodified Python 2.7 applications since the past few hours.

['Nick (Cloud Platform Support)' via Google App Engine]

Hey Folks,
 
The root cause of this issue has been identified and we're in the process 
of rolling out a fix. Please check in to this thread and let us know if 
this issue persists, and email me your project ID at pay...@google.com if 
so.

Cheers,

Nick
Cloud Platform Community Support

On Thursday, March 30, 2017 at 3:56:03 AM UTC-4, Richard Cheesmar wrote:
>
> Hi, Nick, I've been got the same evening/morning 29th 30th March. Some 
> continuous errors all of the the ImportError: No module named pwd kind.
>
> I get this with the dev environment everytime I upgrade to a new version 
> of the cloud software, but that is solved by white listing pwd in one of 
> the local Google Cloud setting files, can't remember which at the moment.
>
> Never had this on the live environment though.
>
> On Tuesday, March 28, 2017 at 12:45:22 PM UTC+3, rrk wrote:
>>
>> Hi,
>>
>> Anyone else facing sudden ImportError issues with previously working 
>> applications in Python 2.7 runtime of Google App Engine?
>>
>> One such error is given below, which happens when *requests *package is 
>> trying to import *netrc*. This code was working fine until today. 
>>
>> Traceback (most recent call last):
>>
>>   File 
>> "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/runtime/wsgi.py",
>>  
>> line 240, in Handle
>>
>> handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
>>
>>   File 
>> "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/runtime/wsgi.py",
>>  
>> line 299, in _LoadHandler
>>
>> handler, path, err = LoadObject(self._handler)
>>
>>   File 
>> "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/runtime/wsgi.py",
>>  
>> line 85, in LoadObject
>>
>> obj = __import__(path[0])
>>
>>   File 
>> "/base/data/home/apps/s~app-id-removed/p31-b.391085029481971487/x.py",
>>  
>> line 36, in 
>>
>> from yy import *
>>
>>   File "/base/data/home/apps/s~app-id-removed/p31-b.391085029481971487/
>> yy.py", line 32, in 
>>
>> import requests
>>
>>   File 
>> "/base/data/home/apps/s~app-id-removed/p31-b.391085029481971487/requests/__init__.py",
>>  
>> line 52, in 
>>
>> from . import utils
>>
>>   File 
>> "/base/data/home/apps/s~app-id-removed/p31-b.391085029481971487/requests/utils.py",
>>  
>> line 19, in 
>>
>> from netrc import netrc, NetrcParseError
>>
>>   File 
>> "/base/data/home/runtimes/python27_experiment/python27_dist/lib/python2.7/netrc.py",
>>  
>> line 7, in 
>>
>> import pwd
>>
>> ImportError: No module named pwd
>>
>>
>> I have the same application deployed for other customers within their 
>> accounts and also in my account, but this error seems to have occurred only 
>> for one deployment.
>>
>> Why is this happening? Does this have something to do with the 
>> python27_experiment version, which I believe has been in use since over 
>> a month (based on error traces on the internet with this version), but this 
>> ImportError seems to have got triggered only today? May be my client's 
>> application has been included within the experimental version?
>>
>> Thanks and regards,
>> Raj
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/85b701df-2316-4532-bab7-1ad9bf9846fb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Deployment Errors

[P K]

You might get more attention from JetBrains itself on this.

I use pycharm too and enjoy all the functionality and productivity gains of 
this mostly awesome IDE, but when deploy time comes I do it from the 
command line and believe me it does not impact my productivity at all. I 
suggest you look into that instead of spending too much energy to get this 
corner case resolved.

Just my 2c :-)

PK
www.gae123.com

On Friday, March 31, 2017 at 3:17:09 AM UTC-7, Richard Cheesmar wrote:
>
> I'm using Pycharm to deploy my App, normally work no problem, but today 
> all I keep getting is the following error:
>
> 01:12 PM Uploading 51 files and blobs.
> 01:13 PM Rolling back the update.
> Error 500: --- begin server output ---
>
>
> Server Error (500)
>
> A server error has occurred.
> --- end server output ---
>
> Process finished with exit code 1
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/d48ebc21-6830-486c-8c79-b0128090b403%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Error occurs after 30-40 consecutive urlfetch

['Nick (Cloud Platform Support)' via Google App Engine]

Hey Chris,

>From reading the thread history, it seems that there are potentially two 
issues brought up by kyeongwook Ma and Dustin Oprea. Although both see an 
expired context, one is about UrlFetch calls and the other is about 
Datastore API calls. The mechanism underlying potentially both of them, if 
they *are* the same issue, was discussed as being the duration of the 
request handler. 

In your case, we haven't got from your post any details other than that you 
see this error (exactly as posted by OP? or a similar error?) quite often 
(how often?) and that your request handler runs under a second. This last 
fact certainly rules out a request deadline root cause, but leaves a lot 
undefined. 

Given that this thread seems to have exhausted itself and in the first 
place was not suited to this forum (which is meant for general discussion, 
not technical support or issue tracking, for which we have Stack Overflow 
 and the Public Issue Tracker 
, respectively), I'd suggest posting to the 
Public Issue Tracker with as many technical details about what's occurring 
as possible. This is the surest way for us to be able to begin looking into 
the root cause. 

Finally, rest assured that we monitor our Stack Overflow tags and the 
Public Issue Tracker thoroughly - this isn't a redirect into the void but 
rather simply a question of directing you to the forum among those we 
monitor where we ourselves can best provide assistance.

Cheers

Nick
Cloud Platform Community Support



On Friday, March 31, 2017 at 1:44:10 PM UTC-4, Chris Olsen wrote:
>
> I get this error quite often as well. I am using standard environment, and 
> as was mentioned, the request itself is not a long one, but under a second.
>
> On Tuesday, August 9, 2016 at 8:29:30 AM UTC-6, kyeongwook Ma wrote:
>>
>> Hi GAE support,
>>
>> I am a developer using the GAE with Golang and gin.
>>
>> I included urlfetch in the service to use a third party service (for 
>> example, AWS SNS push service).
>>
>> The urlfetch is successful for the first few times, but when we send 
>> 30-40 consecutive urlfetch, it keeps on giving an error.
>>
>> The specific error message is:
>>
>> *WARNING  2016-08-09 13:29:56,592 urlfetch_stub.py:540] Stripped 
>> prohibited headers from URLFetch request: ['Content-Length']*
>> *ERROR: context expired before API call urlfetch/Fetch completed*
>>
>> *RequestError: send request failed*
>> *caused by: Post https://sns.us-west-1.amazonaws.com/ 
>> : Call error 3: invalid security 
>> ticket (context expired)*
>>
>> We also tested urlfetch with other third party services, and we still get 
>> the same error messages as above.
>>  
>> Is there anybody that can help us with this situation?
>>
>> Thank you very much for your help.
>>
>> Best,
>> Rio
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/320cff12-0635-4fe9-ad55-ecfa886eaca2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Strange behaviour related to java Endpoints V2

['Nick (Cloud Platform Support)' via Google App Engine]

Hey Þórir,

This is perfect as an issue report for the Public Issue Tracker 
, where we work with users to notify 
product teams and engineering of issues on the platform. Feel free to post 
this there and we'll be able to take a look. I can't say I've seen anything 
like this before. Be sure to include all the details given in this report. 
We may ask for you to reply to a private email we'll send if we require 
copies of the logs or your code.

Cheers,

Nick
Cloud Platform Community Support

On Friday, March 31, 2017 at 8:07:18 AM UTC-4, Þórir Gunnarsson wrote:
>
> Hi
>
> Just wanted to check with the group if anyone is experiencing unusual 
> behaviour related to endpoints v2 on AppEngine. We are using the AppEngine 
> standard environment.
>
> This morning we start getting reports from users where our Android client 
> is crashing a lot. Upon investigation we discover that yesterday 
> (30.03.2017) at 16:36 UTC we start getting unusual entries in the log that 
> look like this:
> /_ah/api/appendpoint/v8/users//dashboarddata
> and a HTTP 302 response according to the logs (this then results in a 
> crash in the Android client, which is a different story)
> On a regular day and most of the time this call from the client looks like 
> this.
> /_ah/spi/is.app.services.dashboard.DashboardEndpointV9.getDashboardData
>
> The strange thing here is the format of the log entry which is consistent 
> with the format of log entries after upgrading to endpoints v2. 
>
> We see the same exact Android client making this call and once in a while 
> it seems to get the new type of response (the endpoints v2 type log entry 
> and a 302).
>
> We did not deploy anything at this time yesterday and we were certainly 
> not using endpoints v2 at that time.
>
> After some testing we have now improved the situation by upgrading our 
> backend to using endpoints v2 (version 2.0.5). 
> I say improved because now most log entries look like they are coming from 
> endpoints v2 but once in a while we get the old style log entries and now 
> the old style log entries return 302 (according to server logs)
> The Android client no longer crashes but doesn't receive a response when 
> this happens.
>
> To further investigate what was happening we plugged in a http proxy to 
> see exactly what was going on.
> On a successful call there is a GET call to the server with a proper HTTP 
> 200 OK response
> On a failure there is a GET call to the server with a HTTP 404 NOT FOUND 
> response and an "unsupportedProtocol" error.
> We don't see any difference between the two calls. The server logs tell us 
> that both are served by the same instance on the backend.
>
>  
> So is anyone experiencing something similar?
> Should we pin this on some Google update or possibly some misconfiguration 
> on our end?
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/13dd22dd-d06e-49b2-bfe3-2cd79237ee95%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Deployment Errors

['Jordan (Cloud Platform Support)' via Google App Engine]

Google Groups is meant for general product discussions and not for 
technical support.

I recommend you file a public Issue Tracker  
report with more 
information. First run 'gcloud components update', then copy the actual 
full command output you used to deploy with and add the "--verbosity=debug" 
option. Also supply all of the information produced by 'gcloud info' in the 
issue tracker report, as this will better help troubleshoot the issue.  

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/badce6c9-e182-4dfb-84b8-815927981a18%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Error occurs after 30-40 consecutive urlfetch

[Chris Olsen]

I get this error quite often as well. I am using standard environment, and 
as was mentioned, the request itself is not a long one, but under a second.

On Tuesday, August 9, 2016 at 8:29:30 AM UTC-6, kyeongwook Ma wrote:
>
> Hi GAE support,
>
> I am a developer using the GAE with Golang and gin.
>
> I included urlfetch in the service to use a third party service (for 
> example, AWS SNS push service).
>
> The urlfetch is successful for the first few times, but when we send 30-40 
> consecutive urlfetch, it keeps on giving an error.
>
> The specific error message is:
>
> *WARNING  2016-08-09 13:29:56,592 urlfetch_stub.py:540] Stripped 
> prohibited headers from URLFetch request: ['Content-Length']*
> *ERROR: context expired before API call urlfetch/Fetch completed*
>
> *RequestError: send request failed*
> *caused by: Post https://sns.us-west-1.amazonaws.com/ 
> : Call error 3: invalid security 
> ticket (context expired)*
>
> We also tested urlfetch with other third party services, and we still get 
> the same error messages as above.
>  
> Is there anybody that can help us with this situation?
>
> Thank you very much for your help.
>
> Best,
> Rio
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/997154d9-9730-4d1c-9a96-ba47a10f184c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Mail API Quotas

['Kim Lewandowski' via Google App Engine]

Hi everyone,

We know that almost every developer has a need to send email and that the
GAE Mail API hasn't been given the love it deservers. I am trying to make
it better for everyone on Google Cloud. Please feel free to email me
directly with your requirements and a bit more about your use cases.

On Fri, Mar 31, 2017 at 7:44 AM, Kaan Soral  wrote:

> I don't know whether I missed it, but we never learned why "exactly" Mail
> was deprecated
>
> I wonder whether it was from a conflict of interest too, for example, it
> bothered me that my emails were marked as "Promotional", while they were
> actually "Social", in Gmail
>
> Anyway, that's why I also added the part about "investing some effort", as
> you pointed out, there are things that can be improved
>
> Fyi, I would love a metric based quota too, similar to AWS's solution to
> the issue, could even be more aggressive
>
> On Friday, March 31, 2017 at 5:38:16 PM UTC+3, Jeff Schnitzer wrote:
>>
>> How do you know that 80% of your emails sent with Google weren’t wasted?
>> GAE never provided deliverability statistics of any sort.
>>
>> On Fri, Mar 31, 2017 at 7:35 AM, Kaan Soral  wrote:
>>
>>> I'm with Luca here
>>>
>>> Using external services introduces new points of failures and introduces
>>> new, edge complications that only arise when you get into things
>>>
>>> For example, on 10/18/16 I replied to this issue, afterwards, after the
>>> mourning period, I ended up using AWS, the email routine I used for GAE was
>>> too fast for AWS, I completely forgot about their throughput limitations,
>>> 80% of my emails were wasted, a bad start to the Halloween period
>>>
>>> On this thread, we end up repeating our views of the issue, however, to
>>> repeat my own view, I want GAE to be as independent as possible, instead of
>>> spawning, supporting other cloud services, why not provide all basic
>>> services in-house?
>>>
>>> For example, a Video/Live-Streaming solution from Google would be pretty
>>> good, I like Zencoder, yet integration was a challenge, it's tooo expensive
>>> for a startup
>>>
>>> Anyway, my point is, in-house is always better than external, survey us,
>>> ask us what we need, invest some effort into building new things, I'm sure
>>> Email will be on top of that list (I hope Video/Streaming is a runner-up,
>>> in the age that we live in, we really need affordable and agile video
>>> solutions too)
>>>
>>>
>>> On Friday, March 31, 2017 at 2:24:04 PM UTC+3, Bookingforce CTO wrote:

 Hello there,

   being CTO of a company that is betting on Google Infrastructure when
 I found out about the limitation to 100 recipients, at first I thought "ah,
 I have to increase that quota", then, after asking (twice) to support team,
 I felt like "what's wrong with GAE ?"

 The only fact that more than 100 emails is referred as "high volume" is
 almost disturbing... we are using GAE to build cloud applications that
 scale up using multiple instances but still we can't contact our customers
 via email if they are more than 100 ? If you want to prevent spam, bill
 each email, we may be willing to pay as much as we pay for other services.

 If you just don't want to offer a serious service about email, you
 should think to discontinue the Mail API, like this it makes no sense.

 Cheers,
   Luca

 On Wednesday, October 26, 2016 at 7:49:05 PM UTC+2, Nick (Cloud
 Platform Support) wrote:
>
> The Public Issue Tracker thread brought our attention has gotten
> tracked now, thanks to PK. We'll update it with any progress.
>
> Now, as much as I'm sure we all appreciate Jeff's kind offer, I don't
> think it will be necessary to pay him $1000, since Cloud Platform 
> Community
> Support are here to do a similar job at no cost. As said in other threads
> on this topic, if anybody at any time has issues migrating from any 
> service
> to any other, has questions about design patterns, has questions about the
> use of our APIs or how to refactor their code, etc., this forum is the
> perfect place for such higher-level discussion and we would be happy to
> discuss in depth.
>
> We never want users to feel left alone, and asking a question once has
> the benefit that all other users in a similar situation can read and learn
> from the thread. This doesn't fit under the umbrella of "platform issues /
> feature requests" (Public Issue Tracker) or "a specific error / stack
> trace" (Stack Overflow), so it would be fine to post such threads in this
> forum.
>
> As an additional, final comment, I'll say that we do pay close
> attention to the requests for services / service improvements that users
> bring forward, whether here or in the Public Issue Tracker. We've taken 
> the
> feedback from many users on the Mail API and bulk sending and while we

[google-appengine] Re: Ongoing App Engine Commitment?

[Tim Consolazio]

These notices are from 2012?

On Friday, March 31, 2017 at 8:39:29 AM UTC-4, Tim Consolazio wrote:
>
> I've been noticing some things:
>
> - App Engine support shunted over to Stack Overflow (so...a non-Google 
> community resource).
> - Mail API pretty much deprecated (no more quota expansions, 100 email 
> limit). 
> - Deprecation of Channel API (sure we have Firebase, but still...what's 
> the commitment to that?)
> - No Java 8 (and 7 was a long time in coming). 
> - Gradual deprecation of local SDK in favor of remote/virtual dev (which 
> to me is unacceptable, I depend on that local SDK). 
>
> I frequently recommend App Engine (and I've built quite a few apps on it). 
> I've been a rogue evangelist of the platform for years (since it was 
> released in fact), and have been contracted by Google three times to build 
> applications using it. So I'm not a hater in any way. 
>
> But I can't look at these events (particularly the Stack Overflow thing) 
> and ignore the impression that App Engine is beginning to feel more like a 
> side project rather than a commercial offering with full resource 
> commitment by the vendor. 
>
> Curious about a formal statement of commitment here, and a complete 
> roadmap of what we can reasonably expect over say, the next two years. 
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/0108d02d-fe15-4693-bf7a-85e56f2c6410%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Ongoing App Engine Commitment?

[Jeff Schnitzer]

Not a Googler, but I’ve been around a while:

 * They moved support to stackoverflow in early 2012. It seems to be a
common practice. I have mixed feelings about it myself, but it’s a thing,
and definitely not a new thing.

 * The Mail API and Channel API have both been zombie products since before
2012. I agree that G should make it more explicit when features like this
have dark futures, but it wasn’t hard to figure out that you should avoid
these APIs even in 2012. The Channel API in particular.

 * The Java8 runtime on GAE standard is in alpha test. There have been
several public announcements here. I think there’s a link somewhere in this
forum for applying to the program…ah, here:
https://groups.google.com/forum/#!searchin/google-appengine/java$208%7Csort:date/google-appengine/Zn7X3D7o584/DrTp64eeAwAJ

Jeff


On Fri, Mar 31, 2017 at 5:24 AM, Tim Consolazio  wrote:

> I've been noticing some things:
>
> - App Engine support shunted over to Stack Overflow (so...a non-Google
> community resource).
> - Mail API pretty much deprecated (no more quota expansions, 100 email
> limit).
> - Deprecation of Channel API (sure we have Firebase, but still...what's
> the commitment to that?)
> - No Java 8 (and 7 was a long time in coming).
> - Gradual deprecation of local SDK in favor of remote/virtual dev (which
> to me is unacceptable, I depend on that local SDK).
>
> I frequently recommend App Engine (and I've built quite a few apps on it).
> I've been a rogue evangelist of the platform for years (since it was
> released in fact), and have been contracted by Google three times to build
> applications using it. So I'm not a hater in any way.
>
> But I can't look at these events (particularly the Stack Overflow thing)
> and ignore the impression that App Engine is beginning to feel more like a
> side project rather than a commercial offering with full resource
> commitment by the vendor.
>
> Curious about a formal statement of commitment here, and a complete
> roadmap of what we can reasonably expect over say, the next two years.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to google-appengine+unsubscr...@googlegroups.com.
> To post to this group, send email to google-appengine@googlegroups.com.
> Visit this group at https://groups.google.com/group/google-appengine.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/google-appengine/ffc4dd64-c827-4c5f-8765-
> b84182e65e90%40googlegroups.com
> 
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/CADK-0uh%3Dt94tRFWReOSW-xARR541mRKatewpu_H%3Darh_sxbWGw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Google App Engine NodeJS log severity

[Marco Galassi]

I am using nodejs on App Engine and I would like to have the ability to log 
my application in an easy manner.
Possibily, I would like to be able to log as easy as in python standard 
environment :

import logging
> import webapp2
> class MainPage(webapp2.RequestHandler):
>   def get(self):
> logging.debug('This is a debug message')
> logging.info('This is an info message')
> logging.warning('This is a warning message')
> logging.error('This is an error message')
> logging.critical('This is a critical message')
> try:
>   raise ValueError('This is a sample value error.')
> except ValueError:
>   logging.exception('A example exception log.')
> self.response.out.write('Logging example.') 

 

app = webapp2.WSGIApplication([ 

('/', MainPage)
> ], debug=True)


This is very useful since it shows the logs with the corresponding logging 
severity in the Stackdriver Logging console.
Unfortunately, it doesn't look that something like this is possible on 
NodeJS.

I have tried to follow what the documentation says 

 
on logging on NodeJS, which says that I can use winston-gae 
, but it doesn't work at all.

Any help is very appreciated.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/537db47a-55a8-4be2-88ca-18a64225f6ee%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Mail API Quotas

[Kaan Soral]

I don't know whether I missed it, but we never learned why "exactly" Mail 
was deprecated

I wonder whether it was from a conflict of interest too, for example, it 
bothered me that my emails were marked as "Promotional", while they were 
actually "Social", in Gmail

Anyway, that's why I also added the part about "investing some effort", as 
you pointed out, there are things that can be improved

Fyi, I would love a metric based quota too, similar to AWS's solution to 
the issue, could even be more aggressive

On Friday, March 31, 2017 at 5:38:16 PM UTC+3, Jeff Schnitzer wrote:
>
> How do you know that 80% of your emails sent with Google weren’t wasted? 
> GAE never provided deliverability statistics of any sort. 
>
> On Fri, Mar 31, 2017 at 7:35 AM, Kaan Soral  > wrote:
>
>> I'm with Luca here
>>
>> Using external services introduces new points of failures and introduces 
>> new, edge complications that only arise when you get into things
>>
>> For example, on 10/18/16 I replied to this issue, afterwards, after the 
>> mourning period, I ended up using AWS, the email routine I used for GAE was 
>> too fast for AWS, I completely forgot about their throughput limitations, 
>> 80% of my emails were wasted, a bad start to the Halloween period
>>
>> On this thread, we end up repeating our views of the issue, however, to 
>> repeat my own view, I want GAE to be as independent as possible, instead of 
>> spawning, supporting other cloud services, why not provide all basic 
>> services in-house?
>>
>> For example, a Video/Live-Streaming solution from Google would be pretty 
>> good, I like Zencoder, yet integration was a challenge, it's tooo expensive 
>> for a startup
>>
>> Anyway, my point is, in-house is always better than external, survey us, 
>> ask us what we need, invest some effort into building new things, I'm sure 
>> Email will be on top of that list (I hope Video/Streaming is a runner-up, 
>> in the age that we live in, we really need affordable and agile video 
>> solutions too)
>>
>>
>> On Friday, March 31, 2017 at 2:24:04 PM UTC+3, Bookingforce CTO wrote:
>>>
>>> Hello there,
>>>  
>>>   being CTO of a company that is betting on Google Infrastructure when I 
>>> found out about the limitation to 100 recipients, at first I thought "ah, I 
>>> have to increase that quota", then, after asking (twice) to support team, I 
>>> felt like "what's wrong with GAE ?" 
>>>
>>> The only fact that more than 100 emails is referred as "high volume" is 
>>> almost disturbing... we are using GAE to build cloud applications that 
>>> scale up using multiple instances but still we can't contact our customers 
>>> via email if they are more than 100 ? If you want to prevent spam, bill 
>>> each email, we may be willing to pay as much as we pay for other services. 
>>>
>>> If you just don't want to offer a serious service about email, you 
>>> should think to discontinue the Mail API, like this it makes no sense.
>>>
>>> Cheers,
>>>   Luca 
>>>
>>> On Wednesday, October 26, 2016 at 7:49:05 PM UTC+2, Nick (Cloud Platform 
>>> Support) wrote:

 The Public Issue Tracker thread brought our attention has gotten 
 tracked now, thanks to PK. We'll update it with any progress.

 Now, as much as I'm sure we all appreciate Jeff's kind offer, I don't 
 think it will be necessary to pay him $1000, since Cloud Platform 
 Community 
 Support are here to do a similar job at no cost. As said in other threads 
 on this topic, if anybody at any time has issues migrating from any 
 service 
 to any other, has questions about design patterns, has questions about the 
 use of our APIs or how to refactor their code, etc., this forum is the 
 perfect place for such higher-level discussion and we would be happy to 
 discuss in depth. 

 We never want users to feel left alone, and asking a question once has 
 the benefit that all other users in a similar situation can read and learn 
 from the thread. This doesn't fit under the umbrella of "platform issues / 
 feature requests" (Public Issue Tracker) or "a specific error / stack 
 trace" (Stack Overflow), so it would be fine to post such threads in this 
 forum.

 As an additional, final comment, I'll say that we do pay close 
 attention to the requests for services / service improvements that users 
 bring forward, whether here or in the Public Issue Tracker. We've taken 
 the 
 feedback from many users on the Mail API and bulk sending and while we 
 don't promise any concrete action given this feedback, know that we have 
 taken it into account and we want to thank you for bringing up how you 
 view 
 things, what you'd like to see, etc.

 Cheers,

 Nick
 Cloud Platform Community Support

 On Tuesday, October 18, 2016 at 10:42:26 PM UTC-4, Jeff Schnitzer wrote:
>
> Since this thread keeps coming up… I’ll make you 

Re: [google-appengine] Mail API Quotas

[Jeff Schnitzer]

How do you know that 80% of your emails sent with Google weren’t wasted?
GAE never provided deliverability statistics of any sort.

On Fri, Mar 31, 2017 at 7:35 AM, Kaan Soral  wrote:

> I'm with Luca here
>
> Using external services introduces new points of failures and introduces
> new, edge complications that only arise when you get into things
>
> For example, on 10/18/16 I replied to this issue, afterwards, after the
> mourning period, I ended up using AWS, the email routine I used for GAE was
> too fast for AWS, I completely forgot about their throughput limitations,
> 80% of my emails were wasted, a bad start to the Halloween period
>
> On this thread, we end up repeating our views of the issue, however, to
> repeat my own view, I want GAE to be as independent as possible, instead of
> spawning, supporting other cloud services, why not provide all basic
> services in-house?
>
> For example, a Video/Live-Streaming solution from Google would be pretty
> good, I like Zencoder, yet integration was a challenge, it's tooo expensive
> for a startup
>
> Anyway, my point is, in-house is always better than external, survey us,
> ask us what we need, invest some effort into building new things, I'm sure
> Email will be on top of that list (I hope Video/Streaming is a runner-up,
> in the age that we live in, we really need affordable and agile video
> solutions too)
>
>
> On Friday, March 31, 2017 at 2:24:04 PM UTC+3, Bookingforce CTO wrote:
>>
>> Hello there,
>>
>>   being CTO of a company that is betting on Google Infrastructure when I
>> found out about the limitation to 100 recipients, at first I thought "ah, I
>> have to increase that quota", then, after asking (twice) to support team, I
>> felt like "what's wrong with GAE ?"
>>
>> The only fact that more than 100 emails is referred as "high volume" is
>> almost disturbing... we are using GAE to build cloud applications that
>> scale up using multiple instances but still we can't contact our customers
>> via email if they are more than 100 ? If you want to prevent spam, bill
>> each email, we may be willing to pay as much as we pay for other services.
>>
>> If you just don't want to offer a serious service about email, you should
>> think to discontinue the Mail API, like this it makes no sense.
>>
>> Cheers,
>>   Luca
>>
>> On Wednesday, October 26, 2016 at 7:49:05 PM UTC+2, Nick (Cloud Platform
>> Support) wrote:
>>>
>>> The Public Issue Tracker thread brought our attention has gotten tracked
>>> now, thanks to PK. We'll update it with any progress.
>>>
>>> Now, as much as I'm sure we all appreciate Jeff's kind offer, I don't
>>> think it will be necessary to pay him $1000, since Cloud Platform Community
>>> Support are here to do a similar job at no cost. As said in other threads
>>> on this topic, if anybody at any time has issues migrating from any service
>>> to any other, has questions about design patterns, has questions about the
>>> use of our APIs or how to refactor their code, etc., this forum is the
>>> perfect place for such higher-level discussion and we would be happy to
>>> discuss in depth.
>>>
>>> We never want users to feel left alone, and asking a question once has
>>> the benefit that all other users in a similar situation can read and learn
>>> from the thread. This doesn't fit under the umbrella of "platform issues /
>>> feature requests" (Public Issue Tracker) or "a specific error / stack
>>> trace" (Stack Overflow), so it would be fine to post such threads in this
>>> forum.
>>>
>>> As an additional, final comment, I'll say that we do pay close attention
>>> to the requests for services / service improvements that users bring
>>> forward, whether here or in the Public Issue Tracker. We've taken the
>>> feedback from many users on the Mail API and bulk sending and while we
>>> don't promise any concrete action given this feedback, know that we have
>>> taken it into account and we want to thank you for bringing up how you view
>>> things, what you'd like to see, etc.
>>>
>>> Cheers,
>>>
>>> Nick
>>> Cloud Platform Community Support
>>>
>>> On Tuesday, October 18, 2016 at 10:42:26 PM UTC-4, Jeff Schnitzer wrote:

 Since this thread keeps coming up… I’ll make you all an offer: For $1k
 I’ll migrate your GAE app to Sendgrid, Mailgun, or whatever other email
 service you want. Assuming your code isn’t spaghetti, it will probably take
 me an hour. It should take you about the same or less.

 This is sooo much kvetching about something that is so trivial to fix.

 Jeff

 On Tue, Oct 18, 2016 at 9:27 AM, PK  wrote:

> +1 for Joshua’s comment.
>
> Nick, wrt "We always encourage users to make use of the Public Issue
> Tracker “ the issue is issue 10560
>  stuck
> in state “New” for 2.5 years like 100s maybe thousands other issues in the
> public tracker.
>

Re: [google-appengine] Mail API Quotas

[Joshua Smith]

Turning off the limit, and changing the pricing for usage that they are already 
tracking could not possibly be a “massive waste of engineering.” It should be 
trivial. Maybe 10 minutes was an exaggeration. 15 minutes?

And I would spend a penny per email when I need to send 1,000. Thankfully, I’ve 
been here long enough that I’m grandfathered in with a much higher quota.

Also, it isn’t always trivial to send through those other services when there 
are attachments or other complexities involved.

-Joshua

> On Mar 31, 2017, at 10:15 AM, Jeff Schnitzer  wrote:
> 
> Nobody who sends email at volume will spend a penny per email. For 1MM 
> messages per month that would be $10k. At Sendgrid that’s $500.
> 
> It’s trivial to enqueue a task that sends an email to any of the hundred 
> other commodity email services on the internet, most of which give you 
> deliverability reporting and fancy graphs and custom IP addresses and 
> whatnot. They run on razor-thin margins and more companies are getting out of 
> the business than into it (eg Mandrill, Messagebus). It would be a massive 
> waste of engineering.
> 
> Jeff
> 
> On Fri, Mar 31, 2017 at 6:56 AM, Joshua Smith  > wrote:
> The existing mail API makes sense for things like “ereporter” that 
> automatically sends the administrator a daily email summarizing unhandled 
> exceptions.
> 
> It also works fine for low-volume cases, like password authentication for 
> apps that only get a dozen or so new users a day.
> 
> It’s quite absurd that google doesn’t offer a high-volume email service, but 
> that doesn’t mean it shouldn’t at least offer a low-volume one.
> 
> And yes, billing a penny, or even a few cents, for every outbound email is an 
> obvious solution to the spam concern. I have no idea why they don’t just do 
> that. It’d probably take them like 10 minutes to implement.
> 
> -Joshua
> 
>> On Mar 31, 2017, at 5:44 AM, Bookingforce CTO > > wrote:
>> 
>> Hello there,
>>  
>>   being CTO of a company that is betting on Google Infrastructure when I 
>> found out about the limitation to 100 recipients, at first I thought "ah, I 
>> have to increase that quota", then, after asking (twice) to support team, I 
>> felt like "what's wrong with GAE ?" 
>> 
>> The only fact that more than 100 emails is referred as "high volume" is 
>> almost disturbing... we are using GAE to build cloud applications that scale 
>> up using multiple instances but still we can't contact our customers via 
>> email if they are more than 100 ? If you want to prevent spam, bill each 
>> email, we may be willing to pay as much as we pay for other services. 
>> 
>> If you just don't want to offer a serious service about email, you should 
>> think to discontinue the Mail API, like this it makes no sense.
>> 
>> Cheers,
>>   Luca 
>> 
>> On Wednesday, October 26, 2016 at 7:49:05 PM UTC+2, Nick (Cloud Platform 
>> Support) wrote:
>> The Public Issue Tracker thread brought our attention has gotten tracked 
>> now, thanks to PK. We'll update it with any progress.
>> 
>> Now, as much as I'm sure we all appreciate Jeff's kind offer, I don't think 
>> it will be necessary to pay him $1000, since Cloud Platform Community 
>> Support are here to do a similar job at no cost. As said in other threads on 
>> this topic, if anybody at any time has issues migrating from any service to 
>> any other, has questions about design patterns, has questions about the use 
>> of our APIs or how to refactor their code, etc., this forum is the perfect 
>> place for such higher-level discussion and we would be happy to discuss in 
>> depth. 
>> 
>> We never want users to feel left alone, and asking a question once has the 
>> benefit that all other users in a similar situation can read and learn from 
>> the thread. This doesn't fit under the umbrella of "platform issues / 
>> feature requests" (Public Issue Tracker) or "a specific error / stack trace" 
>> (Stack Overflow), so it would be fine to post such threads in this forum.
>> 
>> As an additional, final comment, I'll say that we do pay close attention to 
>> the requests for services / service improvements that users bring forward, 
>> whether here or in the Public Issue Tracker. We've taken the feedback from 
>> many users on the Mail API and bulk sending and while we don't promise any 
>> concrete action given this feedback, know that we have taken it into account 
>> and we want to thank you for bringing up how you view things, what you'd 
>> like to see, etc.
>> 
>> Cheers,
>> 
>> Nick
>> Cloud Platform Community Support
>> 
>> On Tuesday, October 18, 2016 at 10:42:26 PM UTC-4, Jeff Schnitzer wrote:
>> Since this thread keeps coming up… I’ll make you all an offer: For $1k I’ll 
>> migrate your GAE app to Sendgrid, Mailgun, or whatever other email service 
>> you want. Assuming your code isn’t spaghetti, it will 

Re: [google-appengine] Mail API Quotas

[Kaan Soral]

I'm with Luca here

Using external services introduces new points of failures and introduces 
new, edge complications that only arise when you get into things

For example, on 10/18/16 I replied to this issue, afterwards, after the 
mourning period, I ended up using AWS, the email routine I used for GAE was 
too fast for AWS, I completely forgot about their throughput limitations, 
80% of my emails were wasted, a bad start to the Halloween period

On this thread, we end up repeating our views of the issue, however, to 
repeat my own view, I want GAE to be as independent as possible, instead of 
spawning, supporting other cloud services, why not provide all basic 
services in-house?

For example, a Video/Live-Streaming solution from Google would be pretty 
good, I like Zencoder, yet integration was a challenge, it's tooo expensive 
for a startup

Anyway, my point is, in-house is always better than external, survey us, 
ask us what we need, invest some effort into building new things, I'm sure 
Email will be on top of that list (I hope Video/Streaming is a runner-up, 
in the age that we live in, we really need affordable and agile video 
solutions too)

On Friday, March 31, 2017 at 2:24:04 PM UTC+3, Bookingforce CTO wrote:
>
> Hello there,
>  
>   being CTO of a company that is betting on Google Infrastructure when I 
> found out about the limitation to 100 recipients, at first I thought "ah, I 
> have to increase that quota", then, after asking (twice) to support team, I 
> felt like "what's wrong with GAE ?" 
>
> The only fact that more than 100 emails is referred as "high volume" is 
> almost disturbing... we are using GAE to build cloud applications that 
> scale up using multiple instances but still we can't contact our customers 
> via email if they are more than 100 ? If you want to prevent spam, bill 
> each email, we may be willing to pay as much as we pay for other services. 
>
> If you just don't want to offer a serious service about email, you should 
> think to discontinue the Mail API, like this it makes no sense.
>
> Cheers,
>   Luca 
>
> On Wednesday, October 26, 2016 at 7:49:05 PM UTC+2, Nick (Cloud Platform 
> Support) wrote:
>>
>> The Public Issue Tracker thread brought our attention has gotten tracked 
>> now, thanks to PK. We'll update it with any progress.
>>
>> Now, as much as I'm sure we all appreciate Jeff's kind offer, I don't 
>> think it will be necessary to pay him $1000, since Cloud Platform Community 
>> Support are here to do a similar job at no cost. As said in other threads 
>> on this topic, if anybody at any time has issues migrating from any service 
>> to any other, has questions about design patterns, has questions about the 
>> use of our APIs or how to refactor their code, etc., this forum is the 
>> perfect place for such higher-level discussion and we would be happy to 
>> discuss in depth. 
>>
>> We never want users to feel left alone, and asking a question once has 
>> the benefit that all other users in a similar situation can read and learn 
>> from the thread. This doesn't fit under the umbrella of "platform issues / 
>> feature requests" (Public Issue Tracker) or "a specific error / stack 
>> trace" (Stack Overflow), so it would be fine to post such threads in this 
>> forum.
>>
>> As an additional, final comment, I'll say that we do pay close attention 
>> to the requests for services / service improvements that users bring 
>> forward, whether here or in the Public Issue Tracker. We've taken the 
>> feedback from many users on the Mail API and bulk sending and while we 
>> don't promise any concrete action given this feedback, know that we have 
>> taken it into account and we want to thank you for bringing up how you view 
>> things, what you'd like to see, etc.
>>
>> Cheers,
>>
>> Nick
>> Cloud Platform Community Support
>>
>> On Tuesday, October 18, 2016 at 10:42:26 PM UTC-4, Jeff Schnitzer wrote:
>>>
>>> Since this thread keeps coming up… I’ll make you all an offer: For $1k 
>>> I’ll migrate your GAE app to Sendgrid, Mailgun, or whatever other email 
>>> service you want. Assuming your code isn’t spaghetti, it will probably take 
>>> me an hour. It should take you about the same or less.
>>>
>>> This is sooo much kvetching about something that is so trivial to fix.
>>>
>>> Jeff
>>>
>>> On Tue, Oct 18, 2016 at 9:27 AM, PK  wrote:
>>>
 +1 for Joshua’s comment. 

 Nick, wrt "We always encourage users to make use of the Public Issue 
 Tracker “ the issue is issue 10560 
  stuck 
 in state “New” for 2.5 years like 100s maybe thousands other issues in the 
 public tracker.

 PK
 p...@gae123.com




 On Oct 18, 2016, at 8:50 AM, Joshua Smith  wrote:

 Nick,

 I know you are stuck in the apologist position, and there’s nothing you 
 can do about it, but come on…

 It is 

Re: [google-appengine] Re: Understanding java.util.ConcurrentModificationException: too much contention on these datastore entities. please try again.

[Jeff Schnitzer]

You want:

Mail mailToSend = getMail();
runInTransaction(() -> {
   enqueueSendMailTask(mailToSend);
   updateMailStatusInDatastore(mailToSend, SENT);
});

Email is fundamentally asynchronous; you don’t get a “real” response when
sending it. What you’re interpreting as a status code is just the fact that
some queueing system somewhere accepted it. Just put it in your task queue
and (as with all queues) make sure you don’t end up with a bunch of broken
tasks.

Emails go from queue to queue to queue to queue. Adding one more queue is
not going to hurt anything and the transaction will make your application a
lot more reliable.

Jeff

On Fri, Mar 31, 2017 at 12:33 AM, Louise Elmose Hedegaard <
louiseelm...@gmail.com> wrote:

> Hi Jeff,
>
> I am not sure I understand how putting the code in a transactional task
> helps me.
>
> The simplified code is:
>
> Mail mailToSend = getMail();
> SentStatus sentStatus = sendMailToExternalMailSystem(mailToSend);
> updateMailStatusInDatastore(mailToSend, sentStatus)
>
> The problem is that the datastore write "updateMailStatusInDatastore"
> might fail due to CME, and at this point the mail has already been sent to
> the external mail system.
> So even though I move this code to a transactional task, a failure in the
> datastore write will still lead me to a scenario where the mail has been
> sent, but the mail status has not been updated properly in my app.
>
> Thanks,
> -Louise
>
> Den torsdag den 30. marts 2017 kl. 20.30.45 UTC+2 skrev Jeff Schnitzer:
>>
>> There may be clock skew in the cluster; 15s is a lot but you can’t assume
>> that log entry timestamps are exact.
>>
>> You should send email by enqueueing a transactional task. Do not put
>> non-idempotent remote calls in your transactions (ideally, don’t put any
>> remote calls in transactions).
>>
>> For example, let’s say you want to email a receipt on purchase. Create a
>> deferred SendReceiptEmailTask that holds the purchase record key; it looks
>> up the record, formulates the email body, and makes the final call to the
>> email service. You can enlist this in your transaction that creates the
>> purchase record. This also has the benefit of moving all that work outside
>> of your transaction and shortening the critical section.
>>
>> There is a very small risk that tasks may execute twice, but it’s small
>> enough and harmless enough to ignore. I’ve sent millions of emails this way
>> with no complaints. And you don’t really have a choice anyway - I haven’t
>> found any email sending companies that allow you to specify an idempotency
>> key or some other way of guaranteeing once-and-only-once behavior.
>>
>> Cheers,
>> Jeff
>>
>>
>>
>> On Thu, Mar 30, 2017 at 3:44 AM, Louise Elmose Hedegaard <
>> louise...@gmail.com> wrote:
>>
>>> Hi Jeff,
>>>
>>> I have CME erratically - so I guess you are saying I should take a look
>>> at the look for the CME runs, and check whether the same EG's are accessed
>>> by other operations?
>>> It does seem that there might be a clash with other operations when the
>>> CME occurs.
>>> I am not sure about the timing though - there is a updateOrder operation
>>> which fails with CME at 14:59:46.684 and the send mail fails with CME
>>> at 15:01:06.583...
>>> When you say "Any change to that EG" it is the entire EG you mean, or
>>> only rows with the same id in the EG?
>>>
>>> I need to ensure that my transaction does not fail, as it will result in
>>> duplicate emails being sent if it does.
>>> There are other operations working on the same EG which comes randomly
>>> as they are caused by a webhook in another application.
>>> What is the best way to ensure that these webhook operations do not
>>> cause the send mail operation to get a CME?
>>> I can only think of very hacked and ugly solutions - e.g. to have a
>>> lock/switch which is on when the send mail operation is running, on only
>>> executing webhook operations when the lock/switch is off.
>>>
>>> Thanks,
>>> -Louise
>>>
>>> Den tirsdag den 28. marts 2017 kl. 22.11.15 UTC+2 skrev Jeff Schnitzer:

 When you load a key in a datastore transaction, that EG is enlisted in
 the transaction. Any change to that EG by any other process in the system
 will cause your commit to rollback with CME. Even if your transaction is
 "read-only”.

 When I said “linear” I mean that if you have a quiet datastore (no
 other activity) and you start a transaction and you perform a series of
 operations and you commit the transaction, you should not see CME.

 If you see CME erratically, then you have contention in your system.
 If you see CME consistently 100% of the time and you aren’t under heavy
 load, then you have a bug in your transaction logic.

 Jeff

 On Tue, Mar 28, 2017 at 6:29 AM, Louise Elmose Hedegaard <
 louise...@gmail.com> wrote:

> Hi,
>
> I stumbled upon an instance in the log where the sendNow list is empty
> - but still the operation fails 

Re: [google-appengine] Mail API Quotas

[Jeff Schnitzer]

Nobody who sends email at volume will spend a penny per email. For 1MM
messages per month that would be $10k. At Sendgrid that’s $500.

It’s trivial to enqueue a task that sends an email to any of the hundred
other commodity email services on the internet, most of which give you
deliverability reporting and fancy graphs and custom IP addresses and
whatnot. They run on razor-thin margins and more companies are getting out
of the business than into it (eg Mandrill, Messagebus). It would be a
massive waste of engineering.

Jeff

On Fri, Mar 31, 2017 at 6:56 AM, Joshua Smith 
wrote:

> The existing mail API makes sense for things like “ereporter” that
> automatically sends the administrator a daily email summarizing unhandled
> exceptions.
>
> It also works fine for low-volume cases, like password authentication for
> apps that only get a dozen or so new users a day.
>
> It’s quite absurd that google doesn’t offer a high-volume email service,
> but that doesn’t mean it shouldn’t at least offer a low-volume one.
>
> And yes, billing a penny, or even a few cents, for every outbound email is
> an obvious solution to the spam concern. I have no idea why they don’t just
> do that. It’d probably take them like 10 minutes to implement.
>
> -Joshua
>
> On Mar 31, 2017, at 5:44 AM, Bookingforce CTO  wrote:
>
> Hello there,
>
>   being CTO of a company that is betting on Google Infrastructure when I
> found out about the limitation to 100 recipients, at first I thought "ah, I
> have to increase that quota", then, after asking (twice) to support team, I
> felt like "what's wrong with GAE ?"
>
> The only fact that more than 100 emails is referred as "high volume" is
> almost disturbing... we are using GAE to build cloud applications that
> scale up using multiple instances but still we can't contact our customers
> via email if they are more than 100 ? If you want to prevent spam, bill
> each email, we may be willing to pay as much as we pay for other services.
>
> If you just don't want to offer a serious service about email, you should
> think to discontinue the Mail API, like this it makes no sense.
>
> Cheers,
>   Luca
>
> On Wednesday, October 26, 2016 at 7:49:05 PM UTC+2, Nick (Cloud Platform
> Support) wrote:
>>
>> The Public Issue Tracker thread brought our attention has gotten tracked
>> now, thanks to PK. We'll update it with any progress.
>>
>> Now, as much as I'm sure we all appreciate Jeff's kind offer, I don't
>> think it will be necessary to pay him $1000, since Cloud Platform Community
>> Support are here to do a similar job at no cost. As said in other threads
>> on this topic, if anybody at any time has issues migrating from any service
>> to any other, has questions about design patterns, has questions about the
>> use of our APIs or how to refactor their code, etc., this forum is the
>> perfect place for such higher-level discussion and we would be happy to
>> discuss in depth.
>>
>> We never want users to feel left alone, and asking a question once has
>> the benefit that all other users in a similar situation can read and learn
>> from the thread. This doesn't fit under the umbrella of "platform issues /
>> feature requests" (Public Issue Tracker) or "a specific error / stack
>> trace" (Stack Overflow), so it would be fine to post such threads in this
>> forum.
>>
>> As an additional, final comment, I'll say that we do pay close attention
>> to the requests for services / service improvements that users bring
>> forward, whether here or in the Public Issue Tracker. We've taken the
>> feedback from many users on the Mail API and bulk sending and while we
>> don't promise any concrete action given this feedback, know that we have
>> taken it into account and we want to thank you for bringing up how you view
>> things, what you'd like to see, etc.
>>
>> Cheers,
>>
>> Nick
>> Cloud Platform Community Support
>>
>> On Tuesday, October 18, 2016 at 10:42:26 PM UTC-4, Jeff Schnitzer wrote:
>>>
>>> Since this thread keeps coming up… I’ll make you all an offer: For $1k
>>> I’ll migrate your GAE app to Sendgrid, Mailgun, or whatever other email
>>> service you want. Assuming your code isn’t spaghetti, it will probably take
>>> me an hour. It should take you about the same or less.
>>>
>>> This is sooo much kvetching about something that is so trivial to fix.
>>>
>>> Jeff
>>>
>>> On Tue, Oct 18, 2016 at 9:27 AM, PK  wrote:
>>>
 +1 for Joshua’s comment.

 Nick, wrt "We always encourage users to make use of the Public Issue
 Tracker “ the issue is issue 10560
  stuck
 in state “New” for 2.5 years like 100s maybe thousands other issues in the
 public tracker.

 PK
 p...@gae123.com




 On Oct 18, 2016, at 8:50 AM, Joshua Smith  wrote:

 Nick,

 I know you are stuck in the apologist 

Re: [google-appengine] Mail API Quotas

[Joshua Smith]

The existing mail API makes sense for things like “ereporter” that 
automatically sends the administrator a daily email summarizing unhandled 
exceptions.

It also works fine for low-volume cases, like password authentication for apps 
that only get a dozen or so new users a day.

It’s quite absurd that google doesn’t offer a high-volume email service, but 
that doesn’t mean it shouldn’t at least offer a low-volume one.

And yes, billing a penny, or even a few cents, for every outbound email is an 
obvious solution to the spam concern. I have no idea why they don’t just do 
that. It’d probably take them like 10 minutes to implement.

-Joshua

> On Mar 31, 2017, at 5:44 AM, Bookingforce CTO  wrote:
> 
> Hello there,
>  
>   being CTO of a company that is betting on Google Infrastructure when I 
> found out about the limitation to 100 recipients, at first I thought "ah, I 
> have to increase that quota", then, after asking (twice) to support team, I 
> felt like "what's wrong with GAE ?" 
> 
> The only fact that more than 100 emails is referred as "high volume" is 
> almost disturbing... we are using GAE to build cloud applications that scale 
> up using multiple instances but still we can't contact our customers via 
> email if they are more than 100 ? If you want to prevent spam, bill each 
> email, we may be willing to pay as much as we pay for other services. 
> 
> If you just don't want to offer a serious service about email, you should 
> think to discontinue the Mail API, like this it makes no sense.
> 
> Cheers,
>   Luca 
> 
> On Wednesday, October 26, 2016 at 7:49:05 PM UTC+2, Nick (Cloud Platform 
> Support) wrote:
> The Public Issue Tracker thread brought our attention has gotten tracked now, 
> thanks to PK. We'll update it with any progress.
> 
> Now, as much as I'm sure we all appreciate Jeff's kind offer, I don't think 
> it will be necessary to pay him $1000, since Cloud Platform Community Support 
> are here to do a similar job at no cost. As said in other threads on this 
> topic, if anybody at any time has issues migrating from any service to any 
> other, has questions about design patterns, has questions about the use of 
> our APIs or how to refactor their code, etc., this forum is the perfect place 
> for such higher-level discussion and we would be happy to discuss in depth. 
> 
> We never want users to feel left alone, and asking a question once has the 
> benefit that all other users in a similar situation can read and learn from 
> the thread. This doesn't fit under the umbrella of "platform issues / feature 
> requests" (Public Issue Tracker) or "a specific error / stack trace" (Stack 
> Overflow), so it would be fine to post such threads in this forum.
> 
> As an additional, final comment, I'll say that we do pay close attention to 
> the requests for services / service improvements that users bring forward, 
> whether here or in the Public Issue Tracker. We've taken the feedback from 
> many users on the Mail API and bulk sending and while we don't promise any 
> concrete action given this feedback, know that we have taken it into account 
> and we want to thank you for bringing up how you view things, what you'd like 
> to see, etc.
> 
> Cheers,
> 
> Nick
> Cloud Platform Community Support
> 
> On Tuesday, October 18, 2016 at 10:42:26 PM UTC-4, Jeff Schnitzer wrote:
> Since this thread keeps coming up… I’ll make you all an offer: For $1k I’ll 
> migrate your GAE app to Sendgrid, Mailgun, or whatever other email service 
> you want. Assuming your code isn’t spaghetti, it will probably take me an 
> hour. It should take you about the same or less.
> 
> This is sooo much kvetching about something that is so trivial to fix.
> 
> Jeff
> 
> On Tue, Oct 18, 2016 at 9:27 AM, PK gae123.com > 
> wrote:
> +1 for Joshua’s comment. 
> 
> Nick, wrt "We always encourage users to make use of the Public Issue Tracker 
> “ the issue is issue 10560 
>  stuck in 
> state “New” for 2.5 years like 100s maybe thousands other issues in the 
> public tracker.
> 
> PK
> p...@ <>gae123.com 
> 
> 
> 
> 
>> On Oct 18, 2016, at 8:50 AM, Joshua Smith gmail.com 
>> > wrote:
>> 
>> Nick,
>> 
>> I know you are stuck in the apologist position, and there’s nothing you can 
>> do about it, but come on…
>> 
>> It is positively absurd that GAE doesn’t include any bulk email capability. 
>> If you guys are concerned about spammers abusing the service, just charge a 
>> penny per email sent or something.
>> 
>> If it’s because google can’t figure out how to provide a reliable email 
>> service, maybe call the team over at gmail, I hear they have some experience 
>> in this area.
>> 
>> You will never convince GAE users that having to use a third-party to send 
>> email is anything but a pathetic hack.
>> 
>> -Joshua
>> 
>>> On Oct 18, 2016, at 10:55 

[google-appengine] Ongoing App Engine Commitment?

[Tim Consolazio]

I've been noticing some things:

- App Engine support shunted over to Stack Overflow (so...a non-Google 
community resource).
- Mail API pretty much deprecated (no more quota expansions, 100 email 
limit). 
- Deprecation of Channel API (sure we have Firebase, but still...what's the 
commitment to that?)
- No Java 8 (and 7 was a long time in coming). 
- Gradual deprecation of local SDK in favor of remote/virtual dev (which to 
me is unacceptable, I depend on that local SDK). 

I frequently recommend App Engine (and I've built quite a few apps on it). 
I've been a rogue evangelist of the platform for years (since it was 
released in fact), and have been contracted by Google three times to build 
applications using it. So I'm not a hater in any way. 

But I can't look at these events (particularly the Stack Overflow thing) 
and ignore the impression that App Engine is beginning to feel more like a 
side project rather than a commercial offering with full resource 
commitment by the vendor. 

Curious about a formal statement of commitment here, and a complete roadmap 
of what we can reasonably expect over say, the next two years. 

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/ffc4dd64-c827-4c5f-8765-b84182e65e90%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Strange behaviour related to java Endpoints V2

[Þórir Gunnarsson]

Hi

Just wanted to check with the group if anyone is experiencing unusual 
behaviour related to endpoints v2 on AppEngine. We are using the AppEngine 
standard environment.

This morning we start getting reports from users where our Android client 
is crashing a lot. Upon investigation we discover that yesterday 
(30.03.2017) at 16:36 UTC we start getting unusual entries in the log that 
look like this:
/_ah/api/appendpoint/v8/users//dashboarddata
and a HTTP 302 response according to the logs (this then results in a crash 
in the Android client, which is a different story)
On a regular day and most of the time this call from the client looks like 
this.
/_ah/spi/is.app.services.dashboard.DashboardEndpointV9.getDashboardData

The strange thing here is the format of the log entry which is consistent 
with the format of log entries after upgrading to endpoints v2. 

We see the same exact Android client making this call and once in a while 
it seems to get the new type of response (the endpoints v2 type log entry 
and a 302).

We did not deploy anything at this time yesterday and we were certainly not 
using endpoints v2 at that time.

After some testing we have now improved the situation by upgrading our 
backend to using endpoints v2 (version 2.0.5). 
I say improved because now most log entries look like they are coming from 
endpoints v2 but once in a while we get the old style log entries and now 
the old style log entries return 302 (according to server logs)
The Android client no longer crashes but doesn't receive a response when 
this happens.

To further investigate what was happening we plugged in a http proxy to see 
exactly what was going on.
On a successful call there is a GET call to the server with a proper HTTP 
200 OK response
On a failure there is a GET call to the server with a HTTP 404 NOT FOUND 
response and an "unsupportedProtocol" error.
We don't see any difference between the two calls. The server logs tell us 
that both are served by the same instance on the backend.

 
So is anyone experiencing something similar?
Should we pin this on some Google update or possibly some misconfiguration 
on our end?


-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/3b03a787-1658-4dc1-b613-e72e0258cd7b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Deployment Issues

[Brian]

I had the same issue during the same time frame. I thought I did something 
wrong, but it cleared-up later, just like you said. Thanks for posting!

On Thursday, March 30, 2017 at 10:05:11 AM UTC-5, Joshua Smith wrote:
>
> And just like that, the problem disappeared, and it’s all working now.
>
> On Mar 30, 2017, at 10:58 AM, Joshua Smith  > wrote:
>
> I got a weird error when I tried to deploy. I updated to the latest Python 
> Launcher code, and tried again, and it said I need to rollback.
>
> This was the result:
>
> $ appcfg.py rollback .
> 10:54 AM Application: kaon-log
> 10:54 AM Host: appengine.google.com
> 10:54 AM Rolling back the update.
> Error 404: --- begin server output ---
>
> 
> 
> 404 Not Found
> 
> 
> Error: Not Found
> The requested URL 
> /api/appversion/rollback?app_id=kaon-logforce_rollback=0version=58
>  
> was not found on this server.
> 
> 
> --- end server output ---
>
>
> Is there an outage?
>
> -Joshua
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/ecfc5d75-5f90-4cde-af71-10ac487f8358%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Mail API Quotas

[Bookingforce CTO]

Hello there,
 
  being CTO of a company that is betting on Google Infrastructure when I 
found out about the limitation to 100 recipients, at first I thought "ah, I 
have to increase that quota", then, after asking (twice) to support team, I 
felt like "what's wrong with GAE ?" 

The only fact that more than 100 emails is referred as "high volume" is 
almost disturbing... we are using GAE to build cloud applications that 
scale up using multiple instances but still we can't contact our customers 
via email if they are more than 100 ? If you want to prevent spam, bill 
each email, we may be willing to pay as much as we pay for other services. 

If you just don't want to offer a serious service about email, you should 
think to discontinue the Mail API, like this it makes no sense.

Cheers,
  Luca 

On Wednesday, October 26, 2016 at 7:49:05 PM UTC+2, Nick (Cloud Platform 
Support) wrote:
>
> The Public Issue Tracker thread brought our attention has gotten tracked 
> now, thanks to PK. We'll update it with any progress.
>
> Now, as much as I'm sure we all appreciate Jeff's kind offer, I don't 
> think it will be necessary to pay him $1000, since Cloud Platform Community 
> Support are here to do a similar job at no cost. As said in other threads 
> on this topic, if anybody at any time has issues migrating from any service 
> to any other, has questions about design patterns, has questions about the 
> use of our APIs or how to refactor their code, etc., this forum is the 
> perfect place for such higher-level discussion and we would be happy to 
> discuss in depth. 
>
> We never want users to feel left alone, and asking a question once has the 
> benefit that all other users in a similar situation can read and learn from 
> the thread. This doesn't fit under the umbrella of "platform issues / 
> feature requests" (Public Issue Tracker) or "a specific error / stack 
> trace" (Stack Overflow), so it would be fine to post such threads in this 
> forum.
>
> As an additional, final comment, I'll say that we do pay close attention 
> to the requests for services / service improvements that users bring 
> forward, whether here or in the Public Issue Tracker. We've taken the 
> feedback from many users on the Mail API and bulk sending and while we 
> don't promise any concrete action given this feedback, know that we have 
> taken it into account and we want to thank you for bringing up how you view 
> things, what you'd like to see, etc.
>
> Cheers,
>
> Nick
> Cloud Platform Community Support
>
> On Tuesday, October 18, 2016 at 10:42:26 PM UTC-4, Jeff Schnitzer wrote:
>>
>> Since this thread keeps coming up… I’ll make you all an offer: For $1k 
>> I’ll migrate your GAE app to Sendgrid, Mailgun, or whatever other email 
>> service you want. Assuming your code isn’t spaghetti, it will probably take 
>> me an hour. It should take you about the same or less.
>>
>> This is sooo much kvetching about something that is so trivial to fix.
>>
>> Jeff
>>
>> On Tue, Oct 18, 2016 at 9:27 AM, PK  
>> wrote:
>>
>>> +1 for Joshua’s comment. 
>>>
>>> Nick, wrt "We always encourage users to make use of the Public Issue 
>>> Tracker “ the issue is issue 10560 
>>>  stuck 
>>> in state “New” for 2.5 years like 100s maybe thousands other issues in the 
>>> public tracker.
>>>
>>> PK
>>> p...@gae123.com 
>>>
>>>
>>>
>>>
>>> On Oct 18, 2016, at 8:50 AM, Joshua Smith >> > wrote:
>>>
>>> Nick,
>>>
>>> I know you are stuck in the apologist position, and there’s nothing you 
>>> can do about it, but come on…
>>>
>>> It is positively absurd that GAE doesn’t include any bulk email 
>>> capability. If you guys are concerned about spammers abusing the service, 
>>> just charge a penny per email sent or something.
>>>
>>> If it’s because google can’t figure out how to provide a reliable email 
>>> service, maybe call the team over at gmail, I hear they have some 
>>> experience in this area.
>>>
>>> You will never convince GAE users that having to use a third-party to 
>>> send email is anything but a pathetic hack.
>>>
>>> -Joshua
>>>
>>> On Oct 18, 2016, at 10:55 AM, 'Nick (Cloud Platform Support)' via Google 
>>> App Engine  wrote:
>>>
>>> Hey PK and Kaan,
>>>
>>> I hear your concerns. I hope that I can moderate worries by saying the 
>>> following in regard to each of the points you brought up:
>>>
>>> *1. Avoid paying for attachment bits through front-end*
>>>
>>> While the GCS integration to Mail API attachments will be discussed in 
>>> the next point, there's another GCS solution here which is very useful. You 
>>> can generate URLs to the attachment objects and merely embed these URLs in 
>>> the email. This ultimately has the same transfer cost for the user's 
>>> machine as downloading from a mail server storing the attachment, but is 
>>> slightly different since you can recall the object at a 

[google-appengine] Re: deploy is very slow

[Cole Peterson]

Hey Nick. Just go and deploy a service using glcoud deploy and you will see 
the issue.
It is unbearably slow.

On Wednesday, February 22, 2017 at 3:11:06 PM UTC-8, Nick (Cloud Platform 
Support) wrote:
>
> Hey Alex,
>
> Slow deploys can be caused by many factors. If you want to submit a Public 
> Issue Tracker  
> issue for us to look at, you should consider including the following 
> information:
>
> 1. The size on disk of your project
> 2. Whether you're using the flexible environment or not, and whether you 
> have a Dockerfile (if so, the contents of the Dockerfile)
> 3. The number of files in the project
> 4. The deploy log stored in a location given by running gcloud info and 
> seeing the "Last Log File" field.
>
> With this information we'll be able to better understand whether the 
> deploy time is expected or exceptional.
>
> Cheers,
>
> Nick
> Cloud Platform Community Support
>
> On Monday, February 20, 2017 at 11:19:58 AM UTC-5, Alex Komoroske wrote:
>>
>> I'm running into this too with the Go flexible environment. Deploys are 
>> taking 30+ minutes.
>>
>> On Monday, February 13, 2017 at 12:44:20 PM UTC-8, Nick (Cloud Platform 
>> Support) wrote:
>>>
>>> Hey Luis, 
>>>
>>> The best way in fact would be to create a Public Issue Tracker 
>>> thread, 
>>> and we'll be able to follow up there. Post the link here once you've 
>>> created a thread, and I'll be happy to assist, sending you an email which 
>>> you can reply to, attaching the log. 
>>>
>>> Cheers,
>>>
>>> Nick
>>> Cloud Platform Community Support
>>>
>>> On Sunday, February 12, 2017 at 11:57:14 AM UTC-5, Luis F De Pombo wrote:

 Hi Nick,
 I am having the same issue as Adam but I am using a nodejs Flex env. I 
 have the file you point to, what is the best email to get this to you? 
 Thank you.

 On Monday, February 6, 2017 at 10:40:48 AM UTC-8, Nick (Cloud Platform 
 Support) wrote:
>
> Just a quick update:
>
> As well, to avoid the timestamp filtering commands above, you could 
> simply run the deployment with the --verbosity debug and --log-http flags 
> and then run gcloud info to check for where the "latest log file" is 
> stored, and upload that file. The output line telling its location will 
> look like:
>
> Last Log File: 
> [/home/anon/.config/gcloud/logs/2017.02.06/18.30.49.455744.log]
>
>
> On Monday, February 6, 2017 at 12:58:05 PM UTC-5, Nick (Cloud Platform 
> Support) wrote:
>>
>> Hey Adam,
>>
>> Could you capture the deployment logs using the flags "--verbosity 
>> debug" and "--log-http", piping this output into the "ts" utility to 
>> prepend a timestamp to each line, and then email me the logs? I'll be 
>> able 
>> to analyze what might be going on that way. On linux / Mac that will 
>> look 
>> like:
>>
>> gcloud --quiet --verbosity debug --log-http app deploy app.yaml 
>> --version ${VERSION} 2>&1 | ts | tee -a deploy_log.txt 
>>
>>
>> (tee is used so that you can still read the output as it hits the 
>> console, and note that you'll have to put whatever version you want in 
>> the 
>> command where ${VERSION} is written)
>>
>> On windows, you can use powershell to run the same command line after 
>> creating the "ts" filter to prepend timestamps by running the following 
>> line:
>>
>> filter timestamp {"$(Get-Date -Format s): $_"}
>>
>>
>> Cheers,
>>
>> Nick 
>>
>> On Saturday, February 4, 2017 at 2:30:00 AM UTC-5, Adam Mathias 
>> Bittlingmayer wrote:
>>>
>>> Yes, it is the Flex Env.
>>>
>>> Reviewing the container build logs, it does not seem deterministic. 
>>>  The total time varies, and the place in the process where multiple 
>>> minutes 
>>> pass between two log lines varies.
>>>
>>>
>>> On Friday, 3 February 2017 12:40:25 UTC-8, Nick (Cloud Platform 
>>> Support) wrote:

 Hey Adam,

 Are you deploying a Flexible Environment app? If so, what do the 
 container builder logs from the Console show as far as activity during 
 this 
 time?

 Also, is this still occurring, or was it only for a certain period 
 of time? If it's transient, it might just be a momentary service 
 degradation, which for a Beta product is to be expected at this stage. 
 When 
 a product goes into General Availability, we'll often have certain 
 Service 
 Level targets in terms of latency, uptime, that you can rely on (for 
 example, read here  for 
 Compute Engine SLA's)

 Cheers,

 Nick
 Cloud Platform Community Support

 On 

[google-appengine] Deployment Errors

[Richard Cheesmar]

I'm using Pycharm to deploy my App, normally work no problem, but today all 
I keep getting is the following error:

01:12 PM Uploading 51 files and blobs.
01:13 PM Rolling back the update.
Error 500: --- begin server output ---


Server Error (500)

A server error has occurred.
--- end server output ---

Process finished with exit code 1

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/4625eba0-dc53-4624-bcb1-121f3d7c2d2d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: Understanding java.util.ConcurrentModificationException: too much contention on these datastore entities. please try again.

[Louise Elmose Hedegaard]

Hi Jeff,

I am not sure I understand how putting the code in a transactional task 
helps me.

The simplified code is:

Mail mailToSend = getMail();
SentStatus sentStatus = sendMailToExternalMailSystem(mailToSend);
updateMailStatusInDatastore(mailToSend, sentStatus)

The problem is that the datastore write "updateMailStatusInDatastore" might 
fail due to CME, and at this point the mail has already been sent to the 
external mail system.
So even though I move this code to a transactional task, a failure in the 
datastore write will still lead me to a scenario where the mail has been 
sent, but the mail status has not been updated properly in my app.

Thanks,
-Louise

Den torsdag den 30. marts 2017 kl. 20.30.45 UTC+2 skrev Jeff Schnitzer:
>
> There may be clock skew in the cluster; 15s is a lot but you can’t assume 
> that log entry timestamps are exact.
>
> You should send email by enqueueing a transactional task. Do not put 
> non-idempotent remote calls in your transactions (ideally, don’t put any 
> remote calls in transactions). 
>
> For example, let’s say you want to email a receipt on purchase. Create a 
> deferred SendReceiptEmailTask that holds the purchase record key; it looks 
> up the record, formulates the email body, and makes the final call to the 
> email service. You can enlist this in your transaction that creates the 
> purchase record. This also has the benefit of moving all that work outside 
> of your transaction and shortening the critical section.
>
> There is a very small risk that tasks may execute twice, but it’s small 
> enough and harmless enough to ignore. I’ve sent millions of emails this way 
> with no complaints. And you don’t really have a choice anyway - I haven’t 
> found any email sending companies that allow you to specify an idempotency 
> key or some other way of guaranteeing once-and-only-once behavior.
>
> Cheers,
> Jeff
>
>
>
> On Thu, Mar 30, 2017 at 3:44 AM, Louise Elmose Hedegaard <
> louise...@gmail.com > wrote:
>
>> Hi Jeff,
>>
>> I have CME erratically - so I guess you are saying I should take a look 
>> at the look for the CME runs, and check whether the same EG's are accessed 
>> by other operations?
>> It does seem that there might be a clash with other operations when the 
>> CME occurs.
>> I am not sure about the timing though - there is a updateOrder operation 
>> which fails with CME at 14:59:46.684 and the send mail fails with CME 
>> at 15:01:06.583...
>> When you say "Any change to that EG" it is the entire EG you mean, or 
>> only rows with the same id in the EG?
>>
>> I need to ensure that my transaction does not fail, as it will result in 
>> duplicate emails being sent if it does.
>> There are other operations working on the same EG which comes randomly as 
>> they are caused by a webhook in another application.
>> What is the best way to ensure that these webhook operations do not cause 
>> the send mail operation to get a CME?
>> I can only think of very hacked and ugly solutions - e.g. to have a 
>> lock/switch which is on when the send mail operation is running, on only 
>> executing webhook operations when the lock/switch is off.
>>
>> Thanks,
>> -Louise
>>
>> Den tirsdag den 28. marts 2017 kl. 22.11.15 UTC+2 skrev Jeff Schnitzer:
>>>
>>> When you load a key in a datastore transaction, that EG is enlisted in 
>>> the transaction. Any change to that EG by any other process in the system 
>>> will cause your commit to rollback with CME. Even if your transaction is 
>>> "read-only”.
>>>
>>> When I said “linear” I mean that if you have a quiet datastore (no other 
>>> activity) and you start a transaction and you perform a series of 
>>> operations and you commit the transaction, you should not see CME. 
>>>
>>> If you see CME erratically, then you have contention in your system.
>>> If you see CME consistently 100% of the time and you aren’t under heavy 
>>> load, then you have a bug in your transaction logic.
>>>
>>> Jeff
>>>
>>> On Tue, Mar 28, 2017 at 6:29 AM, Louise Elmose Hedegaard <
>>> louise...@gmail.com> wrote:
>>>
 Hi,

 I stumbled upon an instance in the log where the sendNow list is empty 
 - but still the operation fails with ConcurrentModificationException.
 How is that possible - it is only the initial read 1) operation which 
 is executed on the datastore.

 Thanks,
 -Louise


 Den tirsdag den 28. marts 2017 kl. 08.58.35 UTC+2 skrev Louise Elmose 
 Hedegaard:
>
> Hi Adam,
>
> Why do you mention timeout - ConcurrentModificationException is not 
> related to timeouts is it?
>
> Thanks,
> -Louise
>
> Den lørdag den 25. marts 2017 kl. 00.02.05 UTC+1 skrev Adam (Cloud 
> Platform Support):
>>
>> A great article that explains this is 'Timeouts due to write 
>> contention 
>> 
>> ': 
>>
>> *Writes to a single