Re: [google-appengine] Re: Google App Engine is slow to deploy, hangs on "Updating service [someproject]..."

['Nick (Cloud Platform Support)' via Google App Engine]

Hey Stanislas,

Great to hear. I think you'll be glad you did, from what I see the 
flexibility and power of kubernetes is something really amazing if you can 
leverage it. Feel free to post to the kubernetes-users 
 group forum if 
you have any open-ended discussions or questions about this. 
Troubleshooting can be done on Stack Overflow  
and issues can be reported at the issuetracker.google.com.

Cheers,

Nick
Cloud Platform Community Support

On Thursday, April 6, 2017 at 4:27:30 AM UTC-4, Stanislas Marion wrote:
>
> Hi Nick,
> Thank you so much for your time and detailed explanations. I've been 
> investigating GKE and it seems like it strikes the right balance between 
> speed and convenience, so I'll probably move progressively towards it.
> Best,
>
> On Thu, Apr 6, 2017 at 1:28 AM 'Nick (Cloud Platform Support)' via Google 
> App Engine  wrote:
>
>> Hey Stanislas,
>>
>> So, I can confirm that my prior investigation and intuitions were on the 
>> right track. I can confirm that a majority of deployment lag comes from 
>> programming the Google Cloud Load Balancers (GCLB). This is what we've seen 
>> in this case. Updates have to go out across the entire infrastructure while 
>> still respecting certain locks used to keep configurations consistent. 
>> These Load Balancers are not visible to users in the Console and not 
>> user-configurable as they're infrastructural and meant to be exactly as 
>> they are, else users may try to modify or accidentally delete them, which 
>> would cause a lot of issues.
>>
>> We're devoting a lot of energy to decreasing GCLB configuration push 
>> times, so rest assured that our efforts in that direction should pay off 
>> going forward.
>>
>> Cheers,
>>
>> Nick
>> Cloud Platform Community Support
>>
>> On Tuesday, March 21, 2017 at 6:41:52 PM UTC-4, Stanislas Marion wrote:
>>>
>>> Great, thank you so much for your help, I'll be very interested in the 
>>> details you'll get from your investigation.
>>>
>>> On Tue, Mar 21, 2017 at 9:22 PM 'Nick (Cloud Platform Support)' via 
>>> Google App Engine  wrote:
>>>
 Hey Stanislas,

 The exact explanation speculated on in my last post shouldn't be taken 
 as any description of what's necessarily going on, however it was an 
 estimate of what might be happening based on the logs observed. I'm 
 corresponding with experts in this area to get a more clear answer at the 
 moment. 

 You could look into deploying on Container Engine 
 , which would mean that 
 the front-end management done by the App Engine Flexible Environment 
 infrastructure wouldn't be happening, rather it would be the 
 responsibility 
 of the resources you deploy on Container Engine (a managed service based 
 pretty transparently on Kubernetes ). Surely 
 deploying new container images to your pool of instances in a cluster (or 
 multiple clusters) would be quite fast, since the master sitting in front 
 of your clusters and the nodes in the cluster are not as massively 
 distributed as the App Engine Flexible Environment serving infrastructure, 
 hence updating their routing rules would be relatively fast. This is 
 something to look into and experiment with if you don't want to wait on 
 the 
 more detailed word from experts, but don't rush to that if you're not all 
 that curious.

 I'll get back to this thread with more details when they're forthcoming 
 in our investigation.


 Cheers,

 Nick
 Cloud Platform Community Support 

 On Tuesday, March 21, 2017 at 3:50:34 PM UTC-4, Stanislas Marion wrote:
>
> Hi Nick,
> Thanks a lot for the lengthy explanation. 
> In this light, is there anything I can do to speed things up? Like for 
> instance take care of the load-balancer myself? Indeed I don't see a 
> reason 
> why it should need to be changed. Could I do this with GAE or would I 
> have 
> to move to G Container/Compute E?
> Cheers,
>
> On Tue, Mar 21, 2017 at 8:42 PM 'Nick (Cloud Platform Support)' via 
> Google App Engine  wrote:
>
>> Hey Stanislas,
>>
>> My initial hunch was that the issue was the deployment of other 
>> resources necessary to support the containers running. My analysis of 
>> deployment-related logs appears to confirm this:
>>
>> I created a simple NodeJS app using your dockerfile and default.yaml. 
>> I then pushed the docker image to gcr.io and ran "gcloud app deploy 
>> --image-url ..."
>>
>> After about 1 minute of waiting, all resources associated with the 
>> deployment had apparently completed, but the command had not returned 

[google-appengine] Re: Sudden import errors in unmodified Python 2.7 applications since the past few hours.

['Ram Sagar Mogalasetty' via Google App Engine]

HI Team,

I am also facing the same issue while importing the module gspread in 
Google App Engine. 
but gspread module is already installed

rmogalase...@google.com:python-practice-164309:~$ pythonPython 2.7.9 
(default, Jun 29 2016, 13:08:31) [GCC 4.9.2] on linux2Type "help", 
"copyright", "credits" or "license" for more information.>>> import gspread
>>>  

*Issue:-*


Traceback (most recent call last):
  File 
"/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/runtime/wsgi.py",
 line 240, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
  File 
"/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/runtime/wsgi.py",
 line 299, in _LoadHandler
handler, path, err = LoadObject(self._handler)
  File 
"/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/runtime/wsgi.py",
 line 85, in LoadObject
obj = __import__(path[0])
  File 
"/base/data/home/apps/s~google.com:python-practice-164309/20170411t174854.400482399619859826/read_file.py",
 line 1, in 
import gspread
ImportError: No module named gspread



Please help me out of this Issue. Thanks in Advance


Regards,
M Ram Sagar


On Tuesday, March 28, 2017 at 3:15:22 PM UTC+5:30, rrk wrote:
>
> Hi,
>
> Anyone else facing sudden ImportError issues with previously working 
> applications in Python 2.7 runtime of Google App Engine?
>
> One such error is given below, which happens when *requests *package is 
> trying to import *netrc*. This code was working fine until today. 
>
> Traceback (most recent call last):
>
>   File 
> "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/runtime/wsgi.py",
>  
> line 240, in Handle
>
> handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
>
>   File 
> "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/runtime/wsgi.py",
>  
> line 299, in _LoadHandler
>
> handler, path, err = LoadObject(self._handler)
>
>   File 
> "/base/data/home/runtimes/python27_experiment/python27_lib/versions/1/google/appengine/runtime/wsgi.py",
>  
> line 85, in LoadObject
>
> obj = __import__(path[0])
>
>   File 
> "/base/data/home/apps/s~app-id-removed/p31-b.391085029481971487/x.py",
>  
> line 36, in 
>
> from yy import *
>
>   File "/base/data/home/apps/s~app-id-removed/p31-b.391085029481971487/
> yy.py", line 32, in 
>
> import requests
>
>   File 
> "/base/data/home/apps/s~app-id-removed/p31-b.391085029481971487/requests/__init__.py",
>  
> line 52, in 
>
> from . import utils
>
>   File 
> "/base/data/home/apps/s~app-id-removed/p31-b.391085029481971487/requests/utils.py",
>  
> line 19, in 
>
> from netrc import netrc, NetrcParseError
>
>   File 
> "/base/data/home/runtimes/python27_experiment/python27_dist/lib/python2.7/netrc.py",
>  
> line 7, in 
>
> import pwd
>
> ImportError: No module named pwd
>
>
> I have the same application deployed for other customers within their 
> accounts and also in my account, but this error seems to have occurred only 
> for one deployment.
>
> Why is this happening? Does this have something to do with the 
> python27_experiment version, which I believe has been in use since over a 
> month (based on error traces on the internet with this version), but this 
> ImportError seems to have got triggered only today? May be my client's 
> application has been included within the experimental version?
>
> Thanks and regards,
> Raj
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/2f0e51e8-69aa-4346-9530-65e26201b38d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Slow cache, slow datastore, pending requests

[Tomas]

Okay, after lot of testing and deployments I've found one weird issue - 
json deserialisation is slow when the app is under load. My microservice 
accepts a json oject through post (simple bean with 10 short text 
properties). I can see that deserialisation of the bean takes 3+ seconds 
when the app experiences spike in requests. If the app is under the load 
the latency of datastore/memcache also goes up significantly (2+ seconds 
for datastore and memcache is very common).

I just wonder why is it happening - I don't think that 20 simple post 
updates should cause such issues for app engine...

The json is just my wrapper around Gson/Jackson...

log.debug("Deserialising with {}", json);
Cast data = this.json.deserialise(c, value);
log.debug("Deserialised with {}", json);

20:55:06.893 [s~/1.400479083523705872].: 2017-04-11 08:55:06 DEBUG 
StringToObjectConverter:32 - Deserialising with JacksonImpl@df29bb7
 20:55:11.389 [s~/1.400479083523705872].: 2017-04-11 08:55:06 DEBUG 
StringToObjectConverter:34 - Deserialised with JacksonImpl@df29bb7


On Tuesday, 11 April 2017 04:38:40 UTC+12, George (Cloud Platform Support) 
wrote:
>
> Hello Thomas, 
>
> You may consider making use of the “Async Datastore API 
> ” 
> and dedicated memcache, as described in the “Memcache Overview” document 
> . 
>
> To investigate this issue properly, we need a minimum of information: 
>
> - What project are you speaking about, exactly?  
>
> - What is the general structure of your Java code? 
>
> - Does your Java application make use of thread synchronization? 
>
> - Is the application designed with easy scaling in mind? The “Designing 
> for Scale” document 
>  provides 
> detailed advice in this respect. 
>
> - A timestamp for an example of slow performance for memcache or 
> datastore. 
>
> - A copy of the app.yaml would come quite handy. 
>
> - Any other piece of information you deem relevant to the issue, in excess 
> with what has been already provided. 
>
> You can send us confidential information, as requested above, by private 
> email, using the drop-down menu at the top right side of the message 
> window.  
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/b093fd4c-b3af-4e7e-aa48-2727152d1d2e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Recommended configuration for a java based App Engine standard application?

[Þórir Gunnarsson]

Excellent news, thanks for looking into this. I will add enabling the 
endpoints v2 tooling to my todo list, the feature set and management 
aspects sound like something I could benefit from.

ÞG
 

On Monday, April 10, 2017 at 5:13:57 PM UTC, Adam (Cloud Platform Support) 
wrote:
>
> The 'com.google.appengine' development server works without issues with 
> endpoints management in all of the v2 projects I've tested it with. The 
> only changes I've needed are the following:
>
> 1) In the pom.xml for the project, change 'com.google.cloud.tools' to 
> 'com.google.appengine' and change the  to the appropriate App 
> Engine SDK version (eg. 1.9.51) in the 'appengine-maven-plugin'  
> section.
>
> 2) Replace the new maven goals on the command line with the old ones (eg. 
> 'mvn appengine:devserver' instead of 'mvn appengine:run').
>
> The Endpoints tooling and managment is handled by the separate JARs 
> 'com.google.endpoints.endpoints:framework' and 
> 'com.google.endpoints:endpoints-management-control-appengine-all' which 
> don't have a dependency on the specific devserver you're using.
>
> The Java parts of the two devservers in fact are virtually identical, it's 
> the Python layer that is different (and the primary source of the issues in 
> the Cloud SDK devserver).
>
> On Friday, April 7, 2017 at 5:23:14 PM UTC-4, Þórir Gunnarsson wrote:
>>
>> I didn't encounter any specific issues with the endpoints management 
>> functionality as such, didn't go that far, the documentation is just all 
>> referencing the com.google.cloud.com tools and I wanted to get some 
>> feedback on where these things are at before going into implementation. 
>> From what I could glimpse from the documentation (
>> https://cloud.google.com/endpoints/docs/frameworks/java/generate-client-libraries-android)
>>  
>> of the configuration it seemed that I need to be more explicit about what 
>> java classes implement endpoints, at least when generating the client 
>> library. At the moment I have 35 different classes with around 300 
>> different @ApiMethods so listing them all up and maintaining the list is 
>> something I need to do in a sustainable way. You could probably argue that 
>> I'm just lazy wanting reassurance before moving into new tech and writing 
>> scripts to support it :-)
>>
>> Which development server should I use if I enable the endpoints 
>> management functionality? Does the one in com.google.appengine work?
>>
>> ÞG
>>
>> On Friday, April 7, 2017 at 8:50:49 PM UTC, Adam (Cloud Platform Support) 
>> wrote:
>>>
>>> You can still use Endpoints v2 tooling with the 'com.google.appengine' 
>>> plugin as it is handled by endpoints-framework-tools and gcloud 
>>> service-management. The only difference should be the version of the dev 
>>> server you launch for local development. What specific issues with 
>>> endpoints management did you encounter when switching back to the 
>>> 'com.google.appengine' plugin?
>>>
>>> On Thursday, April 6, 2017 at 4:57:37 AM UTC-4, Þórir Gunnarsson wrote:

 Does anyone have any insights into endpoints v1 vs. endpoints v2? Any 
 real life success stories perhaps?

 We recently switched to endpoints v2 but without the endpoints 
 management functionality, I'm hesitant to go all in if this isn't being 
 used in production systems by at least a few people, or if it is not 
 production ready in general.

 ÞG


 On Tuesday, April 4, 2017 at 4:42:25 PM UTC, Jeff Schnitzer wrote:
>
> You’re not crazy, I have also seen that problem. Use the 
> com.google.appengine 
> maven plugin for the dev server for now. 
>
> I think this is the most relevant (albeit cryptic) issue that tracks 
> that particular problem: 
> https://github.com/GoogleCloudPlatform/app-maven-plugin/issues/158
>
> The com.google.tools plugin is the future, but the future isn’t quite 
> here yet :-(
>
> Jeff
>
> On Tue, Apr 4, 2017 at 6:49 AM, Þórir Gunnarsson <
> thorirgu...@goodlifeme.com> wrote:
>
>> I am currently using Maven and IntelliJ which in itself is fine. Most 
>> of my issues are with the Maven plugins and recently endpoints versions.
>> The com.google.cloud.tools plugin is not entirely there it seems, but 
>> in a lot of documentation from Google it is the go-to plugin.
>>
>> I tried to switch to the com.google.cloud.tools plugin the other day 
>> but ran into an issue with the local dev server (
>> https://issuetracker.google.com/u/1/issues/36589995) which prompted 
>> this thread here.
>>
>> ÞG
>>
>>
>> On Tuesday, April 4, 2017 at 1:24:52 PM UTC, Jeff Schnitzer wrote:
>>>
>>> Your path of least resistance is to use Maven (as opposed to Gradle 
>>> or Ant/Ivy). If you’re working in Java, you definitely want an IDE - 
>>> Eclipse and IntelliJ both have their following. I think most people 
>>> will be 
>>> happier