[google-appengine] Error while handling "large'ish" number of parallel requests
Hi, I am using appengine flexible environment. I am facing issues when there is a sudden spurt of requests to our system. (About 30 requests in a second). The server gets back to normal in 5-10 seconds (perhaps with the decrease in traffic again) Generally the requests come at about 5-8 per second. I am running with 2 instances. 2 10 120 0.7 AFAICT the requests did not hit my servlet. (I added a log, the first line in the servlet, and for failed request it does not get printed). "loading_request" is 0 for the failed requests as well. The error code is 502 Is it that the requests are queued while a new instance comes up? Is it possible to override that, that is start the process of bringing up the new servers, but use the old one (if it is taking a while?). The log shows some of the request waiting for 600seconds, most though fail in 10-15ms. The failure happens for between 0.5 to 2% of the requests -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/483c4f5e-870e-4d65-b52f-d204385627a7%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Re: Error while handling "large'ish" number of parallel requests
On Sunday, October 2, 2016 at 10:29:31 PM UTC+5:30, Adam (Cloud Platform Support) wrote: > > How many instances exist when the traffic burst occurs? Are you hitting > your defined max of 10 or is it not able to scale up to the maximum? > The number of instances does not go beyond 5. > > Two possibilities come to mind: hitting the instance max and requests are > then being discarded, or scaling isn't happening quickly enough and there > are too many pending requests. > In the case of a "burst", there are ~20-30 concurrent requests. And as I mentioned in the previous mail, the error code is 502, and no logs from my servlet. > > > A value of 0 for "loading_request" means it was not the very first request > to the instance which caused it to spin up, which is normal for the > majority of requests. > > > On Wednesday, September 28, 2016 at 10:36:46 AM UTC-4, Vinay Chitlangia > wrote: >> >> Hi, >> I am using appengine flexible environment. >> >> I am facing issues when there is a sudden spurt of requests to our >> system. (About 30 requests in a second). The server gets back to normal in >> 5-10 seconds (perhaps with the decrease >> in traffic again) >> Generally the requests come at about 5-8 per second. >> >> I am running with 2 instances. >> >> >> 2 >> 10 >> 120 >> >> 0.7 >> >> >> >> AFAICT the requests did not hit my servlet. (I added a log, the first >> line in the servlet, and for failed request it does not get printed). >> "loading_request" is 0 for the failed requests as well. >> >> The error code is 502 >> >> Is it that the requests are queued while a new instance comes up? Is it >> possible to override that, that is start the process of bringing up the new >> servers, >> but use the old one (if it is taking a while?). The log shows some of the >> request waiting for 600seconds, most though fail in 10-15ms. >> The failure happens for between 0.5 to 2% of the requests >> > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/5b90cc3e-594d-4942-90de-15258a415e12%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Re: Error while handling "large'ish" number of parallel requests
On Tue, Oct 4, 2016 at 10:51 PM, 'Adam (Cloud Platform Support)' via Google App Engine wrote: > It's possible that new instances may not be spinning up quickly enough to > respond to spikes in load, causing requests to be discarded. I'd recommend > lowering the value to 60 (the minimum) to allow the > autoscaler checks to occur more frequently, and keep > at 0.5 to give new instances more headroom to spin up. > Thanks Adam. I have restarted our servers with target-utilization of 0.4 (have kept the cool-down-period the same..in the interest of changing one variable at a time!!) > > Try this first to see if the incidences of 502 errors improve. You can try > to adjust further by increasing the number of or > lowering further. > > Java apps in particular can incur large startup times due to classpath > scanning and initialization. If you can give some details about the types > of frameworks and libraries you're using, as well as the request times > your'e seeing for loading requests (loading_request = 1) I may be able to > give some further recommendations. > This is a backend server with cloud bigtable as its major dependency. I could not correlate the 502s in my server with the bigtable errors (as reported by the cluster dashboard). I will try to see if I can get some loading_request=1 request when the server is under duress. The absolute first request at the time of server startup are well behaved, that is to say that my server does not have a very big upfront cost...it does about 2 seconds worth of initialization which is done by a servlet. > On Wednesday, September 28, 2016 at 10:36:46 AM UTC-4, Vinay Chitlangia > wrote: >> >> Hi, >> I am using appengine flexible environment. >> >> I am facing issues when there is a sudden spurt of requests to our >> system. (About 30 requests in a second). The server gets back to normal in >> 5-10 seconds (perhaps with the decrease >> in traffic again) >> Generally the requests come at about 5-8 per second. >> >> I am running with 2 instances. >> >> >> 2 >> 10 >> 120 >> >> 0.7 >> >> >> >> AFAICT the requests did not hit my servlet. (I added a log, the first >> line in the servlet, and for failed request it does not get printed). >> "loading_request" is 0 for the failed requests as well. >> >> The error code is 502 >> >> Is it that the requests are queued while a new instance comes up? Is it >> possible to override that, that is start the process of bringing up the new >> servers, >> but use the old one (if it is taking a while?). The log shows some of the >> request waiting for 600seconds, most though fail in 10-15ms. >> The failure happens for between 0.5 to 2% of the requests >> > -- > You received this message because you are subscribed to a topic in the > Google Groups "Google App Engine" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/google-appengine/TvPHzbMMbhc/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at https://groups.google.com/group/google-appengine. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/google-appengine/f9fc1696-d71f-4684-a490- > e612f9d17d23%40googlegroups.com > <https://groups.google.com/d/msgid/google-appengine/f9fc1696-d71f-4684-a490-e612f9d17d23%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CADt9hj8tbMreqYXcqEN3VLkRZqHKKQF-ZonEr7u%2BdjxpLnoGZQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Re: Error while handling "large'ish" number of parallel requests
On Wed, Oct 5, 2016 at 11:16 AM, Vinay Chitlangia wrote: > > > On Tue, Oct 4, 2016 at 10:51 PM, 'Adam (Cloud Platform Support)' via > Google App Engine wrote: > >> It's possible that new instances may not be spinning up quickly enough to >> respond to spikes in load, causing requests to be discarded. I'd recommend >> lowering the value to 60 (the minimum) to allow the >> autoscaler checks to occur more frequently, and keep >> at 0.5 to give new instances more headroom to spin up. >> > Thanks Adam. > I have restarted our servers with target-utilization of 0.4 (have kept the > cool-down-period the same..in the interest of changing one variable at a > time!!) > So this is what has happened...I am not sure what to make of it... The number of servers are now at 10. The error rate is 0.6% (with the latest deployment) so its at the lower end, certainly has not broken the proverbial glass:) The total number of requests for the day are well within reason...so there is no X factor, in that we do not have an unuusally busy or lax day. While I was at it, I have changed the cool down period to what you suggested...I am all in:):) > >> Try this first to see if the incidences of 502 errors improve. You can >> try to adjust further by increasing the number of or >> lowering further. >> >> Java apps in particular can incur large startup times due to classpath >> scanning and initialization. If you can give some details about the types >> of frameworks and libraries you're using, as well as the request times >> your'e seeing for loading requests (loading_request = 1) I may be able to >> give some further recommendations. >> > This is a backend server with cloud bigtable as its major dependency. I > could not correlate the 502s in my server with the bigtable errors (as > reported by the cluster dashboard). > I will try to see if I can get some loading_request=1 request when the > server is under duress. The absolute first request at the time of server > startup are well behaved, that is to say that my server does not have a > very big upfront cost...it does about 2 seconds worth of initialization > which is done by a servlet. > >> On Wednesday, September 28, 2016 at 10:36:46 AM UTC-4, Vinay Chitlangia >> wrote: >>> >>> Hi, >>> I am using appengine flexible environment. >>> >>> I am facing issues when there is a sudden spurt of requests to our >>> system. (About 30 requests in a second). The server gets back to normal in >>> 5-10 seconds (perhaps with the decrease >>> in traffic again) >>> Generally the requests come at about 5-8 per second. >>> >>> I am running with 2 instances. >>> >>> >>> 2 >>> 10 >>> 120 >>> >>> 0.7 >>> >>> >>> >>> AFAICT the requests did not hit my servlet. (I added a log, the first >>> line in the servlet, and for failed request it does not get printed). >>> "loading_request" is 0 for the failed requests as well. >>> >>> The error code is 502 >>> >>> Is it that the requests are queued while a new instance comes up? Is it >>> possible to override that, that is start the process of bringing up the new >>> servers, >>> but use the old one (if it is taking a while?). The log shows some of >>> the request waiting for 600seconds, most though fail in 10-15ms. >>> The failure happens for between 0.5 to 2% of the requests >>> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "Google App Engine" group. >> To unsubscribe from this topic, visit https://groups.google.com/d/to >> pic/google-appengine/TvPHzbMMbhc/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> google-appengine+unsubscr...@googlegroups.com. >> To post to this group, send email to google-appengine@googlegroups.com. >> Visit this group at https://groups.google.com/group/google-appengine. >> To view this discussion on the web visit https://groups.google.com/d/ms >> gid/google-appengine/f9fc1696-d71f-4684-a490-e612f9d17d23% >> 40googlegroups.com >> <https://groups.google.com/d/msgid/google-appengine/f9fc1696-d71f-4684-a490-e612f9d17d23%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CADt9hj__Ropr56vLbdsRcxb5rgFHM2aaYbYBcQ%2Byr0--mGH2-w%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Dedicated memcache keeps flushing
We are using dedicated memcache in our application. Curiously the oldest cached key time is almost always 1-2 days. The dedicated quota is 1GB and the cache size never crosses 500MB. Entries are made to memcache with 365 days expiration time. Wondering if this is expected that the cache will keep purging data on a 1-2 day scale. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/f32658b8-2760-4a85-8753-f75afead4f6d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Re: Error while handling "large'ish" number of parallel requests
Things have improved with your suggested changes. Have not seen 502s in the
last 2 days.
There is nothing in the logs when 502 does happen. Copy / paste of a sample
log entry.
13:38:54.761POST502281 B1,043.1 sAppEngine-Google;
(+http://code.google.com/appengine; appid: s~yawasa-1077)/read
107.178.195.146 - - [06/Oct/2016:13:38:54 +0530] "POST /read HTTP/1.1" 502
281 - "AppEngine-Google; (+http://code.google.com/appengine;)" ms=1043100
cpu_ms=0 cpm_usd=3.1404e-8 loading_request=0 instance=-
app_engine_release=1.9.44 trace_id=-
{
metadata: {…}
protoPayload: {…}
insertId: "2016-10-06|01:26:18.075585-07|10.106.4.84|873295067"
log: "appengine.googleapis.com/request_log"
httpRequest: {
status: 502
}
operation: {…}
}
On Saturday, October 8, 2016 at 3:22:37 AM UTC+5:30, Adam (Cloud Platform
Support) wrote:
>
> Thanks for the updates. If the issue happens again it will also be useful
> to know the message detail from the 502 response. Usually in the context of
> App Engine flexible it means there are no instances ready to serve the
> request however it's good to avoid any ambiguity.
>
> On Wednesday, October 5, 2016 at 3:11:53 AM UTC-4, Vinay Chitlangia wrote:
>>
>>
>>
>> On Wed, Oct 5, 2016 at 11:16 AM, Vinay Chitlangia wrote:
>>
>>>
>>>
>>> On Tue, Oct 4, 2016 at 10:51 PM, 'Adam (Cloud Platform Support)' via
>>> Google App Engine > wrote:
>>>
>>>> It's possible that new instances may not be spinning up quickly enough
>>>> to respond to spikes in load, causing requests to be discarded. I'd
>>>> recommend lowering the value to 60 (the minimum) to
>>>> allow the autoscaler checks to occur more frequently, and keep
>>>> at 0.5 to give new instances more headroom to spin up.
>>>>
>>> Thanks Adam.
>>> I have restarted our servers with target-utilization of 0.4 (have kept
>>> the cool-down-period the same..in the interest of changing one variable at
>>> a time!!)
>>>
>> So this is what has happened...I am not sure what to make of it...
>> The number of servers are now at 10. The error rate is 0.6% (with the
>> latest deployment) so its at the lower end, certainly has not broken the
>> proverbial glass:)
>> The total number of requests for the day are well within reason...so
>> there is no X factor, in that we do not have an unuusally busy or lax day.
>> While I was at it, I have changed the cool down period to what you
>> suggested...I am all in:):)
>>
>>>
>>>> Try this first to see if the incidences of 502 errors improve. You can
>>>> try to adjust further by increasing the number of or
>>>> lowering further.
>>>>
>>>> Java apps in particular can incur large startup times due to classpath
>>>> scanning and initialization. If you can give some details about the types
>>>> of frameworks and libraries you're using, as well as the request times
>>>> your'e seeing for loading requests (loading_request = 1) I may be able to
>>>> give some further recommendations.
>>>>
>>> This is a backend server with cloud bigtable as its major dependency. I
>>> could not correlate the 502s in my server with the bigtable errors (as
>>> reported by the cluster dashboard).
>>> I will try to see if I can get some loading_request=1 request when the
>>> server is under duress. The absolute first request at the time of server
>>> startup are well behaved, that is to say that my server does not have a
>>> very big upfront cost...it does about 2 seconds worth of initialization
>>> which is done by a servlet.
>>>
>>>> On Wednesday, September 28, 2016 at 10:36:46 AM UTC-4, Vinay Chitlangia
>>>> wrote:
>>>>>
>>>>> Hi,
>>>>> I am using appengine flexible environment.
>>>>>
>>>>> I am facing issues when there is a sudden spurt of requests to our
>>>>> system. (About 30 requests in a second). The server gets back to normal
>>>>> in
>>>>> 5-10 seconds (perhaps with the decrease
>>>>> in traffic again)
>>>>> Generally the requests come at about 5-8 per second.
>>>>>
>>>>> I am running with 2 instances.
>>>>>
>>>>>
>>>>> 2
>>>>> 10
>>>>> 120
>>>>>
>>>>> 0.7
>>>>>
>>>>>
>>>>>
>>>>&g
Re: [google-appengine] Re: Error while handling "large'ish" number of parallel requests
Things have improved with your suggested changes. Have not seen 502s in the
last 2 days.
There is nothing in the logs when 502 does happen. Copy / paste of a sample
log entry.
13:38:54.761POST502281 B1,043.1 sAppEngine-Google; (+
http://code.google.com/appengine; /read
107.178.195.146 - - [06/Oct/2016:13:38:54 +0530] "POST /read HTTP/1.1" 502
281 - "AppEngine-Google; (+http://code.google.com/appengine;)" ms=1043100
cpu_ms=0 cpm_usd=3.1404e-8 loading_request=0 instance=-
app_engine_release=1.9.44 trace_id=-
{
metadata: {…}
protoPayload: {…}
insertId: "2016-10-06|01:26:18.075585-07|10.106.4.84|873295067"
log: "appengine.googleapis.com/request_log"
httpRequest: {
status: 502
}
operation: {…}
}
On Saturday, October 8, 2016 at 3:22:37 AM UTC+5:30, Adam (Cloud Platform
Support) wrote:
>
> Thanks for the updates. If the issue happens again it will also be useful
> to know the message detail from the 502 response. Usually in the context of
> App Engine flexible it means there are no instances ready to serve the
> request however it's good to avoid any ambiguity.
>
> On Wednesday, October 5, 2016 at 3:11:53 AM UTC-4, Vinay Chitlangia wrote:
>>
>>
>>
>> On Wed, Oct 5, 2016 at 11:16 AM, Vinay Chitlangia wrote:
>>
>>>
>>>
>>> On Tue, Oct 4, 2016 at 10:51 PM, 'Adam (Cloud Platform Support)' via
>>> Google App Engine > wrote:
>>>
>>>> It's possible that new instances may not be spinning up quickly enough
>>>> to respond to spikes in load, causing requests to be discarded. I'd
>>>> recommend lowering the value to 60 (the minimum) to
>>>> allow the autoscaler checks to occur more frequently, and keep
>>>> at 0.5 to give new instances more headroom to spin up.
>>>>
>>> Thanks Adam.
>>> I have restarted our servers with target-utilization of 0.4 (have kept
>>> the cool-down-period the same..in the interest of changing one variable at
>>> a time!!)
>>>
>> So this is what has happened...I am not sure what to make of it...
>> The number of servers are now at 10. The error rate is 0.6% (with the
>> latest deployment) so its at the lower end, certainly has not broken the
>> proverbial glass:)
>> The total number of requests for the day are well within reason...so
>> there is no X factor, in that we do not have an unuusally busy or lax day.
>> While I was at it, I have changed the cool down period to what you
>> suggested...I am all in:):)
>>
>>>
>>>> Try this first to see if the incidences of 502 errors improve. You can
>>>> try to adjust further by increasing the number of or
>>>> lowering further.
>>>>
>>>> Java apps in particular can incur large startup times due to classpath
>>>> scanning and initialization. If you can give some details about the types
>>>> of frameworks and libraries you're using, as well as the request times
>>>> your'e seeing for loading requests (loading_request = 1) I may be able to
>>>> give some further recommendations.
>>>>
>>> This is a backend server with cloud bigtable as its major dependency. I
>>> could not correlate the 502s in my server with the bigtable errors (as
>>> reported by the cluster dashboard).
>>> I will try to see if I can get some loading_request=1 request when the
>>> server is under duress. The absolute first request at the time of server
>>> startup are well behaved, that is to say that my server does not have a
>>> very big upfront cost...it does about 2 seconds worth of initialization
>>> which is done by a servlet.
>>>
>>>> On Wednesday, September 28, 2016 at 10:36:46 AM UTC-4, Vinay Chitlangia
>>>> wrote:
>>>>>
>>>>> Hi,
>>>>> I am using appengine flexible environment.
>>>>>
>>>>> I am facing issues when there is a sudden spurt of requests to our
>>>>> system. (About 30 requests in a second). The server gets back to normal
>>>>> in
>>>>> 5-10 seconds (perhaps with the decrease
>>>>> in traffic again)
>>>>> Generally the requests come at about 5-8 per second.
>>>>>
>>>>> I am running with 2 instances.
>>>>>
>>>>>
>>>>> 2
>>>>> 10
>>>>> 120
>>>>>
>>>>> 0.7
>>>>>
>>>>>
>>>>>
>>>>> AFAICT the requests
[google-appengine] Dynamic compilation in appengine flexibile environment
How can we do dynamic compilation of Java in the appengine flexible environment. ToolProvider.getSystemJavaCompiler() returns null in the flexible runtime. Is there some other docker image I can use that has the jdk instead of just jre? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/6b9c47f2-f534-4c31-8e11-9828c8df0eee%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Re: Dynamic compilation in appengine flexibile environment
Thanks Nick. Do you have example of how when can write such a dockerfile. I am particularly curious about how to install the java jdk in a manner that is compatible with java-compat runtime build. I am using appengine flexible and need a dockerfile that is compatible with appengine runtime and at the same time adds in java sdk. On Fri, Oct 28, 2016 at 9:30 PM, 'Nick (Cloud Platform Support)' via Google App Engine wrote: > Hey Vinay, > > You can use the Dockerfile > <https://docs.docker.com/engine/reference/builder/> of a Custom Runtime > <https://cloud.google.com/appengine/docs/flexible/custom-runtimes/build> > based on a default image to install and configure javac on the system. This > would then enable you to run dynamic compilation. > > Let me know if you have any further questions, I'm here to help. > > Cheers! > > Nick > Cloud Platform Community Support > > On Tuesday, October 25, 2016 at 11:41:00 PM UTC-4, Vinay Chitlangia wrote: >> >> How can we do dynamic compilation of Java in the appengine flexible >> environment. >> >> ToolProvider.getSystemJavaCompiler() returns null in the flexible runtime. >> >> Is there some other docker image I can use that has the jdk instead of just >> jre? >> >> -- > You received this message because you are subscribed to a topic in the > Google Groups "Google App Engine" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/google-appengine/AXrpk-I9Bbw/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at https://groups.google.com/group/google-appengine. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/google-appengine/92ff045e-d53f-4293-8b91- > 1d96fd4247f0%40googlegroups.com > <https://groups.google.com/d/msgid/google-appengine/92ff045e-d53f-4293-8b91-1d96fd4247f0%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CADt9hj-8s8QgFcxxz%2BjSAd-%2BV%2BL5aw_rmqPD5NOut%2BgQ289JjQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Problems installing jdk8 in flex
I am facing some problem when trying to install openjdk-8 in the flex environment. Our setup is to use jetty9-compat runtime with openjdk-8 (We install jdk to do dynamic compilation). The setup was working fine till about 2-3 days back. Now when I try to install I get the following error: The following packages have unmet dependencies: openjdk-8-jdk : Depends: openjdk-8-jre (= 8u121-b13-1~bpo8+1) but 8u111-b14-2~bpo8+1 is to be installed Depends: openjdk-8-jdk-headless (= 8u121-b13-1~bpo8+1) but it is not going to be installed Our dockerfile is: FROM gcr.io/google_appengine/jetty9-compat RUN apt-get -q update && \ apt-get -y -q --no-install-recommends install openjdk-8-jdk && \ apt-get clean && \ rm /var/lib/apt/lists/*_* ADD . /app -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/db5794c6-be47-4b58-848e-1667765a49ad%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Re: Problems installing jdk8 in flex
Thanks Adam. In case anybody is facing a similar issue. This is the command that worked. apt-get -y -q --no-install-recommends install -t jessie-backports openjdk-8- jdk && \ On Saturday, February 4, 2017 at 3:05:38 AM UTC+5:30, Adam (Cloud Platform Support) wrote: > > It looks like openjdk-8-jre was updated in jessie-backports but > openjdk-8-jdk-headless wasn't updated in the same repository (by the Debian > maintainers). Two things you can try are: > > 1) Use the version from the 'stable' distribution in your Dockerfile: > apt-get -y -q --no-install-recommends install -t stable openjdk-8-jdk && \ > > > 2) Manually install the needed dependency before openjdk-8-jdk: > apt-get -y -q --no-install-recommends install openjdk-8-jdk-headless=8u121 > -b13-1~bpo8+1 && \ > > The second step would only work if that version existed in a repository, > and you had that repo in your /etc/apt/sources.list (eg. 'testing' or > 'unstable'), so further messing about with that may be necessary. > > On Friday, February 3, 2017 at 11:54:05 AM UTC-5, Vinay Chitlangia wrote: >> >> I am facing some problem when trying to install openjdk-8 in the flex >> environment. >> >> Our setup is to use jetty9-compat runtime with openjdk-8 (We install jdk >> to do dynamic compilation). The setup was working fine till about 2-3 days >> back. >> >> Now when I try to install I get the following error: >> >> >> The following packages have unmet dependencies: >> >> openjdk-8-jdk : Depends: openjdk-8-jre (= 8u121-b13-1~bpo8+1) but >> 8u111-b14-2~bpo8+1 is to be installed >> >> Depends: openjdk-8-jdk-headless (= 8u121-b13-1~bpo8+1) >> but it is not going to be installed >> >> >> >> Our dockerfile is: >> >> FROM gcr.io/google_appengine/jetty9-compat >> >> RUN apt-get -q update && \ >> >> apt-get -y -q --no-install-recommends install openjdk-8-jdk && \ >> >> apt-get clean && \ >> >> rm /var/lib/apt/lists/*_* >> >> ADD . /app >> > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/81f5116c-a398-439c-880b-6a4418127d9e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[google-appengine] 502 Bad Gateway error
Hi,
We are seeing intermittent occurrences of 502 Bad Gateway error in our
server.
About 0.5% requests fail with this error.
Out setup is:
Flex
F1 machine
1 server
Our request pattern is bursty. So the server gets ~30 requests in parallel.
The failures, when they happen are clustered, that is over a period of
10'ish seconds one would see 3-4 errors.
The requests which complete successfully, finish in 50-100 ms, so it does
not appear like the server is under major load and not able to keep up.
To rule out this possibility, I started the servers with 5 replicas.
However the failure percentage did not change.
>From the looks of it, it appears that there is some throttling or quota
issue at play. I tried tweaking max-concurrent-requests param. Set it to
300, but that did not make any difference either.
I do not see new instances being created at the time of failure either.
The request log for the failed request:
09:57:30.686POST502262 B4 msAppEngine-Google;
(+http://code.google.com/appengine; appid: s~village-test)/read
107.178.194.3 - - [07/Feb/2017:09:57:30 +0530] "POST /read HTTP/1.1" 502
262 - "AppEngine-Google; (+http://code.google.com/appengine; ms=4 cpu_ms=0
cpm_usd=2.9278e-8 loading_request=0 instance=-
app_engine_release=1.9.48 trace_id=-
{
protoPayload: {…}
insertId: "58994cb30002335cb47fd364"
httpRequest: {…}
resource: {…}
timestamp: "2017-02-07T04:27:30.686052Z"
labels: {…}
operation: {…}
}
Looking around at other logs at around the time of failure I see.
09:57:30.000[error] 32#32: *35107 recv() failed (104: Connection reset by
peer) while reading response header from upstream, client: 169.254.160.2,
server: , request: "POST /read HTTP/1.1", upstream:
"http://172.17.0.4:8080/read";, host: "bigtable-dev.appspot.com"
AFAICT this request never made it to our servlet.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit
https://groups.google.com/d/msgid/google-appengine/6b151171-e728-4a61-bdac-cd08b6a24580%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[google-appengine] 502 Bad gateway error
Hi,
We are seeing intermittent occurrences of 502 Bad Gateway error in our
server.
About 0.5% requests fail with this error.
Out setup is:
Flex running jetty9-compat
F1 machine
1 server
Our request pattern is bursty. So the server gets ~30 requests in parallel.
The failures, when they happen are clustered, that is over a period of
10'ish seconds one would see 3-4 errors.
The requests which complete successfully, finish in 50-100 ms, so it does
not appear like the server is under major load and not able to keep up.
To rule out this possibility, I started the servers with 5 replicas.
However the failure percentage did not change.
>From the looks of it, it appears that there is some throttling or quota
issue at play. I tried tweaking max-concurrent-requests param. Set it to
300, but that did not make any difference either.
I do not see new instances being created at the time of failure either.
The request log for the failed request:
09:57:30.686POST502262 B4 msAppEngine-Google; (+
http://code.google.com/appengine; appid: s~village-test)/read
107.178.194.3 - - [07/Feb/2017:09:57:30 +0530] "POST /read HTTP/1.1" 502
262 - "AppEngine-Google; (+http://code.google.com/appengine; ms=4 cpu_ms=0
cpm_usd=2.9278e-8 loading_request=0 instance=-
app_engine_release=1.9.48 trace_id=-
{
protoPayload: {…}
insertId: "58994cb30002335cb47fd364"
httpRequest: {…}
resource: {…}
timestamp: "2017-02-07T04:27:30.686052Z"
labels: {…}
operation: {…}
}
Looking around at other logs at around the time of failure I see.
09:57:30.000[error] 32#32: *35107 recv() failed (104: Connection reset by
peer) while reading response header from upstream, client: 169.254.160.2,
server: , request: "POST /read HTTP/1.1", upstream: "
http://172.17.0.4:8080/read";, host: "bigtable-dev.appspot.com"
AFAICT this request never made it to our servlet.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit
https://groups.google.com/d/msgid/google-appengine/2f5a728b-38ff-4de7-b260-32c1e15f5351%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[google-appengine] Re: Problems installing jdk8 in flex
On Wednesday, February 8, 2017 at 2:12:54 AM UTC+5:30, Cesar Oyarzun wrote: > > Hey Vinay I did try your comand but it doesn't work for me I'm getting > this error > > E: The value 'jessie-backports' is invalid for APT::Default-Release as > such a release is not available in the sources > There was one typo in my command (Note to self: Dont type when you can copy!!). Copy pasting the Dockerfile that worked for me. FROM gcr.io/google_appengine/jetty9-compat RUN apt-get -q update && \ apt-get -y -q --no-install-recommends -t jessie-backports install openjdk-8-jdk && \ apt-get clean && \ rm /var/lib/apt/lists/*_* ADD . /app > > On Monday, February 6, 2017 at 11:06:45 PM UTC-5, Vinay Chitlangia wrote: >> >> Thanks Adam. >> In case anybody is facing a similar issue. This is the command that >> worked. >> apt-get -y -q --no-install-recommends install -t jessie-backports openjdk >> -8-jdk && \ >> >> On Saturday, February 4, 2017 at 3:05:38 AM UTC+5:30, Adam (Cloud >> Platform Support) wrote: >>> >>> It looks like openjdk-8-jre was updated in jessie-backports but >>> openjdk-8-jdk-headless wasn't updated in the same repository (by the Debian >>> maintainers). Two things you can try are: >>> >>> 1) Use the version from the 'stable' distribution in your Dockerfile: >>> apt-get -y -q --no-install-recommends install -t stable openjdk-8-jdk && >>> \ >>> >>> >>> 2) Manually install the needed dependency before openjdk-8-jdk: >>> apt-get -y -q --no-install-recommends install openjdk-8-jdk-headless= >>> 8u121-b13-1~bpo8+1 && \ >>> >>> The second step would only work if that version existed in a repository, >>> and you had that repo in your /etc/apt/sources.list (eg. 'testing' or >>> 'unstable'), so further messing about with that may be necessary. >>> >>> On Friday, February 3, 2017 at 11:54:05 AM UTC-5, Vinay Chitlangia wrote: >>>> >>>> I am facing some problem when trying to install openjdk-8 in the flex >>>> environment. >>>> >>>> Our setup is to use jetty9-compat runtime with openjdk-8 (We install >>>> jdk to do dynamic compilation). The setup was working fine till about 2-3 >>>> days back. >>>> >>>> Now when I try to install I get the following error: >>>> >>>> >>>> The following packages have unmet dependencies: >>>> >>>> openjdk-8-jdk : Depends: openjdk-8-jre (= 8u121-b13-1~bpo8+1) but >>>> 8u111-b14-2~bpo8+1 is to be installed >>>> >>>> Depends: openjdk-8-jdk-headless (= 8u121-b13-1~bpo8+1) >>>> but it is not going to be installed >>>> >>>> >>>> >>>> Our dockerfile is: >>>> >>>> FROM gcr.io/google_appengine/jetty9-compat >>>> >>>> RUN apt-get -q update && \ >>>> >>>> apt-get -y -q --no-install-recommends install openjdk-8-jdk && \ >>>> >>>> apt-get clean && \ >>>> >>>> rm /var/lib/apt/lists/*_* >>>> >>>> ADD . /app >>>> >>> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/80f0fdd9-9c2f-4f4c-9dd6-09e1fb375d8e%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Re: 502 Bad gateway error
Might be related:
https://blog.percy.io/tuning-nginx-behind-google-cloud-platform-http-s-load-balancer-305982ddb340#.6k2laoada
The symptoms mentioned in this blog
Somewhat moderate requests
No logs
match our observations.
I do not see the "backend_connection_closed_before_data_sent_to_client"
status in the logs.
The error message for a failed request received by the client is:
11:12:44.549com.yawasa.server.storage.RpcStorageService LogError:
502 Bad Gateway502 Bad
Gatewaynginx (
RpcStorageService.java:137
<https://console.cloud.google.com/debug/fromlog?appModule=default&appVersion=1&file=RpcStorageService.java&line=137&logInsertId=589569d9000e7bf6825479e4&logNanos=1486186963359794000&nestedLogIndex=0&project=village-test>
)
The mention of nginx in the log message appears promising. We are not using
nginx deliberately, so I am assuming this is something happening under the
hood.
On Tuesday, February 7, 2017 at 11:08:55 AM UTC+5:30, Vinay Chitlangia
wrote:
>
> Hi,
> We are seeing intermittent occurrences of 502 Bad Gateway error in our
> server.
> About 0.5% requests fail with this error.
>
> Out setup is:
> Flex running jetty9-compat
> F1 machine
> 1 server
>
> Our request pattern is bursty. So the server gets ~30 requests in
> parallel.
> The failures, when they happen are clustered, that is over a period of
> 10'ish seconds one would see 3-4 errors.
>
> The requests which complete successfully, finish in 50-100 ms, so it does
> not appear like the server is under major load and not able to keep up.
> To rule out this possibility, I started the servers with 5 replicas.
> However the failure percentage did not change.
>
> From the looks of it, it appears that there is some throttling or quota
> issue at play. I tried tweaking max-concurrent-requests param. Set it to
> 300, but that did not make any difference either.
>
> I do not see new instances being created at the time of failure either.
>
>
> The request log for the failed request:
> 09:57:30.686POST502262 B4 msAppEngine-Google; (+
> http://code.google.com/appengine; appid: s~village-test)/read
> 107.178.194.3 - - [07/Feb/2017:09:57:30 +0530] "POST /read HTTP/1.1" 502
> 262 - "AppEngine-Google; (+http://code.google.com/appengine; ms=4
> cpu_ms=0 cpm_usd=2.9278e-8 loading_request=0 instance=-
> app_engine_release=1.9.48 trace_id=-
> {
> protoPayload: {…}
> insertId: "58994cb30002335cb47fd364"
> httpRequest: {…}
> resource: {…}
> timestamp: "2017-02-07T04:27:30.686052Z"
> labels: {…}
>
> operation: {…}
> }
>
> Looking around at other logs at around the time of failure I see.
> 09:57:30.000[error] 32#32: *35107 recv() failed (104: Connection reset by
> peer) while reading response header from upstream, client: 169.254.160.2,
> server: , request: "POST /read HTTP/1.1", upstream: "
> http://172.17.0.4:8080/read";, host: "bigtable-dev.appspot.com"
> AFAICT this request never made it to our servlet.
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit
https://groups.google.com/d/msgid/google-appengine/17ed09f4-1b8c-479b-a267-955f021c3a36%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Re: 502 Bad gateway error
On Wed, Feb 8, 2017 at 10:29 PM, 'Nicholas (Google Cloud Support)' via Google App Engine wrote: > Hey Vinay Chitlangia, > > Thanks for some preliminary troubleshooting and linking this interesting > article. App Engine runs Nginx processes to handle routes to your > application's handlers. Handlers serving static assets for instance are > handled by this Nginx process and the resources are served directly, thus > bypassing the application altogether to save on precious application > resources. > > The Nginx process will often serve a *502* if the application raises an > exception, an internal API call raises an exception or if the request > simply takes too long. As such, the status code by itself does not tell us > much. > > Looking at the GAE logs for your application, I found the *502*s you > mentioned. One thing I noticed is that they all occur from the */read* > endpoint. From the naming, I assume this endpoint is reading some data > from BigTable. Investigating further, perhaps you could provide some > additional information: > >- What exactly is happening at the */read* endpoint? A code sample >would be ideal if that's not too sensitive. > > As you surmised, we are reading some data from bigtable in this endpoint. > >- What kind of error handling exists in said endpoint if the BigTable >API returns non-success responses? > > The entire endpoint is in a try catch block catching Exception. In the case of failure the exception stack trace gets written to the logs. The first line of the endpoint is a log message signalling receiveing the request (this was done for this debugging of course!!) For the successful request the log message (the introductory one) gets written. For the 502 ones never. For requests that fail because of bigtable related errors, the logs have the stacktrace but not for 502s. The 502 failure requests finish in <10ms. > >- >- Can you log various steps in the */read* endpoint? This might help >identify the progress the request reaches before the *502* is served. >It would also help in confirming that your application is actually even >getting the request as I can't currently confirm that from the logs. > > My best guess is that the request does not make it to the servlet. The reason for that is that for the 100s of failed 502 logs that I have seen, not one has the log message, which is the absolute first line in the code of the read handler. > >- >- If said endpoint does in fact read from BigTable, what API and java >library are you using? > > we are using the google provided bigtable hbase1.2 jars version 0.9.4. > Regarding the article you linked, while the configuration of an HTTPS load > balancer and nginx.conf can be very important, both the load balancing > component and nginx.conf are out of the hands of the developer with App > Engine. Your scaling settings, health check settings and handlers in the > app.yaml are the only rules over which you have control that affect load > balancing and nginx rules. > > On Wednesday, February 8, 2017 at 11:27:43 AM UTC-5, Vinay Chitlangia > wrote: >> >> Might be related: >> https://blog.percy.io/tuning-nginx-behind-google-cloud-platf >> orm-http-s-load-balancer-305982ddb340#.6k2laoada >> >> The symptoms mentioned in this blog >> Somewhat moderate requests >> No logs >> >> match our observations. >> >> I do not see the "backend_connection_closed_before_data_sent_to_client" >> status in the logs. >> >> The error message for a failed request received by the client is: >> 11:12:44.549com.yawasa.server.storage.RpcStorageService LogError: >> 502 Bad Gateway> bgcolor="white">502 Bad Gateway> er>nginx (RpcStorageService.java:137 >> <https://console.cloud.google.com/debug/fromlog?appModule=default&appVersion=1&file=RpcStorageService.java&line=137&logInsertId=589569d9000e7bf6825479e4&logNanos=1486186963359794000&nestedLogIndex=0&project=village-test> >> ) >> >> The mention of nginx in the log message appears promising. We are not >> using nginx deliberately, so I am assuming this is something happening >> under the hood. >> >> On Tuesday, February 7, 2017 at 11:08:55 AM UTC+5:30, Vinay Chitlangia >> wrote: >>> >>> Hi, >>> We are seeing intermittent occurrences of 502 Bad Gateway error in our >>> server. >>> About 0.5% requests fail with this error. >>> >>> Out setup is: >>> Flex running jetty9-compat >>> F1 machine >>> 1 server >>> >>> Our request pattern is bursty. So the server ge
Re: [google-appengine] Re: 502 Bad gateway error
On Thu, Feb 9, 2017 at 7:52 PM, 'Nicholas (Google Cloud Support)' via Google App Engine wrote: > I realize that we've already begun investigating this here but I think > this would be most appropriate for the App Engine public issue tracker. > The issue is leading to an increasingly specific situation and I suspect > will require some exchange of code/project to reproduce the behavior you've > described. We monitor that issue tracker closely. > > When filing a new issue on the tracker, please link back to this thread > for context while posting a link to the issue here so that others in the > community can see the whole picture. > >- Be sure to include the latest logs for related to the *502*s. When >viewing the logs in Stackdriver Logging for instance, include *All >logs* rather than just *request_log* as *nginx.error*, *stderr*, >*stdout* and *vm.** logs may reveal clues as to a root cause. >- Mention if your are using any middleware like servlet filters that >may receive request before that actual handler >- Lastly, include what the CPU and/or memory usage looks like on the >instance(s) at the time of the 502s. Screenshots of *Utilization *and > *Memory >Usage* graphs from the Developers Console will likely be sufficient > > I look forward to this issue report. > https://code.google.com/p/googleappengine/issues/detail?id=13543 The logs are "All logs" around the time of the incident, however as a copy/paste from the browser. Couldnt retrieve any logs using gcloud beta logging read. This is the command I tried: gcloud beta logging read 'timestamp >= "2017-02-11T03:00:00Z" AND timestamp <="2017-02-12T03:05:00Z"' > > On Wednesday, February 8, 2017 at 1:24:01 PM UTC-5, Vinay Chitlangia wrote: >> >> >> >> On Wed, Feb 8, 2017 at 10:29 PM, 'Nicholas (Google Cloud Support)' via >> Google App Engine wrote: >> >>> Hey Vinay Chitlangia, >>> >>> Thanks for some preliminary troubleshooting and linking this interesting >>> article. App Engine runs Nginx processes to handle routes to your >>> application's handlers. Handlers serving static assets for instance are >>> handled by this Nginx process and the resources are served directly, thus >>> bypassing the application altogether to save on precious application >>> resources. >>> >>> The Nginx process will often serve a *502* if the application raises an >>> exception, an internal API call raises an exception or if the request >>> simply takes too long. As such, the status code by itself does not tell us >>> much. >>> >>> Looking at the GAE logs for your application, I found the *502*s you >>> mentioned. One thing I noticed is that they all occur from the */read* >>> endpoint. From the naming, I assume this endpoint is reading some data >>> from BigTable. Investigating further, perhaps you could provide some >>> additional information: >>> >>>- What exactly is happening at the */read* endpoint? A code sample >>>would be ideal if that's not too sensitive. >>> >>> As you surmised, we are reading some data from bigtable in this endpoint. >> >>> >>>- What kind of error handling exists in said endpoint if the >>>BigTable API returns non-success responses? >>> >>> The entire endpoint is in a try catch block catching Exception. In the >> case of failure the exception stack trace gets written to the logs. >> The first line of the endpoint is a log message signalling receiveing the >> request (this was done for this debugging of course!!) >> For the successful request the log message (the introductory one) gets >> written. For the 502 ones never. >> For requests that fail because of bigtable related errors, the logs have >> the stacktrace but not for 502s. >> The 502 failure requests finish in <10ms. >> >>> >>>- >>>- Can you log various steps in the */read* endpoint? This might >>>help identify the progress the request reaches before the *502* is >>>served. It would also help in confirming that your application is >>> actually >>>even getting the request as I can't currently confirm that from the logs. >>> >>> My best guess is that the request does not make it to the servlet. The >> reason for that is that for the 100s of failed 502 logs that I have seen, >> not one has the log message, which is the absolute first line in the code >> of the read handler. >> >>
[google-appengine] Life beyond compat
So I started moving the code to life after beyond compat. >From jetty9-compat to jetty9. The transition was alright except... the logs that are written as a part of the request come in stderr/stdout instead of in request log. What is the trick to make them part of the request log. Nothing seems to be getting traced now. What is the recommendation to hook up tracing? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/01ac3b46-68aa-4aa5-af70-7917264f25c8%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Re: Life beyond compat
And here is the dockerfile: (In addition to the one mentioned in the transition doc, we also install openjdk8 in this, and ofcourse add requestlog in the startup command). FROM gcr.io/google-appengine/jetty9 RUN apt-get -q update && \ apt-get -y -q --no-install-recommends -t jessie-backports install openjdk-8-jdk && \ apt-get clean && \ rm /var/lib/apt/lists/*_* ADD backend.war $JETTY_BASE/webapps/root.war WORKDIR $JETTY_BASE RUN java -jar $JETTY_HOME/start.jar --approve-all-licenses \ --add-to-startd=jmx,stats,hawtio,requestlog \ && chown -R jetty:jetty $JETTY_BASE On Sat, Mar 4, 2017 at 10:25 AM, Vinay Chitlangia wrote: > Thanks for the response. > Yes, I have followed the steps in the document > <https://cloud.google.com/appengine/docs/flexible/java/dev-jetty9> you > have mentioned. (You mentioned one of the step has a direct influence, > which step is that) > > I modified the docker file to add requestlog. However that did not seem to > have any impact. Attached is the screenshot of the log. > The message about memcache host and port is written in the readhandler > (and should have been part of the request log). > > The app.yaml is in src/main/appengine directory and fairly vanilla. > > runtime: custom > > env: flex > > handlers: > > - url: /.* > > script: this field is required, but ignored > > secure: always > > env_variables: > > USE_GAE_MEMCACHE: 1 > > On Fri, Mar 3, 2017 at 11:23 PM, 'George (Cloud Platform Support)' via > Google App Engine wrote: > >> How did you conduct the transition, step-by-step? >> >> One of the necessary configuration steps might have a direct influence on >> the issues you mention, namely: “Customizing the Java 8 / Jetty 9.3 >> runtime” from the “The Java 8 / Jetty 9.3 Runtime” document >> <https://cloud.google.com/appengine/docs/flexible/java/dev-jetty9>. >> >> More details, such that you deem relevant, may prove of help as well. >> >> -- >> You received this message because you are subscribed to a topic in the >> Google Groups "Google App Engine" group. >> To unsubscribe from this topic, visit https://groups.google.com/d/to >> pic/google-appengine/FfR1B38yUEo/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> google-appengine+unsubscr...@googlegroups.com. >> To post to this group, send email to google-appengine@googlegroups.com. >> Visit this group at https://groups.google.com/group/google-appengine. >> To view this discussion on the web visit https://groups.google.com/d/ms >> gid/google-appengine/64b6afe7-6f43-4123-9bb5-5bc13e1511b6% >> 40googlegroups.com >> <https://groups.google.com/d/msgid/google-appengine/64b6afe7-6f43-4123-9bb5-5bc13e1511b6%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CADt9hj89ss%2BsjeuFd8G8u2c0S_d_ZDruXFo9PCOz96NU1G9JPw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
[google-appengine] Cloud pubsub notification to all subscribers?
Hi, Wondering if the pubsub notification on a topic, goes to all subscribers listening to the topic or only one. Couldnt find a definitive answer in any doc yet. Suppose I have a topic say a) /topic/invalidate-local-cache. Now if I have servers with say 20 replicas, this is a notification that I would like to go to each of the 20 server. b) /topic/send-mail-to-user. In this case, I would want only one of the replica to receive the notification. Is it possible to setup different topics, some with a) and some with b) I am using flex, which may or may not impact the answer:) -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/0f0d060c-c77a-4460-bd6e-2ca5c97fe989%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Request logs and flex
Thanks for the responses. https://github.com/GoogleCloudPlatform/google-cloud-java/tree/master/google- cloud-logging#add-a-stackdriver-logging-handler-to-a-logger The link that you mention, says that logging can be setup using logging.properties. Wondering in what directory should logging.properties reside in the flex setup. On Wed, Apr 19, 2017 at 2:21 AM, 'Jisha Abubaker' via Google App Engine < google-appengine@googlegroups.com> wrote: > > Hi Vinay > > To get started, I'd recommend looking at this section on Stackdriver > logging using java.util.logging > https://github.com/GoogleCloudPlatform/google- > cloud-java/tree/master/google-cloud-logging#add-a- > stackdriver-logging-handler-to-a-logger > > It's on the page that Justin just shared. > We're also in the process of adding support with more logging frameworks > like Slf4j. > > Thanks > Jisha > > On Friday, April 14, 2017 at 11:25:29 PM UTC-7, Justin Beckwith wrote: >> >> Greetings folks! >> With the non-compat runtimes in the flexible environment, we rely on >> client logging libraries to make request correlation work. >> >> https://github.com/GoogleCloudPlatform/google-cloud-java/ >> tree/master/google-cloud-logging >> <https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2FGoogleCloudPlatform%2Fgoogle-cloud-java%2Ftree%2Fmaster%2Fgoogle-cloud-logging&sa=D&sntz=1&usg=AFQjCNHcbcABlfnD4ktBuJHiWSFJD9ndZA> >> >> Amir / Rajeev / Les should be able to help point you in the right >> direction from here :) >> >> On Fri, Apr 14, 2017 at 9:15 PM, Vinay Chitlangia >> wrote: >> >>> Hi, >>> Once we moved to non-compat flex, the logs emitted as part of the >>> request are coming out in stdout stream instead of with part of the request >>> (all logs >>> with a particular request, used to come together in the request logs of >>> compat runtime). >>> >>> Attached is the screenshot of the logs. The various WARNING logs are >>> those written as part of the request (which is the first line). >>> Notice that the logs are not bundled together with the request. >>> >>> The dockerfile we are using is: >>> >>> FROM gcr.io/google-appengine/jetty9 >>> >>> RUN apt-get -q update && \ >>> >>> apt-get -y -q --no-install-recommends -t jessie-backports install >>> openjdk-8-jdk && \ >>> >>> apt-get clean && \ >>> >>> rm /var/lib/apt/lists/*_* >>> >>> ADD backend.war $JETTY_BASE/webapps/root.war >>> >>> WORKDIR $JETTY_BASE >>> >>> RUN java -jar $JETTY_HOME/start.jar --approve-all-licenses \ >>> >>> --add-to-startd=jmx,stats,hawtio,requestlog \ >>> >>> && chown -R jetty:jetty $JETTY_BASE >>> >>> >>> And finally the app.yaml looks like: >>> >>> runtime: custom >>> >>> env: flex >>> >>> >>> handlers: >>> >>> - url: /.* >>> >>> script: this field is required, but ignored >>> >>> secure: always >>> >>> >>> resources: >>> >>> cpu: 2 >>> >>> memory_gb: 10 >>> >>> automatic_scaling: >>> >>> min_num_instances: 1 >>> >>> max_num_instances: 10 >>> >>> cool_down_period_sec: 120 >>> >>> cpu_utilization: >>> >>> target_utilization: 0.5 >>> >>> >>> env_variables: >>> >>> USE_GAE_MEMCACHE: 1 >>> >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Google App Engine" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to google-appengi...@googlegroups.com. >>> To post to this group, send email to google-a...@googlegroups.com. >>> Visit this group at https://groups.google.com/group/google-appengine. >>> To view this discussion on the web visit https://groups.google.com/d/ms >>> gid/google-appengine/9a919731-3f0a-4432-94c1-833c2024dbd6% >>> 40googlegroups.com >>> <https://groups.google.com/d/msgid/google-appengine/9a919731-3f0a-4432-94c1-833c2024dbd6%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> >> Jus
[google-appengine] Enabling gzip in an appengine-flex service
I am trying to enable gzip in our appengine-flex service. I followed the instructions in https://cloud.google.com/appengine/docs/flexible/java/dev-jetty9 The app.yaml has the following addition env_variables: JETTY_MODULES_ENABLE: 'gzip' However for requests that we make using URLConnection from another appengine module (standard appengine), the responses are not zipped. AFAICT the responses made from chrome are not zipped either. The responses from our service are "text/plain". We are using a custom runtime running java and jetty9. Here is the dockerfile. Thanks for your help. FROM gcr.io/google-appengine/jetty9 RUN apt-get -q update && \ apt-get -y -q --no-install-recommends -t jessie-backports install openjdk-8-jdk && \ apt-get -y -q --no-install-recommends install ssh sshpass netcat && \ apt-get clean && \ rm /var/lib/apt/lists/*_* ADD backend.war $JETTY_BASE/webapps/root.war WORKDIR $JETTY_BASE RUN java -jar $JETTY_HOME/start.jar --approve-all-licenses \ --add-to-startd=jmx,stats,hawtio,requestlog \ -Djava.util.logging.config.file=src/main/appengine/logging.properties \ && chown -R jetty:jetty $JETTY_BASE -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/eb94c68c-1d4e-4945-a150-9da716b8d28a%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
