[google-appengine] Re: Is there something like a "back_off" for ndb.transaction's?

[timh]

retrying https://pypi.python.org/pypi/retrying/1.3.3 provides decorators 
for different type of backoff retry scenarios.

However I would look at you data model to see why you  have such contention 
and what changes you could make to minimise this.

If you can't may also want to consider using a pull queue and use tasks to 
process the transactions

T


On Saturday, October 29, 2016 at 11:10:34 AM UTC+8, Kaan Soral wrote:
>
> Often, when 3-4 transaction's hit my app at the same time, even with 
> retries=6, they cause a contention exception
>
> My custom solution to the issue would be, use retries=0, but manually 
> implement a retry logic, like, sleep a random amount of time, retry
>
> So basically, when 3-4 transaction's occur consecutively, this solution 
> would un-tangle them under 60 seconds (assumption)
>
> I'm just wondering whether this is a good solution and whether there is an 
> existent solution I'm missing, basically, I want to be able to handle 5 
> consecutive transaction's
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/5e3ac51f-f964-45dd-8477-a795a4e5c640%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Is there something like a "back_off" for ndb.transaction's?

[Kaan Soral]

Often, when 3-4 transaction's hit my app at the same time, even with 
retries=6, they cause a contention exception

My custom solution to the issue would be, use retries=0, but manually 
implement a retry logic, like, sleep a random amount of time, retry

So basically, when 3-4 transaction's occur consecutively, this solution 
would un-tangle them under 60 seconds (assumption)

I'm just wondering whether this is a good solution and whether there is an 
existent solution I'm missing, basically, I want to be able to handle 5 
consecutive transaction's

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/789b10e1-48bf-4cca-9e45-aa780969e630%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: Dynamic compilation in appengine flexibile environment

[Vinay Chitlangia]

Thanks Nick.
Do you have example of how when can write such a dockerfile.
I am particularly curious about how to install the java jdk in a manner
that is compatible with java-compat runtime build.
I am using appengine flexible and need a dockerfile that is compatible with
appengine runtime and at the same time adds in java sdk.

On Fri, Oct 28, 2016 at 9:30 PM, 'Nick (Cloud Platform Support)' via Google
App Engine  wrote:

> Hey Vinay,
>
> You can use the Dockerfile
>  of a Custom Runtime
> 
> based on a default image to install and configure javac on the system. This
> would then enable you to run dynamic compilation.
>
> Let me know if you have any further questions, I'm here to help.
>
> Cheers!
>
> Nick
> Cloud Platform Community Support
>
> On Tuesday, October 25, 2016 at 11:41:00 PM UTC-4, Vinay Chitlangia wrote:
>>
>> How can we do dynamic compilation of Java in the appengine flexible
>> environment.
>>
>> ToolProvider.getSystemJavaCompiler() returns null in the flexible runtime.
>>
>> Is there some other docker image I can use that has the jdk instead of just 
>> jre?
>>
>> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Google App Engine" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/google-appengine/AXrpk-I9Bbw/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> google-appengine+unsubscr...@googlegroups.com.
> To post to this group, send email to google-appengine@googlegroups.com.
> Visit this group at https://groups.google.com/group/google-appengine.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/google-appengine/92ff045e-d53f-4293-8b91-
> 1d96fd4247f0%40googlegroups.com
> 
> .
>
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/CADt9hj-8s8QgFcxxz%2BjSAd-%2BV%2BL5aw_rmqPD5NOut%2BgQ289JjQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: AppEngine (AppInventor TinyWebDb application) access from a python local application

['George (Cloud Platform Support)' via Google App Engine]

Hello Christian,

Thank you for posting your issue here!

App Engine applications are meant to be accessed from a browser. The app 
itself might be written in Python. You can find such an example here 
.
 


If you want to access your data, such that contained in form fields on a 
web page, from a python program running on a PC, you may put to good use 
some of the many examples and tutorials on the web 
. 

Let us know if this addresses your concerns, and if you need more 
information on the App Engine. 

I hope this helps. Cheers!

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/ff6e0127-4dfb-4323-9617-0340eba839f7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: 403 Forbidden for some clients/unable to connect

[Evan Jones]

Sorry I should clarify: We have a slightly different symptom. Our affected 
customer gets timeouts connecting to 
https://(projectid).appspot.com/(static path), but it works fine if they 
use HTTP.



On Friday, October 28, 2016 at 5:09:21 PM UTC-4, Evan Jones wrote:
>
> Funny, I've been dealing with the same issue this week. We have a customer 
> who is on their corporate network, and they cannot access our static 
> resources if they use https://(projectid).appspot.com. If they use HTTP, 
> or our domain alias (https://www.bluecore.com/) it works. I haven't been 
> able to track down the cause, but there are minor certificate, DNS and IP 
> differences between *.appspot.com and ghs.googlehosted.com.
>
> This is a long winded way of saying: I've run into similar issues, I don't 
> know how to fix them, but try plain HTTP or a domain alias as potential 
> workarounds. Good luck, and if you figure out anything, please let me know.
>
>
>
> On Friday, October 28, 2016 at 3:08:41 PM UTC-4, Nick (Cloud Platform 
> Support) wrote:
>>
>> Hey xBy,
>>
>> This issue is a bit difficult to comment on in the abstract. There could 
>> be many reasons why a network link would experience failure. The best 
>> advice I can give is to use traceroute 
>>  or mtr 
>>  from their machines to 
>> the machine they wanted to connect to and this might help diagnose the 
>> network issue.
>>
>> Cheers,
>>
>> Nick
>> 
>>
>> On Monday, October 24, 2016 at 9:32:40 AM UTC-4, xBy Tez wrote:
>>>
>>> Hi everyone,
>>>
>>> Lately I've noticed that some of our clients email us saying they are 
>>> unable to connect to our Google App Engine domain. (It's CNAMEd to 
>>> ghs.googlehosted.com)
>>>
>>> This seems to happen for Russian people. Some will get a 403 Forbidden. 
>>>
>>> "Your client does not have permission to get URL (...) from this
>>> server. " is the error message they will get.
>>>
>>> Another Russian customer got "ERR::CONNECTION_RESET" errors when trying 
>>> to access our GAE domain.
>>>
>>> What should we do in order to resolve this and make this page available 
>>> for everyone?
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/cb986716-6c5d-410e-bd31-8faa82553df8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: 403 Forbidden for some clients/unable to connect

[Evan Jones]

Funny, I've been dealing with the same issue this week. We have a customer 
who is on their corporate network, and they cannot access our static 
resources if they use https://(projectid).appspot.com. If they use HTTP, or 
our domain alias (https://www.bluecore.com/) it works. I haven't been able 
to track down the cause, but there are minor certificate, DNS and IP 
differences between *.appspot.com and ghs.googlehosted.com.

This is a long winded way of saying: I've run into similar issues, I don't 
know how to fix them, but try plain HTTP or a domain alias as potential 
workarounds. Good luck, and if you figure out anything, please let me know.



On Friday, October 28, 2016 at 3:08:41 PM UTC-4, Nick (Cloud Platform 
Support) wrote:
>
> Hey xBy,
>
> This issue is a bit difficult to comment on in the abstract. There could 
> be many reasons why a network link would experience failure. The best 
> advice I can give is to use traceroute 
>  or mtr 
>  from their machines to the 
> machine they wanted to connect to and this might help diagnose the network 
> issue.
>
> Cheers,
>
> Nick
> 
>
> On Monday, October 24, 2016 at 9:32:40 AM UTC-4, xBy Tez wrote:
>>
>> Hi everyone,
>>
>> Lately I've noticed that some of our clients email us saying they are 
>> unable to connect to our Google App Engine domain. (It's CNAMEd to 
>> ghs.googlehosted.com)
>>
>> This seems to happen for Russian people. Some will get a 403 Forbidden. 
>>
>> "Your client does not have permission to get URL (...) from this
>> server. " is the error message they will get.
>>
>> Another Russian customer got "ERR::CONNECTION_RESET" errors when trying 
>> to access our GAE domain.
>>
>> What should we do in order to resolve this and make this page available 
>> for everyone?
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/fe318245-08c1-42d6-a907-4cd37cfc4ecf%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: Why resident instances in auto scaling are idle?

['Jordan (Cloud Platform Support)' via Google App Engine]

Hey Vidya.

You are correct that the instance start time is greatly based on your code, 
as each time a new instance is created it must load and prepare a fresh 
copy of your code to serve.

As for the reason why you are seeing a single instance handling the bulk of 
your requests, this comes down to the App Engine scheduler as you have 
mentioned. The scheduler will simply ask the first instance if it can 
handle a request. Based on your scaling configuration for pending latency 
and concurrent requests, your first instance will tell the scheduler that 
it can handle an extra request, and so it does; leaving the rest of your 
instances waiting to handle any overflow. 

If App Engine thinks you may need an extra instance warmed up just in case 
of overflow, it will create one. This is why you see a single Dynamic 
instance at the bottom handling no requests. Again, App Engine sends 
requests to Dynamic instances and not idle Resident instance. If there is 
no available Dynamic instance, your Resident Instance will be treated as a 
Dynamic instance and a new Resident Instance will be kicked up to meet your 
configured 
 
minimum idle instances.  

To configure your scaling options 
to 
force requests to be more spread across available instances, simply reduce 
the amount of concurrent requests a single instance is allowed to handle, 
reduce the minimum pending latency a request is allowed to wait in an 
instance's pending queue for, and reduce the max pending latency to force a 
request to be handled by a new instance after a period of time. Note, I 
would not recommend setting any of these to zero forcing each request to be 
handled by a single instance. This is because you still want multiple 
requests to be handled by each instance, to balance cost and performance. 

Continue to use the Stackdriver Trace  
tool to see the breakdown of latency for requests, and use this to 
configure the optimal scaling settings for your app so that requests are 
not waiting too long in a pending queue for other requests in front of it 
to finish. Ideally optimizing your code to execute requests very quickly in 
an asynchronous style (such as using the Task Queue to perform long image 
manipulation tasks instead of forcing a user to wait) will make your 
application scalable for Cloud computing. 

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/231d24e7-ac1c-4eba-bfb3-8fada9677094%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Changing location of google_appengine_launcher.ini

['George (Cloud Platform Support)' via Google App Engine]

Hi Grzegorz!
Thank you for posting your issue here!

There is no way of changing this user home location for the Google 
directory. The Google App Engine Launcher would re-create it each time it 
is launched, in exactly the same location, together with a new 
google_appengine_launcher.ini file in it. 
You may consider moving the directory and creating symbolic links to allow 
proper functioning of the Launcher, if moving this directory is a must. 
What is the main reason for wanting to get rid of this directory? Let me 
know if I may help further. 

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/74a62f18-069a-41ce-a779-9a683d252862%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: 403 Forbidden for some clients/unable to connect

['Nick (Cloud Platform Support)' via Google App Engine]

Hey xBy,

This issue is a bit difficult to comment on in the abstract. There could be 
many reasons why a network link would experience failure. The best advice I 
can give is to use traceroute  or 
mtr  from their machines to 
the machine they wanted to connect to and this might help diagnose the 
network issue.

Cheers,

Nick


On Monday, October 24, 2016 at 9:32:40 AM UTC-4, xBy Tez wrote:
>
> Hi everyone,
>
> Lately I've noticed that some of our clients email us saying they are 
> unable to connect to our Google App Engine domain. (It's CNAMEd to 
> ghs.googlehosted.com)
>
> This seems to happen for Russian people. Some will get a 403 Forbidden. 
>
> "Your client does not have permission to get URL (...) from this
> server. " is the error message they will get.
>
> Another Russian customer got "ERR::CONNECTION_RESET" errors when trying to 
> access our GAE domain.
>
> What should we do in order to resolve this and make this page available 
> for everyone?
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/c7c225f6-9fa4-4488-bb5d-f7a160bc9f18%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Backing up AppEngine

[Evan Jones]

To chime in on this: I agree with you that backups are important to protect 
against operator error. As a concrete example: we made a thankfully minor 
error with BigQuery, so now we periodically backup all our BigQuery tables.

The datastore backup tool is not great, but we do it for the same reason, 
and it does seem to work for us. Although now that I think about it: We 
haven't carefully manually checked our backups in a while, so I should go 
do that.

Worst case: We've discussed writing something ourselves multiple times, 
that would scan our Datastore and writes entities out in a more useful way, 
but we haven't prioritized it.



On Friday, October 28, 2016 at 5:07:13 AM UTC-4, Joshua Fox wrote:
>
>
>
> On Fri, Oct 28, 2016 at 12:17 AM, George (Cloud Platform Support) <
> gsuce...@google.com > wrote:
>
>> Hello Joshua,
>>
>> The necessity to backup data in your App Engine environment is not overly 
>> intense. A data protection policy 
>>  is in place, covering 
>> backups and all kind of risks to data in the cloud. If you backup your 
>> data, you do what has already been done for you in a thorough and 
>> systematic manner. 
>>
>
> But this data protection mechanism does not protect against team-member 
> error (nor hacker vandalism). Every team writes code and runs admin tools 
> to purposely delete and overwrite data. Although everyone  works hard to 
> avoid bugs and mistakes, we are all human. Backups are needed to deal with 
> that.
>
> My approach to this topic seems quite different from that of the creators 
> of GAE, and I am wondering if I am missing something fundamental. Are most 
> of the thousands of GAE customers, not to mention Google itself (as a GAE 
> "customer")  really doing without protection against such scenarios? The 
> Google Data Liberation Front showed a clear understanding of the need for 
> easy data exportability, yet that is missing here.
>
>  
>
>> There is still a legitimate need to control your data and download it, 
>> and tools are in place to backup and restore 
>> 
>>  
>> . If you would like to set up an object versioning policy for your app 
>> environment, cloud storage buckets allow that 
>> . 
>>
>
> That is only for Storage, not Datastore. (Of course it might not be 
> reasonable to expect that for Datastore, but the point is that versioning, 
>  which woul help meet my requirement, is not a possibility, unless I am 
> really missing something.)t  
>
>> Each tool individually offers ways to backup data.
>>
> Neither the Google Datastore Backup utility nor the Managed Backup utility 
> work; I have confirmed these bugs with Google support. Hopefully these bugs 
> will be fixed, and perhaps other users' datasets to not trigger the bugs, 
> but again this raises questions about how hard it is to backup data.
>  
> Storage can be backed up easily  with *cp*. 
>
> But there is absolutely no backup tool for Blobstore. (Is there?) Again, 
> do most customers write their own, or do they just leave their data in a 
> GAE Blobstore  and hope for the best?
>
> The backup process does not include values stored in Blobstore or Cloud 
>> Storage 
>> 
>> .
>>
>>
>
> I hope this helps. Cheers!
>>
>>
> Thank you. It is helping me understand the  philosophy behind the paucity 
> of backup tools, but I still don't get the difference in attitude towards 
> accidental deletion or overwrite.
>
> Joshua 
>
>> On Tuesday, October 25, 2016 at 2:29:17 AM UTC-4, Joshua Fox wrote:
>>>
>>> What do you do for backing up AppEngine data?
>>>
>>> From what I can see, there is no simple way to say "back up everything 
>>> in this GAE project" with a button-click or API call, nor even a way to do 
>>> this for most specific data-storage mechanisms.
>>>
>>> E.g., Datastore has a  backup tool which  has serious bugs that make it 
>>> inoperable. Blobstore has no backup mechanism.  
>>>
>>> Do major users of AppEngine not have off-Google backups? True, GAE deals 
>>> with disk failure, but you have to protect against hacker vandalism as well 
>>> as  team-member error. (One response I have received is "Don't make 
>>> mistakes." I won't bet on that.) 
>>>
>>> The Google Data Liberation Front 
>>>  seems to 
>>> have missed GAE. 
>>>
>>> So, do you make do without? Do you write your own tools to backup (and 
>>> regularly restore), where Google tools are lacking?
>>>
>>> Joshua
>>>
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to goog

[google-appengine] AppEngine (AppInventor TinyWebDb application) access from a python local application

[Christian Feniou]

Hello,

I built an application with App Inventor which uses AppEngine application. 
It work well with my Androïd application. 
Then, for certain usages, I'd like to access the data from a PC or other 
machine, thru a Python application.

Is anyone to tell me the way to do this access ? For example, here is a 
test AppEngine application : https://exe-chords.appspot.com/

If you try to access the above address from a browser, you'l get datas sent 
by AppEngine to an Index.html file like this :

(see the jointed file)

When I do the access from the AppInventor application, I receive the data 
directly into the application.

How to do this direct access from a Python application ?

Thank you per advance for any answer !

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/cc02337c-40f6-455b-99ea-d43aa5e249ae%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Title: App Inventor Database







 Search database for a tag

   Tag:
   
   

   
Returned as value to TinyWebDB component:
   {{result}}

  
Store a tag-value pair in the databse

	   Tag: 
	   Value: 
	   
	   
	
 

	
	  
		 Key
		 Value
		 Created (GMT)
	  
	  {% for entry in entryList %}
	  
	 {{entry.tag}}
		{{entry.value}}
		{{entry.date}}
  
	  	
		
		
			
		
  
	  {% endfor %}

	

[google-appengine] Re: Dynamic compilation in appengine flexibile environment

['Nick (Cloud Platform Support)' via Google App Engine]

Hey Vinay,

You can use the Dockerfile 
 of a Custom Runtime 
 
based on a default image to install and configure javac on the system. This 
would then enable you to run dynamic compilation.

Let me know if you have any further questions, I'm here to help.

Cheers!

Nick
Cloud Platform Community Support

On Tuesday, October 25, 2016 at 11:41:00 PM UTC-4, Vinay Chitlangia wrote:
>
> How can we do dynamic compilation of Java in the appengine flexible 
> environment.
>
> ToolProvider.getSystemJavaCompiler() returns null in the flexible runtime.
>
> Is there some other docker image I can use that has the jdk instead of just 
> jre?
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/92ff045e-d53f-4293-8b91-1d96fd4247f0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Backing up AppEngine

['George (Cloud Platform Support)' via Google App Engine]

Hello Joshua,

Leaving data in the Blobstore and hoping for the best, as you say, is not 
such an unreasonable policy: as mentioned, there is a standards-validated data 
protection policy  in place, 
covering backups and all kind of risks to data in the cloud, including what 
you mention about hacker interference. 

Your view on versioning and team member error seems to be influenced by a 
code development environment mindset. GAE is not specifically built for 
developing code on-line, you may refer to specialized third-party tools for 
that, for instance git . These tools take care of 
versioning, and are able to revert data to a healthy previous state in case 
of team-member/human error. There are in fact versioning features offered 
for the instances running in the cloud; please refer to the “Versioning and 
instances” paragraph in the overview of the app engine 
. 

One can find a Backing up or restoring data chapter on the Managing 
Datastore from the Console 
 
page.  

You are perfectly right about the Blobstore, the backup process does not 
include values stored in Blobstore or Cloud Storage 
.
 
Applications do not create or modify blob data directly; instead, blobs are 
created indirectly, by a submitted web form or other HTTP POST request 
. This means data 
is not modified within Blobstore workings, so it maintains identity with 
the original data uploaded by the user. There is no immediate need to 
backup cloud data that is already stored in identical state in your system.

Some tools may not yet work 100% as expected. You already submitted bugs 
for that, and we hope these issues are going to be address within 
reasonable timeframes. 

In short,  the difference in attitude towards accidental deletion or 
overwrite originates in our confidence in the highest standards implemented 
by our data protection policy . 


>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/744ca04f-3117-4a7b-9739-dbf8cbc484c8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Endpoints v2 InconsistentApiConfigurationException

['Nick (Cloud Platform Support)' via Google App Engine]

Hey Thomas,

Thanks for your persistent reporting here. I've now managed to reproduce 
the behaviour and I've made a Public Issue Tracker thread 
 to 
follow this for anybody along with you who is also interested in its 
progress. 

As a workaround for now, simply renaming one of the APIs will remove the 
issue. Of course your pattern of using ApiClass for differentiated 
properties in a multiclass API should be defined as the docs say 
,
 
for now this will allow you to circumvent the issue.

In future, don't hesitate to make a post like this to the Public Issue 
Tracker  if you 
suspect the error is not in your code but the platform. We'll be happy to 
assist!

Cheers,

Nick
Cloud Platform Community Support

On Wednesday, October 12, 2016 at 9:39:06 AM UTC-4, Thomas Wiradikusuma 
wrote:
>
> Sorry I forgot to attach the project.
>
> I'm sorry for not making it clear. The sample code is _working_, you need 
>> to migrate it to Endpoints v2 as mentioned in the README.md to make it not 
>> working. Anyway, for your convenience, I've attached the migrated version 
>> (which is not working).
>>
>>
> Regards,
> Thomas Wiradikusuma 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/1a077a15-325a-4ebf-b027-bf8f7357a4df%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: Why resident instances in auto scaling are idle?

[Vidya]

I think we were talking about orthogonal points to an extent here. There
are two separate components:

1. Why does it take so long to startup an instance?

The response here is that we need to work on our app startup times and
that's fair. I am positive we have room for optimization there, although
there will come a point where the tradeoff is really about using external
libraries vs getting down to the low level APIs - and we need to make that
an acceptable 2016 coding solution without needing to go back to circa 2000
:)

2. While the new instance is being brought up, why aren't requests being
served by the resident instance and why do we always see the resident
instance to be idle?

This is a question I haven't seen an answer to yet. Really, this is the
bigger question, since even if we brought down our app startup times to
say, 5 seconds, that's still unacceptable user latencies (and an average of
5s will imply enough variances that reach 2-3x that).

Now, after this discussion, I dug up a bunch of things and have learned a
thing or two about how the GAE schedulers *may* be working. A particularly
interesting thread I found was this -
https://groups.google.com/forum/#!topic/google-appengine/sA3o-PTAckc%5B26-50%5D
.

Based on my understanding so far, it looks like new requests will be routed
to the new instance as soon as it is "instantiated", without necessarily
waiting for it to actually be up (or they may be routed based on some
thresholds of the request rates). Obviously, this will put the instance
startup latencies right in the path of the user response times.

If this is indeed true, this is going to be a terrible experience for
anyone that is not operating at, well, essentially Google scale :). I can
imagine that this works very well at massive scale (and that may be the
great benefit of using GAE) - but, an app wouldn't live to see that day if,
in its initial days, it is providing a sucky UX due to long response times.

Our own experience matches this explanation - with min pending latency
values of 30ms (we left it at default), a new instance is getting spun up
the minute there are any requests in the queue whatsoever - which means the
new requests are routed immediately to the new instance, while the resident
instance remains idle practically all the time.

I would imagine that the scheduler would adapt to particular app startup
times + total number of operating instances to determine when to move new
requests over to a new instance.

Given little to no official information about how the scheduler works, I'd
like to understand if the experimentation and observations that the GAE
community has developed is in fact correct - otherwise, we're going to be
launching into a mini manual Monte Carlo simulation to really tune the
knobs that may work for our case. The risk here, of course, is that as soon
as the operating parameters change for our app, we need to be re-running
the simulation.

I have to say that the GAE docs leave a lot to be desired. For the number
of people that have suffered through this topic, one would hope by now that
there are more insights on what's actually going on and how to maximize the
efficiency of app engine.

Unfortunately, "optimize your app's startup times" is a necessary but
insufficient answer. As I wrote, unless we're talking 100ms app startup
times, in today's expected UX metrics, we aren't going very far with this.

All that said, we're currently starting to experiment with longer min
pending time values to see if that causes the resident instances to kick
in. It looks like some people have had luck with that. This will obviously
cause problems at scale, but, if it solves our problem at the small scales
now, we'd start there and change it as we scale up.

One more hack to our list there - but, if you have better suggestions, I'd
like to know.

Thanks,
Vidya

On Thu, Oct 27, 2016 at 6:59 PM, Nick  wrote:

> A few years ago I came across a suggestion that you can improve startup
> time by including you war class files in a jar, implying that maybe the war
> is loaded in an exploded manner. Not sure if this is true or relevant, I've
> never bothered. You could test it out.
>
> I've never seen resident instance work or do anything, I always observe
> cold starts on requests I make even when a resident instance is running.
> Quite often you'll also see the full load being born by one or two
> instances while others lie around for a long time only serving one or two
> requests. It would be great if someone had time to play and understand
> practically what matters.
>
> I found it interesting that resident instances are supposed to be
> converted to dynamic though - I never noticed that.
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "Google App Engine" group.
> To unsubscribe from this topic, visit https://groups.google.com/d/
> topic/google-appengine/0ZBLsyc51gk/unsubscribe.
> To unsubscribe from this group and all its to

[google-appengine] Changing location of google_appengine_launcher.ini

[Grzegorz Rekowski]

Hi,

By default Google appengine launcher is saving data in HOME deirectory 
typically: C:\Users\username\Google\

In this folder there are two files: google_appengine_launcher.ini 
and google_appengine_projects.ini



Is there a way to change this default location?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/331127ad-91f3-4738-bf0c-f369308450de%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Backing up AppEngine

[Joshua Fox יהושע פוקס]

On Fri, Oct 28, 2016 at 12:17 AM, George (Cloud Platform Support) <
gsuceve...@google.com> wrote:

> Hello Joshua,
>
> The necessity to backup data in your App Engine environment is not overly
> intense. A data protection policy
>  is in place, covering
> backups and all kind of risks to data in the cloud. If you backup your
> data, you do what has already been done for you in a thorough and
> systematic manner.
>

But this data protection mechanism does not protect against team-member
error (nor hacker vandalism). Every team writes code and runs admin tools
to purposely delete and overwrite data. Although everyone  works hard to
avoid bugs and mistakes, we are all human. Backups are needed to deal with
that.

My approach to this topic seems quite different from that of the creators
of GAE, and I am wondering if I am missing something fundamental. Are most
of the thousands of GAE customers, not to mention Google itself (as a GAE
"customer")  really doing without protection against such scenarios? The
Google Data Liberation Front showed a clear understanding of the need for
easy data exportability, yet that is missing here.



> There is still a legitimate need to control your data and download it, and
> tools are in place to backup and restore
> 
> . If you would like to set up an object versioning policy for your app
> environment, cloud storage buckets allow that
> .
>

That is only for Storage, not Datastore. (Of course it might not be
reasonable to expect that for Datastore, but the point is that versioning,
 which woul help meet my requirement, is not a possibility, unless I am
really missing something.)t

> Each tool individually offers ways to backup data.
>
Neither the Google Datastore Backup utility nor the Managed Backup utility
work; I have confirmed these bugs with Google support. Hopefully these bugs
will be fixed, and perhaps other users' datasets to not trigger the bugs,
but again this raises questions about how hard it is to backup data.

Storage can be backed up easily  with *cp*.

But there is absolutely no backup tool for Blobstore. (Is there?) Again, do
most customers write their own, or do they just leave their data in a GAE
Blobstore  and hope for the best?

The backup process does not include values stored in Blobstore or Cloud
> Storage
> 
> .
>
>

I hope this helps. Cheers!
>
>
Thank you. It is helping me understand the  philosophy behind the paucity
of backup tools, but I still don't get the difference in attitude towards
accidental deletion or overwrite.

Joshua

> On Tuesday, October 25, 2016 at 2:29:17 AM UTC-4, Joshua Fox wrote:
>>
>> What do you do for backing up AppEngine data?
>>
>> From what I can see, there is no simple way to say "back up everything in
>> this GAE project" with a button-click or API call, nor even a way to do
>> this for most specific data-storage mechanisms.
>>
>> E.g., Datastore has a  backup tool which  has serious bugs that make it
>> inoperable. Blobstore has no backup mechanism.
>>
>> Do major users of AppEngine not have off-Google backups? True, GAE deals
>> with disk failure, but you have to protect against hacker vandalism as well
>> as  team-member error. (One response I have received is "Don't make
>> mistakes." I won't bet on that.)
>>
>> The Google Data Liberation Front
>>  seems to
>> have missed GAE.
>>
>> So, do you make do without? Do you write your own tools to backup (and
>> regularly restore), where Google tools are lacking?
>>
>> Joshua
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/CABfTu2Ce%3DT3MSTEfQ2f-1X99NQMvL5STh9f2NO9p2FyMRv2GmQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.