Thank you George for the response. In our case, There is a single endpoint which reads a large file from amazon s3 line by line and then ingests a batch of 50 lines as 50 documents on cloud firestore. It roughly takes around 1.5 hours to insert all the data from a file to cloud firestore. Given that instance uptime is not guaranteed, is compute engine a good solution to perform long running task? Or is there any other option you can recommend?
On the flex environment, we tried to first download the whole file on a disk and then start reading line by line from it which has worked for us and not observed abrupt termination. But wanted to know an opinion around what should be the best option to perform long running task on google cloud platform. On Friday, August 9, 2019 at 11:39:00 PM UTC+5:30, George (Cloud Platform Support) wrote: > > Instance re-starting is normal in the present case, and your application > should take this into consideration. Hardware and software failures that > cause early termination or frequent restarts can occur without prior > warning and can take considerable time to resolve; thus, you should > construct your application in a way that tolerates these failures. You may > find related detail on the "How Instances are Managed" documentation page > <https://cloud.google.com/appengine/docs/standard/python/how-instances-are-managed#instance_uptime>. > > Some good strategies for avoiding downtime are listed on that page. You may > also consider changing your app.yaml with line basic_scaling: instances: 2, > or more than 1. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/76d363ed-91cd-40d8-a54d-56c33e6899df%40googlegroups.com.
