It sounds like the root problem is "OOM errors fail to produce good log messages".
What CSV parser are you using? If you're on Java, most of them are crap. Jackson has a pretty good CSV plugin, but you probably still need to be very careful about streaming - it's really easy to blow up the heap with large CSV imports. I've had this problem before. Jeff On Tue, Aug 18, 2015 at 12:27 AM, Stefano Ciccarelli < stefano.ciccare...@mmbsoftware.it> wrote: > Nick, I completely disagree. > > The CSV import job is not so heavy because stream a file from GCS and > enqueue a task for every row. We designed that job with the GAE rules in > mind. Moreover that jobs are very few compared to the app traffic and are > handled by a backend module, but my instances restarts on the default > module. > When the app is used the instances restart *at the same time* every hour > or less. I still have no errors and I still have no evidence that our is > wrong. We are sure to have followed all the rules and all the docs. We > tried to configure the instance class to F4 but was the same. If the app > has very few traffic the instances last days but as the traffic rise the > instances begin restarting. > > I have not responded to the issue because I found it shallow and > offensive, seems that someone looked at the logs and seeing an url like > "importcsv" found the gold mine. The issue mention a warning in the log > about a concurrency issue as an evidence of our fault, well, the warning is > to log when our 'entity group limiter' start to limit the write rate on a > specific entity group to consume less resources. > > Now I'm on vacation, the instances restart, I'm very upset, in September I > will decide the role of GCP on our business. > > > Il giorno mar 18 ago 2015 alle 01:28 Nick (Cloud Platform Support) < > pay...@google.com> ha scritto: > >> For the record of any future users: an investigation of the issue >> reported by Stefano appeared to turn up that the cause of the instances >> shutting down was low instance specs combined with a memory-heavy CSV >> import job. >> >> Mauricio, I've located the support ticket you have open, and it appears >> that you're currently receiving help there for a similar but unrelated >> issue. I encourage you to follow up there. >> >> >> On Friday, August 7, 2015 at 7:11:49 AM UTC-4, Stefano Ciccarelli wrote: >>> >>> Hello to all! >>> >>> Our application has always suffered from high latencies due to instances >>> respawning continuously. >>> We always used custom settings in "automatic-scaling" to try to save >>> money. >>> >>> Following this post ( >>> http://googlecloudplatform.blogspot.it/2015/08/How-to-Troubleshoot-Latency-in-Your-App-Engine-Application.html) >>> I've decided to reset the "automatic-scheduling" settings of our app to >>> default values. >>> >>> So I've configured only min-idle-instances to 1 (otherwise the >>> /_ah/warmup handler is never called). The instance class is F2. >>> >>> After two days this is our state. >>> >>> So: >>> - why I have 4 instances 2 days old not used? (The logs says that one >>> instance served ~20 request today, but the other instances stopped to serve >>> yesterday) >>> - why the scheduler stop and spawn instances every 40/50 minutes if 4 >>> instances are there to do nothing? >>> - why the resident instance is restarted every 40/50 minutes if it is >>> supposed to be... resident? >>> - is there something wrong with our code? >>> >>> Thanks >>> >>> >>> >>> >>> [image: nimbus-image-1438944970211.png] >>> -- >>> *Stefano Ciccarelli* >>> GAE Application Division >>> / Director >>> stefano.ciccare...@mmbsoftware.it >>> >>> *M.M.B. s.r.l.* >>> via Granarolo, 177/7 - 48018 Faenza (RA) - Italy >>> tel. +39.0546.637711 - fax +39.0546.46077 >>> www.mmbsoftware.it - i...@mmbsoftware.it >>> >>> Le informazioni contenute in questa comunicazione sono riservate e >>> destinate esclusivamente alla/e persona/e o all'ente sopra indicati. E' >>> vietato ai soggetti diversi dai destinatari qualsiasi uso, copia, >>> diffusione di quanto in esso contenuto sia ai sensi dell'art. 616 c.p., sia >>> ai sensi del DL n. 196/03. Se questa comunicazione Vi e' pervenuta per >>> errore, Vi preghiamo di rispondere a questa e-mail e successivamente >>> cancellarla dal Vostro sistema. >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to google-appengine+unsubscr...@googlegroups.com. >> To post to this group, send email to google-appengine@googlegroups.com. >> Visit this group at http://groups.google.com/group/google-appengine. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/google-appengine/1ce70118-f593-4ea3-b6d3-47be766d36f2%40googlegroups.com >> <https://groups.google.com/d/msgid/google-appengine/1ce70118-f593-4ea3-b6d3-47be766d36f2%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- > *Stefano Ciccarelli* > GAE Application Division > / Director > stefano.ciccare...@mmbsoftware.it > > *M.M.B. s.r.l.* > via Granarolo, 177/7 - 48018 Faenza (RA) - Italy > tel. +39.0546.637711 - fax +39.0546.46077 > www.mmbsoftware.it - i...@mmbsoftware.it > > Le informazioni contenute in questa comunicazione sono riservate e > destinate esclusivamente alla/e persona/e o all'ente sopra indicati. E' > vietato ai soggetti diversi dai destinatari qualsiasi uso, copia, > diffusione di quanto in esso contenuto sia ai sensi dell'art. 616 c.p., sia > ai sensi del DL n. 196/03. Se questa comunicazione Vi e' pervenuta per > errore, Vi preghiamo di rispondere a questa e-mail e successivamente > cancellarla dal Vostro sistema. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at http://groups.google.com/group/google-appengine. > To view this discussion on the web visit > https://groups.google.com/d/msgid/google-appengine/CAArRkeUmDUsUchN-6wCaq4QQfiQmPVkyFG%3DtVVCNeEJ7jF96Rg%40mail.gmail.com > <https://groups.google.com/d/msgid/google-appengine/CAArRkeUmDUsUchN-6wCaq4QQfiQmPVkyFG%3DtVVCNeEJ7jF96Rg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CADK-0uj_%2B0qyBQ3OE9PtfSkxd%2B8HjrBo%2Be-k-dRkk6WwCWJyQg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
