Re: [google-appengine] Datastore NDB migration?
You can add a default value, if that is enough. If not, mapreduce works great. If you have too many namespaces, pipeline can roll across the namespaces and launch mapreduces. Jacob On Dec 11, 2014 11:48 PM, "John Louis Del Rosario" wrote: > I mean migrating a record's "schema", or just setting a field's value for > all existing records. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at http://groups.google.com/group/google-appengine. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Can Application Settings anyhow guarantee me exactly 2 instances?
One suggestion...I would check out the modules API. If you make everything a module, you could potentially set it to manual scaling and pick 2 instances. If I am correct, this would mean you always had 2, but never more. On Sat, Mar 15, 2014 at 6:15 PM, Barry Hunter wrote: > > > > On 16 March 2014 00:59, PK wrote: > >> My understanding is that you might see more than 2 but you will not be >> charged for more than 2 with these settings. >> > > You wont be charged for more than 2 *idle* instances. > > But if the scheduler, spins up many instances, and they are serving > traffic (however small) they are chargeable. > > min_idle_instances says appengine should always aim to keep this number of > *deliberately* idle instances. So it will spin up enough to keep at least > two idle at any time. (and because of the max_idle_instances, it shouldnt > grow much *beyond* 2, and if it does wont be charged) > > > (an idle instance, doesnt normally serve traffic, but in the case of a > sudden spike, it should be able to serve a few, while new instances are > being spun up. Once they are spun up, idle instances will go back to well, > being idle) > > > ... there simply isnt a max_instances, setting. > > > >> >> PK >> http://www.gae123.com >> >> On March 15, 2014 at 5:28:11 PM, Aron Gombas (aron.gom...@gmail.com) >> wrote: >> >> Hi! >> >> I have an important, low-traffic, internal app. There are typically 0-2 >> concurrent users, the "peak" would be maybe 5-6. Plus, there are some cron >> jobs and warmup requests that are also needs to be served, but I believe 1 >> or 2 resident instances should be able to handle all traffic easily. >> >> As an experiment, what I want to completely eliminate is loading >> requests, after those 2 instances have been started. This is Java app, and >> the 10-15s additional response time caused by loading requests totally >> kills the experience. So, I'm happy to pay for the idle time, but I'd like >> to be sure that new instances are never started, and I pretty much bypass >> the heuristics and auto-scaling of the Scheduler. >> >> Is this possible at all? >> >> These are my current settings: >> >> min_idle_instances: 2 >> max_idle_instances: 2 >> min_pending_latency: 10ms >> max_pending_latency: 15.0s >> >> With these, when I look at the instances I can see 2 resident plus 2 >> dynamic instances, although I'm the only user currently using the app. >> (There are short cron jobs and occasional warmup requests.) >> >> Cheers, >> >> Aron >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to google-appengine+unsubscr...@googlegroups.com. >> To post to this group, send email to google-appengine@googlegroups.com. >> Visit this group at http://groups.google.com/group/google-appengine. >> For more options, visit https://groups.google.com/d/optout. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to google-appengine+unsubscr...@googlegroups.com. >> To post to this group, send email to google-appengine@googlegroups.com. >> Visit this group at http://groups.google.com/group/google-appengine. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at http://groups.google.com/group/google-appengine. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/d/optout.
Re: [google-appengine] Show and Tell: Plexi Voice a Windows Phone 8 App with AppEngine backend.
http://groups.google.com/group/google-appengine. > For more options, visit https://groups.google.com/groups/opt_out. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at http://groups.google.com/group/google-appengine. > For more options, visit https://groups.google.com/groups/opt_out. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at http://groups.google.com/group/google-appengine. > For more options, visit https://groups.google.com/groups/opt_out. > -- Jacob Taylor Co-Founder and CTO, Kahuna (650) 302-1864 www.useKahuna.com <http://www.usekahuna.com/> http://www.linkedin.com/in/thejacobtaylor -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: Snapchat
gt;>>>> system-to-system (B2B type) transaction volumes. >>>>>>>>>> >>>>>>>>>> When we created our infrastructure for these systems we needed >>>>>>>>>> multiple geographically dispersed data centers, high levels of >>>>>>>>>> fault-tolerance within any given data center, n-tier architecture, >>>>>>>>>> secure >>>>>>>>>> systems, scalable databases and front-end servers, system, security >>>>>>>>>> and >>>>>>>>>> network monitoring and administration, etc. When you spec that all >>>>>>>>>> out >>>>>>>>>> from scratch, you will have a hard time doing it for less than >>>>>>>>>> several >>>>>>>>>> hundred thousand dollars capex with big ongoing opex expense. Any >>>>>>>>>> growth >>>>>>>>>> beyond your initial headroom will require additional capex >>>>>>>>>> expenditure and >>>>>>>>>> incremental ongoing opex. >>>>>>>>>> >>>>>>>>>> Depending on the profile of your application and the system load, >>>>>>>>>> at some point you will pass the threshold of it being cheaper to >>>>>>>>>> build and >>>>>>>>>> maintain your own equivalent infrastructure, but that threshold is >>>>>>>>>> very, >>>>>>>>>> very high. So it makes me think people who say GAE is 'expensive' >>>>>>>>>> are not >>>>>>>>>> making a comparison such as this. Maybe they don't really need >>>>>>>>>> everything >>>>>>>>>> that GAE offers. >>>>>>>>>> >>>>>>>>>> Or perhaps they are comparing GAE to other cloud offerings such >>>>>>>>>> as AWS? Amazon's pricing doesn't seem to be radically different than >>>>>>>>>> Google's to me, for similar services. And given that Amazon's PaaS >>>>>>>>>> solution is not yet as complete at GAE, I think that any complete >>>>>>>>>> appliation built on AWS is going to require some level of >>>>>>>>>> system-engineering. System engineers are not cheap. One of the >>>>>>>>>> things we >>>>>>>>>> like about GAE is that, at this point in our corporate evolution, we >>>>>>>>>> can >>>>>>>>>> focus entirely on our Customers and our Software and not spend money >>>>>>>>>> or >>>>>>>>>> time configuring hardware, OS and other "low level" stuff that we (as >>>>>>>>>> application software guys) don't want to mess with. There are very >>>>>>>>>> real >>>>>>>>>> hard and soft monetary benefits to this. >>>>>>>>>> >>>>>>>>>> Or maybe when people say "expensive" they mean as compared to >>>>>>>>>> other "cloud" offerings that are more along the lines of rented >>>>>>>>>> physical or >>>>>>>>>> virtual machines. Yes, some of these can be cheap compared to GAE. >>>>>>>>>> But >>>>>>>>>> these are really apples-to-oranges comparisons when you consider all >>>>>>>>>> the >>>>>>>>>> things you need to provision a global, "utility-grade" >>>>>>>>>> (aspirationally, >>>>>>>>>> anyway) SaaS offering. >>>>>>>>>> >>>>>>>>>> So I guess this post is a long-winded way of me saying "GAE >>>>>>>>>> Expensive? Really? What exactly do you mean by that? Compared to >>>>>>>>>> what?" >>>>>>>>>> >>>>>>>>>> On Monday, January 20, 2014 4:19:54 AM UTC-6, coto wrote: >>>>>>>>>>> >>>>>>>>>>> We all should be surprised, because Google App Engine is very >>>>>>>>>>> expensive!! >>>>>>>>>>> >>>>>>>>>>> On Sunday, January 19, 2014 5:23:13 AM UTC-3, alex wrote: >>>>>>>>>>>> >>>>>>>>>>>> Why were you surprised? >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "Google App Engine" group. >>>>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>>>> send an email to google-appengi...@googlegroups.com. >>>>>>>>>> To post to this group, send email to google-a...@googlegroups.com >>>>>>>>>> . >>>>>>>>>> Visit this group at http://groups.google.com/group >>>>>>>>>> /google-appengine. >>>>>>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "Google App Engine" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to google-appengi...@googlegroups.com. >>>>>>> To post to this group, send email to google-a...@googlegroups.com. >>>>>>> Visit this group at http://groups.google.com/group/google-appengine. >>>>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>>>> >>>>>> >>>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Google App Engine" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to google-appengi...@googlegroups.com. >>>>> To post to this group, send email to google-a...@googlegroups.com. >>>>> Visit this group at http://groups.google.com/group/google-appengine. >>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>> >>>> >>>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Google App Engine" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to google-appengi...@googlegroups.com. >>> To post to this group, send email to google-a...@googlegroups.com. >>> Visit this group at http://groups.google.com/group/google-appengine. >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >> >> -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at http://groups.google.com/group/google-appengine. > For more options, visit https://groups.google.com/groups/opt_out. > -- Jacob Taylor CTO, Kahuna (650) 302-1864 www.useKahuna.com <http://www.usekahuna.com/> http://www.linkedin.com/in/thejacobtaylor<http://www.linkedin.com/in/adammarchick> You -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: MapReduce Roadmap
r seeing >>>> issues staying in NEW state for months concerns me the most about how >>>> active the effort is. >>>> >>>> (examples of such bugs: >>>> e.g. Issue >>>> 182<https://code.google.com/p/appengine-mapreduce/issues/detail?id=182> >>>> mapreduce/include.yaml doesn't work with python2.7 and thread >>>> safe<https://code.google.com/p/appengine-mapreduce/issues/detail?id=182&can=5&colspec=ID%20Type%20Status%20Priority%20Component%20Owner%20Summary> >>>> or Issue >>>> 203<https://code.google.com/p/appengine-mapreduce/issues/detail?id=203> >>>> map reduce is broken since r534) >>>> >>>> Thanks, >>>> PK >>>> >>>> On December 5, 2013 at 8:27:33 PM, Chris Ramsdale (cram...@google.com) >>>> wrote: >>>> >>>> PK, >>>> >>>> We're definitely planning on moving mapreduce to GA. Plan is to get the >>>> API finalized and then move it through the std Preview => GA channel. >>>> >>>> Questions for you: >>>> >>>> - how would prefer to access the library itself? >>>> >>>> - is there something about having it outside of the SDK that causes >>>> substantial friction? >>>> >>>> - or, is it the fact that it's sat in experimental for way too long >>>> that is the larger concern? >>>> >>>> -- Chris >>>> >>>> Product Manager, Google App Engine >>>> >>>> -- Chris >>>> On Dec 5, 2013 6:12 PM, "PK" wrote: >>>> >>>>> I am resending this in case the right Product Manager(s) at Google >>>>> missed my question. >>>>> >>>>> Thanks, >>>>> PK >>>>> http://www.gae123.com >>>>> >>>>> On November 25, 2013 at 12:53:44 AM, PK (p...@gae123.com) wrote: >>>>> >>>>> For a very very long time MapReduce has not been integrated with >>>>> the GAE SDK and remains experimental (https://developers.google. >>>>> com/appengine/docs/python/dataprocessing/). >>>>> >>>>> Could somebody shed some light on the roadmap plan? >>>>> >>>>> Thanks, >>>>> PK >>>>> http://www.gae123.com >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "Google App Engine" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to google-appengi...@googlegroups.com. >>>>> To post to this group, send email to google-a...@googlegroups.com. >>>>> Visit this group at http://groups.google.com/group/google-appengine. >>>>> For more options, visit https://groups.google.com/groups/opt_out. >>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Google App Engine" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to google-appengi...@googlegroups.com. >>>> To post to this group, send email to google-a...@googlegroups.com. >>>> Visit this group at http://groups.google.com/group/google-appengine. >>>> For more options, visit https://groups.google.com/groups/opt_out. >>>> >>>> >>> -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at http://groups.google.com/group/google-appengine. > For more options, visit https://groups.google.com/groups/opt_out. > -- Jacob Taylor CTO, Kahuna (650) 302-1864 www.useKahuna.com <http://www.usekahuna.com/> http://www.linkedin.com/in/thejacobtaylor<http://www.linkedin.com/in/adammarchick> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: major MEMCACHE changes started October 5th
>From my experience using dedicated memcache it is billed by GB stored. You are limited in transactions by GB stored as well. We have been using one GB for a little while and have not had any cache drops. I have loved being able to rely on it more than I could when it was shared. Shared cage is based on load, frequency of reads, and other people's usage. We had good success with it but quickly found out that short term rate limiting by tokens were not sticking around long enough. Now we have the space to keep the locks around for days. We are putting millions of items in the cache though. Jacob On Oct 8, 2013 7:27 PM, "Vinny P" wrote: > On Tue, Oct 8, 2013 at 12:24 PM, James Gilliam > wrote: > >> Memcache continues to perform poorly; I suspect they are making many >> changes to it. >> >> Saturday was awful -- aggressively purging memcache, Sunday was better, >> Monday was worse than Sunday. >> >> At this point, not sure what to do ... are we dealing with just code >> instability or some fundamental change? >> >> > > Can you try changing the instance class size to a larger class ( ie from > F1 to F2 ) and then back again? Sometimes applications get stuck with > "damaged" instances and experience reduced reliability. Changing the > instance class size forces the scheduler to reallocate new instances, which > will hopefully open on a more reliable machine/different place within the > physical Google infrastructure. > > > - > -Vinny P > Technology & Media Advisor > Chicago, IL > > App Engine Code Samples: http://www.learntogoogleit.com > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at http://groups.google.com/group/google-appengine. > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] SDK / Production _async behaviour differences - is _async's safe on production?
My understanding was that any async request that was not completed will attempt to abort if you return a response. It may or may not be stopped. We collect a pool of futures and then call wait as the last thing to make sure everything has finished. On Wed, Sep 11, 2013 at 9:30 AM, Kaan Soral wrote: > @ndb.toplevel solves ndb related async issues > > I've recently started using Task().add_async() and immediately discovered > that async calls are disregarded if you don't manually wait for the > response - on SDK > *WARNING 2013-09-11 16:17:18,006 recording.py:653] Found 1 RPC > request(s) without matching response (presumably due to timeouts or other > errors)* > You see the above message - and no results > > I've tested the same call on production, it works, the task executes > > I've modified recording.py to manually wait for those RPC's - it works > great - it forces the async calls to execute on SDK, if anyone wants I can > share the modification > > However the question is, are those calls 100% safe of production, can you > just add a Task with add_async and not worry about the task not executing? > > For example if you add 500 tasks with Task add_async, and just complete > the request without waiting or anything, is the execution rate 500/500 - > assuming no task exceptions? > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at http://groups.google.com/group/google-appengine. > For more options, visit https://groups.google.com/groups/opt_out. > -- Jacob Taylor CTO, Kahuna (650) 302-1864 www.useKahuna.com <http://www.usekahuna.com/> http://www.linkedin.com/in/thejacobtaylor<http://www.linkedin.com/in/adammarchick> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Do pull queues exhibit "consistent read" behaviour when leasing?
We have a nice little fibonacci sequence being calculated at the moment. Feels very dirty though. We also have multiple parallel threads where one might be backlogged and the other is idle. This means we slow down the backlogged thread. We are looking at changing the way we do some of this, but at the moment, our system actually performs better with more load. I was thinking about making a call to a non-responsive web server (setup one with a sleep) and use URL timeouts. Thanks, Jacob On Wed, Sep 11, 2013 at 11:28 AM, Vinny P wrote: > On Tue, Sep 10, 2013 at 9:57 PM, Jacob Taylor > wrote: > > Is there a safe way to sleep when using a multi-threaded python >> application? We were seeing huge issues when we had a few regular small >> sleep statements. i.e. common use cases that were sleeping for 1/4 second. >> >> > > Well, that's implementation dependent. Instead of doing sleep, you could > move time-consuming application logic to where you need a delay. For > instance, suppose you needed to do a urlfetch later on in your code. Why > not do a synchronous urlfetch in place of the delay? > > Or you could use a push task, set it to countdown a short time before > executing ( > https://developers.google.com/appengine/docs/python/taskqueue/tasks#Task see > the countdown option). The push task then notifies whatever service to > start leasing tasks from the pull queue. > > You could also fake it - for example, do a while loop that counts up to an > outrageously high number. It's a bit of a waste of instance time, but > whatever works! > > > > - > -Vinny P > Technology & Media Advisor > Chicago, IL > > App Engine Code Samples: http://www.learntogoogleit.com > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at http://groups.google.com/group/google-appengine. > For more options, visit https://groups.google.com/groups/opt_out. > -- Jacob Taylor CTO, Kahuna (650) 302-1864 www.useKahuna.com <http://www.usekahuna.com/> http://www.linkedin.com/in/thejacobtaylor<http://www.linkedin.com/in/adammarchick> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Logs Stored Data Quota not being reset
Google delete the logs only when it needs to in order to keep you within quota. They keep three kids as long as possible. Once you fill to the quotes, it stays full. Think of it as a first in first out queue that is emptied only when the quota is hit. It fills and stays full. Does that help? Jacob On Sep 11, 2013 2:12 AM, "Guillermo Mazzola" wrote: > Has been several weeks now that the Logs Stored Data quota does not reset > when the others does. It's always at 100%, event after the first minute of > the new day. Can anyone tell me why? > > PS: logs API never stop working and I can access the logs just fine. > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at http://groups.google.com/group/google-appengine. > For more options, visit https://groups.google.com/groups/opt_out. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Do pull queues exhibit "consistent read" behaviour when leasing?
Is there a safe way to sleep when using a multi-threaded python application? We were seeing huge issues when we had a few regular small sleep statements. i.e. common use cases that were sleeping for 1/4 second. Jacob On Tue, Sep 10, 2013 at 6:44 PM, Vinny P wrote: > On Tue, Sep 10, 2013 at 4:18 PM, Peter D wrote: > > If I add a task to a named pull queue synchronously, and after the enqueue >> operation I lease from that queue immediately (locally or via a notified >> backend), is it guaranteed that the task will be there? >> >> Or is it similar to the datastore, in that after adding a task, it may or >> may not be there for a while when attempting to lease? >> >> > > App Engine needs a bit of time between adding a task to a queue to then > making it available in the queue to lease. Sometimes the task is available > immediately, and other times there may be a few hundred milliseconds delay. > To be on the safe side, have your application sleep for a moment before > leasing tasks. > > > > - > -Vinny P > Technology & Media Advisor > Chicago, IL > > App Engine Code Samples: http://www.learntogoogleit.com > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at http://groups.google.com/group/google-appengine. > For more options, visit https://groups.google.com/groups/opt_out. > -- Jacob Taylor CTO, Kahuna (650) 302-1864 www.useKahuna.com <http://www.usekahuna.com/> http://www.linkedin.com/in/thejacobtaylor<http://www.linkedin.com/in/adammarchick> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Best Practices for Continuous Deployment on AppEngine
s://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out> >>>> . >>>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Google App Engine" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to google-appengi...@**googlegroups.com. >>> To post to this group, send email to google-a...@googlegroups.**com. >>> >>> Visit this group at >>> http://groups.google.com/**group/google-appengine<http://groups.google.com/group/google-appengine> >>> . >>> For more options, visit >>> https://groups.google.com/**groups/opt_out<https://groups.google.com/groups/opt_out> >>> . >>> >> -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at http://groups.google.com/group/google-appengine. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- Jacob Taylor CTO, Kahuna (650) 302-1864 www.useKahuna.com <http://www.usekahuna.com/> http://www.linkedin.com/in/thejacobtaylor<http://www.linkedin.com/in/adammarchick> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] SDK running reeeeeallly slow
One issue that I ran into was the way the system monitors for file changes. It does not respect the ignore list from app.yaml, it watches for each instance of the front ends and/or back ends, and it restarts all servers on any change. Our system has at least two watchers checking ~10,000 files per second each. Servers are bounced when we do something like 'git status', 'git push', ... This was using a ton of CPU and slowing down our dev servers. I modified the "watcher" file to ignore some common directory names (.git, .svn, ...) and log when it is bouncing the server. Attached is the modified file, use at will. Would appreciate any enhancements sent back. To use this file, copy it over the dev app server's file. You have to replace on each SDK upgrade. On mac, this can be done with: cp mtime_file_watcher.py /Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/devappserver2 Jacob On Fri, Aug 16, 2013 at 9:01 AM, PK wrote: > Hi Mike, > > I have noticed that too. The CPU shows 25% even when the server is idle. > It is not as bad as it was at the time they introduced the log server APIs > but it is worse than what used to be. > > As another data point, I sporadically keep track of the time it takes to > run a suite of end-to-end tests for my app and the time has increased 75% > since 1.7.0. I have definitely not added so many new test cases, hardware > is the same etc.: > > > Date: Wed Aug 7 15:40:54 2013 -0700 > Linux ro2 2.6.32-33-server > Python 2.7.5 > GAE release: "1.8.2" > > real*26m49.428s* > user12m47.280s > sys 3m29.460s > > Date: Thu May 9 13:37:31 2013 -0700 > real*23m34.431s* > user10m12.420s > sys 2m18.190s > Linux ro2 2.6.32-33-server > Python 2.7.3 > GAE release: "1.8.0" > > Date: Wed Jul 25 21:29:57 2012 -0700 > Python 2.7.2 > GAE release: "1.7.0" > real*16m8.724s* > user8m21.050s > sys 1m45.360s > > PK > http://www.gae123.com > > On August 16, 2013 at 8:08:47 AM, Mike Knapp (m...@shoesofprey.com) wrote: > > Has anyone else noticed the SDK consuming a lot of system resources > recently? My computer is grinding to a halt, with the local dev server > taking 60% of the CPU even when it's just sitting idle. It's painful to use. > > This seems to be a recent issue after upgrading. I'm on a Mac. > > Mike > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at http://groups.google.com/group/google-appengine. > For more options, visit https://groups.google.com/groups/opt_out. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at http://groups.google.com/group/google-appengine. > For more options, visit https://groups.google.com/groups/opt_out. > -- Jacob Taylor CTO, Kahuna (650) 302-1864 www.useKahuna.com <http://www.usekahuna.com/> http://www.linkedin.com/in/thejacobtaylor<http://www.linkedin.com/in/adammarchick> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out. mtime_file_watcher.py Description: Binary data
Re: [google-appengine] apple-touch-icon-precomposed.png
Apple automatically looks here for shortcut icons. You need to provide one to avoid the errors, or just ignore it. On Thu, Aug 8, 2013 at 1:30 PM, Satyanarayana Govindachandra < satyanarayana.govindachan...@a-cti.com> wrote: > Hi > > Why do i get these requests triggered in my application, the count for > these requests in thousands and the error % is always 100. > > apple-touch-icon-precomposed.png > > I found that , when ever user loads the website from Iphone Chrome Browser > this happens. Why does like this, and why it requests on its own when we > have not given any link such in the site. > > -- > Thank you > Satya > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at http://groups.google.com/group/google-appengine. > For more options, visit https://groups.google.com/groups/opt_out. > -- Jacob Taylor CTO, Kahuna (650) 302-1864 www.useKahuna.com <http://www.usekahuna.com/> http://www.linkedin.com/in/thejacobtaylor<http://www.linkedin.com/in/adammarchick> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: Cron jobs auto repeat on failure option?
A fun little gotcha about this. You may not really know why that 10 minute interval was triggered and it is likely destructive. The timeout can be triggered by something else. If any thread on the instance takes too long, the entire instance is killed immediately along with all active requests. We ran into an issue with a huge job that was making it look like logging.info was taking way too long. logging.info was just where the context switching was occurring. We have also had our fair share of problems adding items to the queue "transient error" when load surges. On Fri, Jul 19, 2013 at 9:34 AM, Marcel Manz wrote: > Likely the same as our crons are doing - they just do some key scanning > and enqueue tasks to the taskqueue, having their own retry policy. The cron > job itself only runs for a few seconds until all tasks are dispatched. > > Unfortunately from time to time this simple operation fails. When it > happens we can see that the cron request was aborted after 10 minutes. For > some reason the handler doesn't start correctly and app engine times out > after 10 minutes marking the cron as failed. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at http://groups.google.com/group/google-appengine. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- Jacob Taylor CTO, Kahuna (650) 302-1864 www.useKahuna.com <http://www.usekahuna.com/> http://www.linkedin.com/in/thejacobtaylor<http://www.linkedin.com/in/adammarchick> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: Has anyone received a warning email like this?
Waleed, are you checking on "all versions" of your application? If they compromised the account or you had a vulnerability, email could be going out on a non-default version. On Fri, Jul 12, 2013 at 12:22 AM, Waleed Abdulla wrote: > Julie, > I replied to the email, and I hope to get an answer. Although, the > way they've worded it, "we'll do our best to answer...", doesn't give me > much confidence that they will in time. The email seems automated, and it's > possible that GAE might block the app automatically in 5 days if whatever > triggered this isn't fixed. I don't want to have a downtime, so I'm trying > to solve the problem. I just need to know what to solve :) > > And, yes, I agree that the reference to bulk email is interesting. > We've switched emails of all our apps to a different provide a long time > ago. And this particular app only sends a few emails, and none of them > through App Engine. I thought maybe someone hacked into the app and used it > to send emails, but GAE's dashboard confirms that we're not sending any. > > > > > > On Thu, Jul 11, 2013 at 10:50 PM, Google Tasks Backup Moderator < > gtb_modera...@logic-a-to-b.com> wrote: > >> Hi Waleed, >> >> Have you received a response from Google? "If you would like us to help >> you understand that unusual activity, please reply directly to this email. >> " >> >> It is indeed puzzling that they send a link to "Bulk Email Sender >> guidelines" if your app does not send emails. >> >> Regards, >> >> Julie >> >> >> >> On 12 July 2013 15:30, Waleed Abdulla wrote: >> >>> On Thu, Jul 11, 2013 at 8:04 PM, Lawrence Mok wrote: >>> >>>> so what have you done to get this email? >>> >>> >>> That's exactly what I'm trying to figure out. This is the site, in case >>> someone can point out something wrong about it. >>> >>> http://www.symphonytools.com >>> >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "Google App Engine" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to google-appengine+unsubscr...@googlegroups.com. >>> To post to this group, send email to google-appengine@googlegroups.com. >>> Visit this group at http://groups.google.com/group/google-appengine. >>> For more options, visit https://groups.google.com/groups/opt_out. >>> >>> >>> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to google-appengine+unsubscr...@googlegroups.com. >> To post to this group, send email to google-appengine@googlegroups.com. >> Visit this group at http://groups.google.com/group/google-appengine. >> For more options, visit https://groups.google.com/groups/opt_out. >> >> >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to google-appengine+unsubscr...@googlegroups.com. > To post to this group, send email to google-appengine@googlegroups.com. > Visit this group at http://groups.google.com/group/google-appengine. > For more options, visit https://groups.google.com/groups/opt_out. > > > -- Jacob Taylor CTO, Kahuna (650) 302-1864 www.useKahuna.com <http://www.usekahuna.com/> http://www.linkedin.com/in/thejacobtaylor<http://www.linkedin.com/in/adammarchick> -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Annourcing aeta: a test runner for Python App Engine applications
Hello, I am happy to announce that we open sourced aeta, the App Engine Test Appendix. aeta is a tool for running tests on your Python App Engine application. It was created at Google and is used to run many of our own tests. It features both a web interface and a command-line interface to run tests. Code and documentation are available at http://code.google.com/p/aeta/. We also set up a group at https://groups.google.com/group/appengine-aeta. If you have questions, suggestions or other issues you want to talk about, don't hesitate to reach out. Thanks, Jacob -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/KlOOvw6e0DoJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
