Re: [google-appengine] Re: Task Creation failing at a high rate since 1.5 pre release
We are looking into this issue. On 4 May 2011 10:01, Anders wrote: > Same problem here, also getting: > DeadlineExceededError: The API call taskqueue.BulkAdd() took too long > to respond and was cancelled. > > Have been going on all day with about the same frequency (mostly > TransientErrors that is) > > On 4 Maj, 18:55, Andrei Cosmin Fifiiţă > wrote: >> I still get these errors... I can't say its a high percentage, but >> it's a significant rate. >> >> On 4 May 2011 19:47, Robert Kluin wrote: >> >> >> >> > I've seen a few bursts of transient errors on a couple Python apps as >> > well. They seemed to have cleared up for me though. >> >> > What percentage of task adds are giving errors? >> >> > Robert >> >> > On Wed, May 4, 2011 at 12:31, Ice13ill wrote: >> >> I get this exception too... It seams that there is a problem when >> >> adding tasks to a queue ? >> >> >> On May 4, 5:55 pm, nischalshetty wrote: >> >>> Hi, >> >> >>> I'm seeing a lot of task failures after the announcement of the pre >> >>> release >> >>> 1.5 sdk. I haven't changed my sdk (I'm on 1.4.2 java sdk for gae). Tasks >> >>> are very critical to my application. My app creates a lot of tasks every >> >>> minute. Can you please look into this? >> >> >>> Here's the stack trace : >> >> >>> com.google.appengine.api.taskqueue.TransientFailureException: Unknown >> >>> at >> >>> com.google.appengine.api.taskqueue.QueueApiHelper.translateError(QueueApiHe >> >>> lper.java:58) >> >>> at >> >>> com.google.appengine.api.taskqueue.QueueApiHelper.translateError(QueueApiHe >> >>> lper.java:112) >> >>> at >> >>> com.google.appengine.api.taskqueue.QueueApiHelper.makeSyncCall(QueueApiHelp >> >>> er.java:32) >> >>> at >> >>> com.google.appengine.api.taskqueue.QueueImpl.add(QueueImpl.java:466) >> >>> at >> >>> com.google.appengine.api.taskqueue.QueueImpl.add(QueueImpl.java:412) >> >>> at >> >>> com.google.appengine.api.taskqueue.QueueImpl.add(QueueImpl.java:397) >> >> >>> at sun.reflect.GeneratedMethodAccessor62.invoke(Unknown Source) >> >>> at >> >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImp >> >>> l.java:43) >> >>> at java.lang.reflect.Method.invoke(Method.java:616) >> >>> at >> >>> com.google.apphosting.runtime.security.shared.intercept.java.lang.reflect.M >> >>> ethod_$1.run(Method_.java:165) >> >>> at java.security.AccessController.doPrivileged(Native Method) >> >>> at >> >>> com.google.apphosting.runtime.security.shared.intercept.java.lang.reflect.M >> >>> ethod_.privilegedInvoke(Method_.java:163) >> >>> at >> >>> com.google.apphosting.runtime.security.shared.intercept.java.lang.reflect.M >> >>> ethod_.invoke_(Method_.java:124) >> >>> at >> >>> com.google.apphosting.runtime.security.shared.intercept.java.lang.reflect.M >> >>> ethod_.invoke(Method_.java:43) >> >>> at >> >>> com.opensymphony.xwork2.DefaultActionInvocation.invokeAction(DefaultActionI >> >>> nvocation.java:404) >> >>> at >> >>> com.opensymphony.xwork2.DefaultActionInvocation.invokeActionOnly(DefaultAct >> >>> ionInvocation.java:267) >> >> >>> at >> >>> com.opensymphony.xwork2.DefaultActionInvocation$2.doProfiling(DefaultAction >> >>> Invocation.java:224) >> >>> at >> >>> com.opensymphony.xwork2.DefaultActionInvocation$2.doProfiling(DefaultAction >> >>> Invocation.java:223) >> >>> at >> >>> com.opensymphony.xwork2.util.profiling.UtilTimerStack.profile(UtilTimerStac >> >>> k.java:455) >> >>> at >> >>> com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocat >> >>> ion.java:221) >> >>> at >> >>> com.opensymphony.xwork2.interceptor.DefaultWorkflowInterceptor.doIntercept( >> >>> DefaultWorkflowInterceptor.java:221) >> >>> at >> >>> com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(Metho >> >>> dFilterInterceptor.java:86) >> >>> at >> >>> com.opensymphony.xwork2.DefaultActionInvocation$2.doProfiling(DefaultAction >> >>> Invocation.java:224) >> >>> at >> >>> com.opensymphony.xwork2.DefaultActionInvocation$2.doProfiling(DefaultAction >> >>> Invocation.java:223) >> >>> at >> >>> com.opensymphony.xwork2.util.profiling.UtilTimerStack.profile(UtilTimerStac >> >>> k.java:455) >> >>> at >> >>> com.opensymphony.xwork2.DefaultActionInvocation.invoke(DefaultActionInvocat >> >>> ion.java:221) >> >>> at >> >>> com.opensymphony.xwork2.validator.ValidationInterceptor.doIntercept(Validat >> >>> ionInterceptor.java:150) >> >>> at >> >>> org.apache.struts2.interceptor.validation.AnnotationValidationInterceptor.d >> >>> oIntercept(AnnotationValidationInterceptor.java:48) >> >>> at >> >>> com.opensymphony.xwork2.interceptor.MethodFilterInterceptor.intercept(Metho >> >>> dFilterInterceptor.java:86) >> >>> at >> >>> com.opensymphony.xwork2.DefaultActionI
Re: [google-appengine] Re: Backends clarifications
I would suggest having a look at the example code in http://backends-io.appspot.com/static/counter_demo.zip. The code gives examples of how to use urlfetch to communicate between a fronend and a backend. The backend (counter.py) also keeps the value in memory, and writes out changes either within two seconds of modifying the data, or when a shutdown notification occurs (which ever happens first). On 12 May 2011 08:23, Vanni Totaro wrote: > Hi Richard. > Memcache by design will do eviction. You should keep your backend instance > data in memory (e.g. global var). Be aware that even backend instance could > be killed by the engine, so periodically store live status data in db. > Hope it helps. > Vanni > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] 'Datastore Viewer' web interface bug?
This does indeed sound like a bug. Can you please file a bug about this on the bug tracker? http://code.google.com/p/googleappengine/issues/entry?template=Other%20defect If you include the issue number in this thread I can ensure it get triaged correctly. On 12 May 2011 13:32, Loren wrote: > I've got a simple db.Expando class that I'm trying to edit fields > within from the Google App Engine data store viewer interface. There > appears to be a bug with modifying some of the properties; > Specifically, I have a property called 'type'. Attempting to modify > any of the fields including or 'after' this property fails to > persist. > > I can view the POST data that goes to: /datastore/edit.do. It returns > status 302, and here's a small part of it: > > &type-15=string&name-16=temperature&value_string-16=&type-16=null&name-17=type&value_string-17=MODIFYATTEMPT&type-17=string&name-18=zones&value_string-18=USDA > +Zone+6+to+USDA+Zone+9&type-18=string... > > Attempting to modify either the 'type' or 'zones' property fails, but > anything before that like 'temperature' succeeds. > > It looks like my property name 'type' here is interfering with the > handler parsing out the post data from this web request. Is this a > legitimate bug in the web interface? > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Backends and the task queue
Yes. The taskqueue.add function may only add 100 tasks at a time. On 12 May 2011 15:35, Jeff Schnitzer wrote: > With appengine frontends, each request is limited to 100 inserts into > the task queue. > > Do long-running backend requests have the same limitation? > > Jeff > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Backends and the task queue
By 'at a time' I mean per call to taskqueue.Queue.add, not per http
request. You can call that function twice in a row with two different
batches of tasks.
For example, in the python api you can do this:
queue = taskqueue.Queue('default')
tasks = [taskqueue.Task(params={'id': i}) for i in xrange(100)]
queue.add(tasks)
If you have more than 100 tasks that you want to insert, you will need
to split them into groups of 100 tasks, and call queue.add for each
group.
For example:
queue = taskqueue.Queue('default')
tasks = [taskqueue.Task(params={'id': i}) for i in xrange(1000)]
def _yield_in_groups(l, group_size):
for start_index in xrange(0, len(l), group_size):
yield l[start_index: start_index + group_size]
for task_group in _yield_in_groups(tasks, 100):
queue.add(task_group)
On 12 May 2011 19:11, Jeff Schnitzer wrote:
> Would you please clarify that? When you say "at a time", do you mean
> "within the scope of a single HTTP request"? The docs say that 100
> tasks is "the maximum number of tasks that can be added in a batch"
> but they don't specify what "batch" means.
>
> If this is indeed the limit, is there any talk of eliminating it for
> long-running Backend requests? It seems to seriously limit the
> utility of Backends for long-running data processing that involves
> task creation - sure, I no longer need to break my work down into 30s
> (now 10m) increments, but now I need to respawn a new task anyways
> every time I hit 100 tasks.
>
> Here's the specific issue I have right now:
>
> I have a button in one of my apps that sends push notifications to a
> few million iphone app users. A "map" task iterates through the
> users, creating a new "reduce" task that sends the relevant data for a
> single push to a REST service wired up to the APNS. The map task has
> to restart every 100 reduce tasks it creates.
>
> For technical reasons related to filtering specific users to push to,
> it would be a lot easier if the "map" task didn't have to restart.
> I'd like to fire up a Backend to run through the entire set in one go.
> But this would require millions of tasks to be created in the context
> of that long-running request.
>
> Is this just not an option?
>
> Thanks,
> Jeff
>
> On Thu, May 12, 2011 at 5:01 PM, Greg Darke (Google)
> wrote:
>> Yes.
>>
>> The taskqueue.add function may only add 100 tasks at a time.
>>
>> On 12 May 2011 15:35, Jeff Schnitzer wrote:
>>> With appengine frontends, each request is limited to 100 inserts into
>>> the task queue.
>>>
>>> Do long-running backend requests have the same limitation?
>>>
>>> Jeff
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "Google App Engine" group.
>>> To post to this group, send email to google-appengine@googlegroups.com.
>>> To unsubscribe from this group, send email to
>>> google-appengine+unsubscr...@googlegroups.com.
>>> For more options, visit this group at
>>> http://groups.google.com/group/google-appengine?hl=en.
>>>
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To post to this group, send email to google-appengine@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Backends and the task queue
On a related note,
You should also look into using the asynchronous urlfetch api. This
would allow you to send multiple requests at the same time to the rest
service you are using to make push notifications.
This fits in very well with the new taskqueue pull queue api. You
could still use a task per notification to send to a user, but when
you are sending them (in your 'reduce' phase) you could lease multiple
tasks at once and send them all at the same time.
On 12 May 2011 20:20, Greg Darke (Google) wrote:
> By 'at a time' I mean per call to taskqueue.Queue.add, not per http
> request. You can call that function twice in a row with two different
> batches of tasks.
>
> For example, in the python api you can do this:
> queue = taskqueue.Queue('default')
> tasks = [taskqueue.Task(params={'id': i}) for i in xrange(100)]
> queue.add(tasks)
>
> If you have more than 100 tasks that you want to insert, you will need
> to split them into groups of 100 tasks, and call queue.add for each
> group.
>
> For example:
> queue = taskqueue.Queue('default')
> tasks = [taskqueue.Task(params={'id': i}) for i in xrange(1000)]
>
> def _yield_in_groups(l, group_size):
> for start_index in xrange(0, len(l), group_size):
> yield l[start_index: start_index + group_size]
>
> for task_group in _yield_in_groups(tasks, 100):
> queue.add(task_group)
>
>
>
> On 12 May 2011 19:11, Jeff Schnitzer wrote:
>> Would you please clarify that? When you say "at a time", do you mean
>> "within the scope of a single HTTP request"? The docs say that 100
>> tasks is "the maximum number of tasks that can be added in a batch"
>> but they don't specify what "batch" means.
>>
>> If this is indeed the limit, is there any talk of eliminating it for
>> long-running Backend requests? It seems to seriously limit the
>> utility of Backends for long-running data processing that involves
>> task creation - sure, I no longer need to break my work down into 30s
>> (now 10m) increments, but now I need to respawn a new task anyways
>> every time I hit 100 tasks.
>>
>> Here's the specific issue I have right now:
>>
>> I have a button in one of my apps that sends push notifications to a
>> few million iphone app users. A "map" task iterates through the
>> users, creating a new "reduce" task that sends the relevant data for a
>> single push to a REST service wired up to the APNS. The map task has
>> to restart every 100 reduce tasks it creates.
>>
>> For technical reasons related to filtering specific users to push to,
>> it would be a lot easier if the "map" task didn't have to restart.
>> I'd like to fire up a Backend to run through the entire set in one go.
>> But this would require millions of tasks to be created in the context
>> of that long-running request.
>>
>> Is this just not an option?
>>
>> Thanks,
>> Jeff
>>
>> On Thu, May 12, 2011 at 5:01 PM, Greg Darke (Google)
>> wrote:
>>> Yes.
>>>
>>> The taskqueue.add function may only add 100 tasks at a time.
>>>
>>> On 12 May 2011 15:35, Jeff Schnitzer wrote:
>>>> With appengine frontends, each request is limited to 100 inserts into
>>>> the task queue.
>>>>
>>>> Do long-running backend requests have the same limitation?
>>>>
>>>> Jeff
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups
>>>> "Google App Engine" group.
>>>> To post to this group, send email to google-appengine@googlegroups.com.
>>>> To unsubscribe from this group, send email to
>>>> google-appengine+unsubscr...@googlegroups.com.
>>>> For more options, visit this group at
>>>> http://groups.google.com/group/google-appengine?hl=en.
>>>>
>>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "Google App Engine" group.
>>> To post to this group, send email to google-appengine@googlegroups.com.
>>> To unsubscribe from this group, send email to
>>> google-appengine+unsubscr...@googlegroups.com.
>>> For more options, visit this group at
>>> http://groups.google.com/group/google-appengine?hl=en.
>>>
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To post to this group, send email to google-appengine@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] B8 vs B4 vs B2 vs B1 , B8 = B4 ?
Please see the Backends talk from Google IO
(http://www.youtube.com/watch?v=-kepYfCBg6w).
It is mentioned in this talk that a single thread is only able to use
2.4Ghz of cpu. This is why a B4 and B8 appear the to have the same
amount of cpu time in your example. To take full advantage of the cpu
available in a B8, you have to use the java runtime at the moment.
It is important to note that a B8 also has twice the memory that a B4
has. This means you may still want to use a B8 python backend to take
advantage of the 1GB of ram.
On 13 May 2011 09:23, Robert Schuppenies wrote:
> And you could also use the timeit module
> (http://docs.python.org/library/timeit.html):
> """
> import timeit
> print timeit.Timer('"喔喔".decode("utf-8")').timeit()
> """
> On Fri, May 13, 2011 at 2:48 AM, Brandon Wirtz wrote:
>>
>> Add 50 characters and double your loop amount. You aren't pushing it
>> hard enough that you get an accurate number.
>>
>>
>>
>> -Original Message-
>> From: google-appengine@googlegroups.com
>> [mailto:google-appengine@googlegroups.com] On Behalf Of ??
>> Sent: Friday, May 13, 2011 2:07 AM
>> To: Google App Engine
>> Subject: [google-appengine] B8 vs B4 vs B2 vs B1 , B8 = B4 ?
>>
>> def test(i = 100):
>> t = datetime.datetime.now()
>> while i:
>> i -= 1
>> '喔喔'.decode('utf-8')
>> print datetime.datetime.now()-t
>>
>> test()
>>
>>
>> B8 time: 0:00:01.520756
>>
>> B4 time: 0:00:01.524307
>>
>> B2 time: 0:00:02.342202
>>
>> B1 time: 0:00:03.492686
>>
>>
>> B8 = B4 ?
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To post to this group, send email to google-appengine@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To post to this group, send email to google-appengine@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Backends clarifications
This sounds like a excellent use of backends. I believe that there is currently a limitation that a channel created in a frontend (ie, your default version) is not able to be used to send messages from a backend. As noted in the Google IO talk, we are currently working on removing this restriction. For the moment, you could work around this limitation by creating the channel in the backend that you want to send messages from, then pass it back to the frontend/user. On 13 May 2011 02:37, Richard Arrano wrote: > Ah that's great news then, I didn't realize that. > > Greg: Thanks for the link, I was a bit misunderstood and didn't see it > in the App Engine Samples. > > One thing that's intrigued me about the reserved backend, and it > mentions in the documentation, is to keep track of game states. I was > hoping someone could tell me if this setup would make sense: if I need > to update possibly hundreds of game states, and by update I mean > sending out actual updates via the Channel API, I thought I could do > this by two separate instances. One of which holds the data(there > isn't much data to keep track of, 128MB may be sufficient), the other > operates on a while loop that only exits when all games have completed > and runs every second or every other second using time.sleep(1000) or > 2000. It sends out updates to each client in each game that requires > updates, and gets the data from the other backend instance. Then it > would sleep until the next iteration. Is this a sensible use of > backends or is this actually more suited to tasks? I was originally > using tasks that enqueue a new task each following second to > accomplish this. Which one would be better and why? > > Thanks, > Richard > > On May 12, 11:10 am, "Gregory D'alesandre" wrote: >> On Thu, May 12, 2011 at 3:32 AM, Richard Arrano wrote: >> >> > Hello, >> > I have a few questions about the new Backends feature: >> > It seems like a reserved backend allows us to have access to a sort of >> > unevictable memcache, is this correct? If I were doing something like >> > a game, would it be reasonable to have a single instance keep track of >> > a few pieces of vital information about each game? The sum of which >> > might be around 1MB of information at any given time. >> >> > How would I go about this anyway? Address the reserved instance and >> > how would I set a value in its memory? >> >> > And just to be clear, there's no free version of Backends, correct? >> >> Actually, there is! You get $0.72 worth of backends free per day (this can >> be used in any configuration you'd like 9 hours of a B1 or 1 hour of a B8 + >> 1 hour of a B1, etc). >> >> Greg >> >> >> >> > Thanks, >> > Richard >> >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "Google App Engine" group. >> > To post to this group, send email to google-appengine@googlegroups.com. >> > To unsubscribe from this group, send email to >> > google-appengine+unsubscr...@googlegroups.com. >> > For more options, visit this group at >> >http://groups.google.com/group/google-appengine?hl=en. >> >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Drop Table equivalent?
The currently recommended way of deleting all entities of a particular kind is to use datastore admin (http://code.google.com/appengine/docs/adminconsole/datastoreadmin.html#Deleting_Entities_in_Bulk). If the primary version of your code is not in python, you can upload a simple application to an alternate version that has datastore admin enabled. On 13 May 2011 13:12, Brandon Wirtz wrote: > I know people have asked variations of this question, and the solution has > always been to drop entities 1000 at a time, > > > > But have an app that the cleanup code had a misspelling so it never did > anything except clean things that didn’t exist… > > > > Is there a “Drop Table” equivalent? Or do I really query 1000 and delete > over and over. Cause it appears that’s going to be like $100 which is > fine…. If that is the best solution, but if there is a drop table equivalent > that is $5 I’d really like that so that I can use it in my code that would > normally get charged the $100 over 6 months … > > > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Possible to Defer Tasks to Backend
You should be able to send a deferred task to a backend. Some points to remember though: * You should ensure that any classes/etc you reference from the frontend must exist in the backend. * The deferred function should pass the 'target' parameter on to taskqueue * You must enable the deferred handler on the backend (just make sure it is enabled in app.yaml when you upload the backend). On 14 May 2011 14:41, Aaron wrote: > Hi, > > The way my application is set up involves deferring a lot of long- > running tasks, which I now would like to move to a backend. Is it > possible to defer a task so that it is processed in a backend? Or, do > I have to create handlers for every deferred task in order to take > advantage of this feature? > > Best, > Aaron > > P.S. Thanks so much for the hard work! The new features are very > exciting :). > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Possible to Defer Tasks to Backend
That is correct, all tasks get the same host header as the request that created them. This means that if you access the url http://0.backend-name.app-id.appspot.com/ and it inserts a task, the task will execute against instance 0 of 'backend-name'. If you access the url http://backend-name.app-id.appspot.com/ and it inserts a task, the task will execute against any instance of 'backend-name'. On 14 May 2011 15:31, Aaron wrote: > A follow up to this question is if we can use the new pipeline API > with backends. If we start a pipeline in a backend, will all future > pipeline tasks be processed by the backend? > > On May 14, 2:41 pm, Aaron wrote: >> Hi, >> >> The way my application is set up involves deferring a lot of long- >> running tasks, which I now would like to move to a backend. Is it >> possible to defer a task so that it is processed in a backend? Or, do >> I have to create handlers for every deferred task in order to take >> advantage of this feature? >> >> Best, >> Aaron >> >> P.S. Thanks so much for the hard work! The new features are very >> exciting :). > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Possible to Defer Tasks to Backend
Since backends and frontends are updated separately, the code in each may be different. Updating a frontend does not change the code that is running in a backend/vice versa. As for the syntax, I believe it should be: deferred.defer(deferred_fn_name, param1, param2, _target='backend-name') On 14 May 2011 18:22, Aaron wrote: > Hi Greg, > > Thanks for the response. Just to be clear, the syntax to defer a task > to a backend is the following? > deferred.defer(deferred_fn_name, param1, param2, _target='http:// > backend-name.app-id.appspot.com') > > Also, you mentioned that we should be sure that the backend has the > same classes and configurations as the frontend? As far as I have > read, the frontend and backend are using the same code base, so is > this implied already, or is there something extra I am missing? > > Best, > Aaron > > On May 14, 5:56 pm, "Greg Darke (Google)" > wrote: >> You should be able to send a deferred task to a backend. >> >> Some points to remember though: >> * You should ensure that any classes/etc you reference from the >> frontend must exist in the backend. >> * The deferred function should pass the 'target' parameter on to taskqueue >> * You must enable the deferred handler on the backend (just make sure >> it is enabled in app.yaml when you upload the backend). >> >> On 14 May 2011 14:41, Aaron wrote: >> >> >> >> >> >> >> >> > Hi, >> >> > The way my application is set up involves deferring a lot of long- >> > running tasks, which I now would like to move to a backend. Is it >> > possible to defer a task so that it is processed in a backend? Or, do >> > I have to create handlers for every deferred task in order to take >> > advantage of this feature? >> >> > Best, >> > Aaron >> >> > P.S. Thanks so much for the hard work! The new features are very >> > exciting :). >> >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "Google App Engine" group. >> > To post to this group, send email to google-appengine@googlegroups.com. >> > To unsubscribe from this group, send email to >> > google-appengine+unsubscr...@googlegroups.com. >> > For more options, visit this group >> > athttp://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Backends Sample does not work
In the dev-appserver you need to pass the '--backends' parameter.
On 15 May 2011 11:07, uri twig wrote:
> Hi I'm tring to work with the new backends api, I downloaded the
> sample code form http://backends-io.appspot.com/
>
> and I run it on a few machines with the new appengine 1.5 sdk
>
> this is what I get when http://localhost:8080/welcome
>
> What am I'm doing wrong ?!?
>
> Traceback (most recent call last):
> File "C:\Program Files\Google\google_appengine\google\appengine\tools
> \dev_appserver.py", line 4053, in _HandleRequest
> self._Dispatch(dispatcher, self.rfile, outfile, env_dict)
> File "C:\Program Files\Google\google_appengine\google\appengine\tools
> \dev_appserver.py", line 3977, in _Dispatch
> base_env_dict=env_dict)
> File "C:\Program Files\Google\google_appengine\google\appengine\tools
> \dev_appserver.py", line 588, in Dispatch
> base_env_dict=base_env_dict)
> File "C:\Program Files\Google\google_appengine\google\appengine\tools
> \dev_appserver.py", line 3050, in Dispatch
> self._module_dict)
> File "C:\Program Files\Google\google_appengine\google\appengine\tools
> \dev_appserver.py", line 2954, in ExecuteCGI
> reset_modules = exec_script(handler_path, cgi_path, hook)
> File "C:\Program Files\Google\google_appengine\google\appengine\tools
> \dev_appserver.py", line 2834, in ExecuteOrImportScript
> exec module_code in script_module.__dict__
> File "D:\test\counter_demo\demo_frontend.py", line 22, in
> backends.get_url('counter'))
> File "C:\Program Files\Google\google_appengine\google\appengine\api
> \backends\backends.py", line 101, in get_url
> return _get_dev_url(backend, instance)
> File "C:\Program Files\Google\google_appengine\google\appengine\api
> \backends\backends.py", line 171, in _get_dev_url
> return 'http://%s' % _get_dev_hostname(backend, instance)
> File "C:\Program Files\Google\google_appengine\google\appengine\api
> \backends\backends.py", line 190, in _get_dev_hostname
> raise InvalidBackendError(error)
> InvalidBackendError: Backend not found: counter
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] How can I test backends with my free app
To upload this backend to App Engine, you need to perform a backends up: appcfg.py backends update path/to/application/ On 12 May 2011 10:04, Sergey wrote: > I want to test new feature - backends. > As I understand it's enough to create backends.yaml and describe there > parameters of my backends. > I've done that: > > backends: > - name: testbe > class: B1 > instances: 1 > > I ran this app on dev server. At admin console I see "This application > doesn't define any backends. See the documentation for more." > > I tried to deploy this into GAE - same effect. > > What am I doing wrong? > Is there some code examples to better understand the idea of backends? > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Possible to Defer Tasks to Backend
Hi Aaron, I did not realise that the deferred api does not pass the target parameter on. For the moment, you can either modify your app.yaml file to ensure that all tasks inserted on a queue are sent to the specified backend or you can patch deferred.py and upload it in your application. As for your question about the pipeline api, if you the call to pipeline.start occurs in a backend, then all tasks created in that pipeline will execute on that backend. If you would like it to support the target parameter, please file a bug on the issue tracker for that project - http://code.google.com/p/appengine-pipeline/issues/list. On 15 May 2011 14:49, Aaron wrote: > Hi Greg, > > I'm having trouble figuring out the exact syntax for deferring tasks > to a backend. > > When I try to use the following syntax: > deferred.defer(PullReport,userkey,dates,_queue="googleanalytics",_target='processga') > I get the following error: > TypeError: PullReport() got an unexpected keyword argument '_target' > > Also, how exactly do we specify to start a pipeline in a specific > backend. I've tried pipeline.start(_target='backend-name') but get an > unexpected keyword argument as well. > > Thanks! > Aaron > > On May 14, 7:19 pm, "Greg Darke (Google)" > wrote: >> Since backends and frontends are updated separately, the code in each >> may be different. Updating a frontend does not change the code that is >> running in a backend/vice versa. >> >> As for the syntax, I believe it should be: >> deferred.defer(deferred_fn_name, param1, param2, _target='backend-name') >> >> On 14 May 2011 18:22, Aaron wrote: >> >> >> >> >> >> >> >> > Hi Greg, >> >> > Thanks for the response. Just to be clear, the syntax to defer a task >> > to a backend is the following? >> > deferred.defer(deferred_fn_name, param1, param2, _target='http:// >> > backend-name.app-id.appspot.com') >> >> > Also, you mentioned that we should be sure that the backend has the >> > same classes and configurations as the frontend? As far as I have >> > read, the frontend and backend are using the same code base, so is >> > this implied already, or is there something extra I am missing? >> >> > Best, >> > Aaron >> >> > On May 14, 5:56 pm, "Greg Darke (Google)" >> > wrote: >> >> You should be able to send a deferred task to a backend. >> >> >> Some points to remember though: >> >> * You should ensure that any classes/etc you reference from the >> >> frontend must exist in the backend. >> >> * The deferred function should pass the 'target' parameter on to >> >> taskqueue >> >> * You must enable the deferred handler on the backend (just make sure >> >> it is enabled in app.yaml when you upload the backend). >> >> >> On 14 May 2011 14:41, Aaron wrote: >> >> >> > Hi, >> >> >> > The way my application is set up involves deferring a lot of long- >> >> > running tasks, which I now would like to move to a backend. Is it >> >> > possible to defer a task so that it is processed in a backend? Or, do >> >> > I have to create handlers for every deferred task in order to take >> >> > advantage of this feature? >> >> >> > Best, >> >> > Aaron >> >> >> > P.S. Thanks so much for the hard work! The new features are very >> >> > exciting :). >> >> >> > -- >> >> > You received this message because you are subscribed to the Google >> >> > Groups "Google App Engine" group. >> >> > To post to this group, send email to google-appengine@googlegroups.com. >> >> > To unsubscribe from this group, send email to >> >> > google-appengine+unsubscr...@googlegroups.com. >> >> > For more options, visit this group >> >> > athttp://groups.google.com/group/google-appengine?hl=en. >> >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "Google App Engine" group. >> > To post to this group, send email to google-appengine@googlegroups.com. >> > To unsubscribe from this group, send email to >> > google-appengine+unsubscr...@googlegroups.com. >> > For more options, visit this group >> > athttp://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: How can I test backends with my free app
Oh, right. To get backends to work in the development app server you need to pass it the '--backends' option. On 16 May 2011 21:19, Richard Arrano wrote: > What I think he was referring to, and a problem I've encountered, is > working with Backends in the development server. The dev server will > not acknowledge the presence of Backends - the link is listed in the > admin console, but it won't list any backends you've configured in the > yaml file. Are these meant to only work on the production server? I > actually was trying to experiment with the backends-io you linked to > and I could not get it to work on the dev server at all. > > -Richard > > On May 16, 2:46 pm, "Greg Darke (Google)" > wrote: >> To upload this backend to App Engine, you need to perform a backends up: >> >> appcfg.py backends update path/to/application/ >> >> On 12 May 2011 10:04, Sergey wrote: >> >> > I want to test new feature - backends. >> > As I understand it's enough to create backends.yaml and describe there >> > parameters of my backends. >> > I've done that: >> >> > backends: >> > - name: testbe >> > class: B1 >> > instances: 1 >> >> > I ran this app on dev server. At admin console I see "This application >> > doesn't define any backends. See the documentation for more." >> >> > I tried to deploy this into GAE - same effect. >> >> > What am I doing wrong? >> > Is there some code examples to better understand the idea of backends? >> >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "Google App Engine" group. >> > To post to this group, send email to google-appengine@googlegroups.com. >> > To unsubscribe from this group, send email to >> > google-appengine+unsubscr...@googlegroups.com. >> > For more options, visit this group >> > athttp://groups.google.com/group/google-appengine?hl=en. >> >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Backends Sample does not work
Ah, Thanks Robert. This is fixed with the next release of the SDK. Until it is released you can use Robert's patch or the production environment. On 17 May 2011 09:03, Robert Kluin wrote: > Hi Uri, > It looks like the remote_api doesn't support lease_tasks. You > should be able to enable leasing and deleting tasks by adding the > following lines to the taskqueue service list: > > 'QueryAndOwnTasks': > (taskqueue_service_pb.TaskQueueQueryAndOwnTasksRequest, > > taskqueue_service_pb.TaskQueueQueryAndOwnTasksResponse), > 'Delete': (taskqueue_service_pb.TaskQueueDeleteRequest, > taskqueue_service_pb.TaskQueueDeleteResponse), > > Just put those after 'FetchQueueStats' (should be around line 110) in: > google_appengine/google/appengine/ext/remote_api/remote_api_services.py > > Attached is a patch file. This worked in my local testing at least. :) > > > Robert > > > > > > > On Tue, May 17, 2011 at 07:09, uri twig wrote: >> Thanks Greg >> >> That works OK. >> >> I have another problem: i have two back ends 1 'tester' inserts tasks >> to a queue with 'PULL' method. >> and another 'worker' that leases the tasks and process them. the >> 'tester' backend works fine. >> the 'worker' I get an exception at lease_tasks function >> >> the stack is as follows: >> >> [Remote API Server] [handler.py:308] ERROR Exception while handling >> service_name: "taskqueue" >> method: "QueryAndOwnTasks" >> request: "\n\011workitems\021\000\000\000\000\000 \254@\030d" >> Traceback (most recent call last): >> File "C:\Program Files\Google\google_appengine\google\appengine\ext >> \remote_api\handler.py", line 304, in post >> response_data = self.ExecuteRequest(request) >> File "C:\Program Files\Google\google_appengine\google\appengine\ext >> \remote_api\handler.py", line 324, in ExecuteRequest >> raise apiproxy_errors.CallNotFoundError() >> CallNotFoundError >> [Backend Instance] [worker.0] [utilities.py:143] ERROR ***worker*** >> CallNotFoundError >> >> Traceback (most recent call last): >> File "D:\vodio\trunk\Software\Samples\backends\test-backends >> \worker.py", line 48, in main >> tasks = q.lease_tasks(3600, 100) >> File "C:\Program Files\Google\google_appengine\google\appengine\api >> \taskqueue\taskqueue.py", line 1041, in lease_tasks >> response) >> File "C:\Program Files\Google\google_appengine\google\appengine\api >> \apiproxy_stub_map.py", line 94, in MakeSyncCall >> return stubmap.MakeSyncCall(service, call, request, response) >> File "C:\Program Files\Google\google_appengine\google\appengine\api >> \apiproxy_stub_map.py", line 308, in MakeSyncCall >> rpc.CheckSuccess() >> File "C:\Program Files\Google\google_appengine\google\appengine\api >> \apiproxy_rpc.py", line 156, in _WaitImpl >> self.request, self.response) >> File "C:\Program Files\Google\google_appengine\google\appengine\ext >> \remote_api\remote_api_stub.py", line 177, in MakeSyncCall >> self._MakeRealSyncCall(service, call, request, response) >> File "C:\Program Files\Google\google_appengine\google\appengine\ext >> \remote_api\remote_api_stub.py", line 197, in _MakeRealSyncCall >> raise pickle.loads(response_pb.exception()) >> CallNotFoundError >> >> >> >> On May 17, 12:44 am, "Greg Darke (Google)" >> wrote: >>> In the dev-appserver you need to pass the '--backends' parameter. >>> >>> On 15 May 2011 11:07, uri twig wrote: >>> >>> >>> >>> >>> >>> >>> >>> > Hi I'm tring to work with the new backends api, I downloaded the >>> > sample code formhttp://backends-io.appspot.com/ >>> >>> > and I run it on a few machines with the new appengine 1.5 sdk >>> >>> > this is what I get whenhttp://localhost:8080/welcome >>> >>> > What am I'm doing wrong ?!? >>> >>> > Traceback (most recent call last): >>> > File "C:\Program Files\Google\google_appengine\google\appengine\tools >>> > \dev_appserver.py", line 4053, in _HandleRequest >>> > self._Dispatch(dispatcher, self.rfile, outfile, env_dict) >>> > File "C:\Program Files\Google\google_appengine\google\appengine\tools >>> >
Re: [google-appengine] Re: Backends clarifications
Sorry about the delay. I did not notice the reply to this thread.
I will have a look into this next week.
On 26 May 2011 20:13, Chiguireitor wrote:
> Hi Greg... i would like to know how are we supposed to open the
> Channels in the current state of the server and SDK that are served
> from a backend, i'm currently opening the channel from within the
> backend and passing the token to my frontend but to no avail, here's a
> little sample code:
>
> *** On the frontend:
>
> cmd = Command.login(id, password)
> campaign= "campanamsl"
>
> # Enqueue on the backend command list
> task_name = "backend_%s_counter" % (campaign)
>
> dct = cmd.to_dictionary()
> idx = int(memcache.incr(task_name,
> initial_value=0))
>
> key_name = "%s_cmd_%i" % (task_name, idx)
> memcache.set(key_name, dct, time=1000)
>
> # Wait the token from the backend
> channel = None
> num = 0
> while (channel is None) and (num < 10): # This (1)
> loop will unlock when the backend replies with the usr info
> channel = memcache.get(key_name+"_resp")
> num += 1
> if channel is None:
> time.sleep(0.01)
>
> if channel is not None:
> ent = User(id)
>
> memcache.set(id, ent)
> sess['usr'] = ent
> sess.save()
>
> self.response.out.write(simplejson.dumps({"sta": ":-)", "chtok":
> channel}))
>
> *** On the backend:
>
> cnt = int(memcache.get(self.counter_name)) # counter_name is
> the same as task_name in the frontend
>
> if cnt is not None:
> while (cnt > self.cmds_count):
> self.cmds_count += 1
> cmd = memcache.get("%s_cmd_%i" % (self.counter_name,
> self.cmds_count))
>
> if (cmd is not None):
> command= Command()
> command.from_dictionary(cmd)
> ent = None
>
> if command.mapa is None:
> map = self.campaign.pos['map']
> else:
> map = int(command.map)
>
> ents = self.ents_per_map[map]
> try:
> ent = ents[command.origin]
> except KeyError:
> # Probably a SPAWN or LOGIN
> if command.tipo == Command.SPAWN:
> ent = memcache.get(command.origin)
> elif command.tipo == Command.LOGIN:
> # Not verifying user password yet
> chan =
> channel.create_channel(command.origin)
> memcache.set("%s_cmd_%i_resp" %
> (self.counter_name, self.cmds_count), chan, time=1000) # This unblock
> the (1) loop in the frontend
>
> ***
>
> The client side channel gets the token correctly, however nothing is
> received on the onmessage event of my channel handler. After a while
> the backend instance begins throwing "ProtocolBufferDecodeError:
> corrupted" on line 481 on google\net\proto\ProtocolBuffer.py :-(
>
> Please, tell me what i'm doing wrong here (if you can figure it out,
> of course)
>
> On 13 mayo, 14:16, "Greg Darke (Google)"
> wrote:
>> This sounds like a excellent use of backends.
>>
>> I believe that there is currently a limitation that a channel created
>> in a frontend (ie, your default version) is not able to be used to
>> send messages from a backend.
>>
>> As noted in the Google IO talk, we are currently working on removing
>> this restriction. For the moment, you could work around this
>> limitation by creating the channel in the backend that you want to
>> send messages from, then pass it back to the frontend/user.
>>
>> On 13 May 2011 02:37, Richard Arrano wrote:
>>
>>
>>
>>
>>
>>
>>
>> > Ah that's great news then, I didn't realize that.
>>
>> > Greg: Thanks for the link, I was a bit misunderstood and didn't see it
>> > in the App Engine Samples.
>>
>> > One thing that's intrigued me about the reserved backend, and it
>> > mentions in the documentation, is to keep track of
Re: [google-appengine] The API call taskqueue.Delete() required more quota than is available?
Check the value of "Task Queue API Calls" on the Task Queues page in admin console. Are you exceeding this limit? If not, what is the app id so that I can check why you are getting this error. On 1 June 2011 11:29, working wrote: > Dear All, > I started getting this error today. > > "The API call taskqueue.Delete() required more quota than is > available" > > I only lease_tasks(500). After processing, I want to delete_tasks > them. But failed. > Any suggestion? The limit for lease_tasks is 1000, and there is no > mention about the limit for delete_tasks. I check the dashboard, and I > have plenty quota for everything. > So strange~~ > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: The memory tax
On 1 June 2011 14:54, Bart Thate wrote: > > Once memory is billable, you gonna be sure that people actually try to make > good use of it. If you do instance price only who cares ? > As for debugging you might have point maybe google could provide the an app > stats API where the app could monitor its memory usage itself (or is there > already such a API) ? > You can use the google.appengine.api.runtime api to determine how much memory you are currently using ( http://code.google.com/appengine/docs/python/backends/runtimeapi.html). -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Running a task on a backend?
Are you using the taskqueue labs api? The taskqueue labs api (google.appengine.api.labs.taskqueue) was deprecated in 1.4.0, and thus new features will no longer be added to the old api. To use new features such as pull queues and the target parameter you will need to change your import statement to use the new api: from google.appengine.api import taskqueue On 22 June 2011 14:34, Waleed Abdulla wrote: > I'm following the steps explained in [1] trying to have my tasks run on > backend instances. I created 3 backend instances and I can see them in the > dashboard. However, when I run this code to insert a task, I'm getting an > error: > > > taskqueue.Task( > url='/work/smart_poll_bootstrap', > params=dict(sequence=name, poll_type=poll_type, end_time=end_time), > target='bootstrap', > ).add(POLLING_QUEUE, transactional=True) > > The error I'm getting is: > TypeError: Invalid arguments: target > > When I remove the "target" parameter, it works (but obviously doesn't go > to the backend instances). I also tried values of "1.bootstrap" and got the > same error. This is my backends.yaml file: > backends: > - name: bootstrap > class: B1 > instances: 3 > options: dynamic > > Anyone see a problem in the code above? This is failing in both, dev and > production. > Thanks, > Waleed > [1] http://code.google.com/appengine/docs/python/taskqueue/overview-push.html#Push_Queues_and_Backends > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] write to file system in development mode?
Hi Gerad,
The following code will work (at least in the current version of the
dev_appserver). I would like to point out that this is accessing a
private variable and may break in the future without any notice.
"""
from google.appengine.tools import dev_appserver
RealFileClass = dev_appserver.FakeFile._original_file
f = RealFileClass('filename', 'w')
"""
On 24 June 2011 10:36, Gerad Suyderhoud wrote:
> Long story, but... I need to be able to write to the filesystem when
> running the development appserver.
>
> Anybody know of a way to get around Google's monkey patches preventing
> this? (Again, I only need to do this in development).
>
> Notes:
> 1. Yes, I truly do need to write to the file system (file gets picked
> up by another service).
> 2. No, it doesn't need file system write access in production.
>
> Thanks in advance for any help
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Any mechanisms to block/wait without polling?
If you are interested in this feature (blocking for new pull queue tasks), please file an feature request in the issue tracker (or if one already exists, star it). We use the issue tracker to see what features user's are asking for and also to gauge how popular a specific request is. http://code.google.com/p/googleappengine/issues/list On 20 July 2011 11:03, Jim Mittler wrote: > Thanks for the feedback. I can certainly understand why the blocking > techniques are discouraged; it's still hard to get out of that frame > of mind! > > As for an example of "what I shouldn't be doing", the pull queue API > has a lease_task method, but as far as I can tell there is no way to > call it and have it block (for any duration) until the first task > becomes available. > > If I try to lease a task via the REST API or the internal API, I ill > need to keep calling "lease_task" over and over again until I get > some work. So either I burn CPU or I sleep in between calls and I > introduce an unwelcome delay in my processing. > > Jim > > On Jul 18, 6:45 pm, "Nick Johnson (Google)" > wrote: >> Hi Jim, >> >> Blocking in App Engine is generally strongly discouraged. Can you provide an >> example use-case in which you need to do this? >> >> -Nick Johnson >> >> >> >> >> >> >> >> >> >> On Tue, Jul 19, 2011 at 4:43 AM, Jim Mittler wrote: >> > I have some patterns that I would very much like to try to implement >> > in GAE, but I can't seem to get past a requirement to coordinate a >> > signal between two tasks running in GAE. >> >> > There seems to be no GAE mechanism to wait for an event without going >> > into some sort of polling loop. Is there something maybe that I'm >> > missing or is this just wishful thinking? >> >> > Thanks!! >> >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "Google App Engine" group. >> > To post to this group, send email to google-appengine@googlegroups.com. >> > To unsubscribe from this group, send email to >> > google-appengine+unsubscr...@googlegroups.com. >> > For more options, visit this group at >> >http://groups.google.com/group/google-appengine?hl=en. >> >> -- >> Nick Johnson, Developer Programs Engineer, App Engine > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] App Engine office hours for 2011/07/03 [PDT]
Chat log from today's App Engine office hours. All times are in AEST (UTC+10). 12:01 -!- Irssi: #appengine: Total of 115 nicks [0 ops, 0 halfops, 2 voices, 113 normal] 12:01 < jwbnyc> Thanks for joining, us Wesley! 12:01 < Wesley_Google> who? where? 12:01 < robertk> Wesley_Google: there -> 12:01 < robertk> ;) 12:02 <+gregdarke> Hey all 12:02 < johnlockwood> howdy 12:02 < mbw> ok, ive got one. Have you guys made any changes (improvements) in parallel db.get across entity groups on HRD? We had a developer testing async db.get and we were just wondering if that has gotten better yet. 12:02 < robertk> so what do you work on dave_google? don't remember seeing you around here 12:02 < dave_google> i'm on app engine 12:03 < dave_google> wes usually schedules these for when i can't make it. :) 12:03 < Wesley_Google> mbw> you mean batch get by key? 12:03 < jwbnyc> For our part, we're butting up against issues of code size. We're starting to encounter difficulty due to instance startup times. 12:03 < mbw> Wesley_Google: yep 12:03 < robertk> yeah which part of AE? gregdarke is a task queue guy, wesley is dev rel... just want to know which questions to direct at you :) 12:04 < dave_google> a bit of this, a bit of that. ask your question, and the right person will chime in. 12:05 < Wesley_Google> thanks dave!! (no, these are pre-scheduled all the time... it's either 7-8p PDT or 9-10a PDT) 12:05 < mbw> Wesley_Google: I know we have spoken to you guys about this before and you said things where being done to improve performance of that... since it was very slow comparead to master slave (when we first started using it... we havent done much benchmarking lately and didnt really notice a change) 12:05 < dave_google> 9am exists as an abstraction. 12:07 < jwbnyc> Has anyone been able to get a handle on where the initialization time goes? Our effort have shown wide differences between the Java dev server and GAE in terms of how our initial request performs. 12:07 < ronoaldojlp> I'm experiencing some instances startup latency, and posted an issue today. Does someone else is having the same issues too? 12:07 < robertk> mbw: when i was poking around in the sdk i did notice that they are putting in the code to auto parallelize the rpcs now 12:08 < ronoaldojlp> our Java instances usually take 3-6 sec to startup, but some of them throws (Hard)DeadlineExceededExceptions 12:09 < jwbnyc> ronoaldojlp: We've been trying to track down a similar set of issues. What's the URL for the issue you raised? 12:09 < mbw> robertk: right, but thats in the case where it would be too large to do it in one put right? 12:10 < robertk> mbw: nah, looked like something more general to me. judging by the comments at least 12:10 < mbw> robertk: which could be a bit scary actually since you could be writing them all to the same entitiy group, thus contention 12:10 < ronoaldojlp> jwbnyc: http://code.google.com/p/googleappengine/issues/detail?id=5477 12:10 < robertk> mbw: haven't dug as deeply as i wanted to yet though 12:10 < mbw> ya, who knows... frigen docs would be nice 12:10 < robertk> mbw: yeah there logic seemed to take entity groups into account 12:11 < dave_google> create an issue for where you want to see docs expanded. those do get listened to. 12:11 < robertk> mbw: seemed to break up the rpcs based on entity group -- there is now na entity_grpups_per_rpc (or something similar) config option 12:11 < jwbnyc> Thanks, Ron! I posted to the Appengine for Java group: https://groups.google.com/d/topic/google-appengine-java/rWfC6cypiwg/discussion 12:12 < robertk> mbw: here is one part of what i saw today: http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/datastore/datastore_rpc.py#488 12:12 < robertk> dave_google: mbw and i probably dig in a bit deeper than normal ;) 12:12 < robertk> dave_google: if you could get the internal comments left in when you push the code that would be great ;) 12:13 < ronoaldojlp> jwbnyc: we are trying to do some lazy initialization but not sure if there is any good "patter" on that area... we are using Google Guice to do a lot of work on that. 12:15 < JasonAtBobber> Anyone have thoughts on how best to profile GAE apps? Our initialization times are fairly short when run locally, but can be crazy-long when on AppEngine. Anyone use anything other than Appstats? AppWrench looked promising, except it appears to be dead. ;) 12:16 < ronoaldojlp> Googlers, is the "min idle instances" knob of the scheduler avaiable as preview? We're expecting a huge traffic tomorrow for a TV commercial and we thing that it may be a good Idea to leave some instances waiting ... Can I "emulate" it in a good way, by warming up some instances? 12:17 < robertk> what other scheduler config settings will be rolling out? i know min-idle-instances is planned. what about additional information to help us making decisions on scheduler settings, will there be more logging or analysis charts? 12:17 < robertk> and what i
Re: [google-appengine] Re: Automated way to switch from M/S to HR?
On 5 August 2011 10:13, johnP wrote: > can you clarify: "You need to persist re-read the Entities and persist > them as keys. " If your entities are stored as a String, or a list of strings, you will need to read the entities, and change their type to be either a Key (or reference property) or a list of Keys and then perform a Put to store them back in your datastore. If datastore keys are stored as strings, the migration tool will not be able to update them during the migration. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Is an APP ENGINE Application Attacking websites? - DOTS user agent / browser
Do you have the full useragent? App engine applications state that they are running on google app engine in their user agent. They also state the application id that is making the request. If the user agent does not contain google app engine, then it is not an app engine application making the requests. On 8 August 2011 15:57, Viper wrote: > Apparently this has been reported in several other places on Google > http://www.google.com/support/forum/p/Webmasters/thread?tid=6dd38e35376e723e&hl=en > http://www.google.com/support/forum/p/Google+Apps/thread?tid=0a4d894e858660f6&hl=en > http://www.google.com/support/forum/p/Webmasters/thread?tid=1ef5e9959bb406c1&hl=en > > And no one seems to know what it is, but found a reference for one of the IP > addresses having to do something with the Google App Engine > > Has anyone noticed the new DOTS browser / spider / user agent > browsing/scanning their site from Google IP Addresses? Here is an example > of a log entry from one > > Log File: X:\W3SVC1\ex110804.log > Log Entries For:74.125.64.84 > Browser: dots > Referrer: > Started: 8/4/2011 5:20:26 PM > > 2011-08-04 21:20:26 GET /- 74.125.64.84 dots - - > www.--- 403 6 0 5521 109 62 > > 1 Log Entries > > They seem to be coming from blocks > 74.125.0.0 - 74.125.255.255 > > and a few other Google blocks. > > The problem is that had to no enable the Plus (+1 button) on a site because > apparently the Spider (it uses a different user agent) comes from the same > blocks as this mysterious DOTS browser / spider / useragent. > > Haven't seem to find an concrete information as to what the purpose of DOTS > is and whether it should be trusted or not. > > Anyone have any ideas? > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/qMxZ0KLbzy4J. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Using memcache might be free, but it's costing you money with instance pricing! Use instance caches if possible.
Rishi: This sounds like an excellent idea. Even with concurrent requests in python 2.7, using the asynchronous apis like this will make you application perform much better. On 9 September 2011 06:18, Rishi Arora wrote: > Actually I thought using RPCs is a good way to achieve parallelism in > python, before python 2.7 comes along. My app does a lot of URL fetches > from external websites. This is very inefficient in terms of GAE instance > uptime. If the external website takes 30 seconds to respond, my instance > stays up for 30 seconds, and can't service any requests. So my solution has > been to use asynchronous URL fetch using create_rpc. If I call 10 URL > fetches asynchronously, and each takes 30 seconds to return, then my 30 > second GAE instance uptime is now amortized over 10 URL fetches instead of a > single one. Does my logic sound wrong? I have yet to implement this, but I > imagine this will lower my costs by allowing me to set a really low value > for MaxNumInstances (like 1 or 2), and by lowering my instance hours, > without sacrificing request latency. > > On Thu, Sep 8, 2011 at 2:35 PM, Brandon Thomson wrote: >> >> Good suggestion. At least until we have workable multithreadding I have >> become very reluctant to use RPCs unless absolutely necessary. It helps keep >> the instance number down. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To view this discussion on the web visit >> https://groups.google.com/d/msg/google-appengine/-/4UBt0EEiOj4J. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Is there a hard deadline when apps will switch to new pricing
Please see the thread: https://groups.google.com/group/google-appengine/browse_thread/thread/ac9ed0ea83e4d4e1 On 9 September 2011 04:54, Ikai Lan (Google) wrote: > We had promised previously that we would not turn on the new pricing before > September 26th. We'll make a formal announcement of a date in the next few > days. > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > plus.ikailan.com | twitter.com/ikai > > > On Thu, Sep 8, 2011 at 6:09 AM, Rishi Arora > wrote: >> >> We're in the process of optimizing our application, but are not yet >> finished doing so. We're wondering what day will Google App Engine leave >> preview made and switch to the new pricing scheme. Anyone know if there >> will be a formal communication the day before, or the hour before this >> happens? I think I read something like mid-September. Is that true? >> Rishi >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] I can't turn on datastore admin from dashboard.
Please visit this page in a new tab. This will ask you to log in and then the datastore admin page will work inside admin console. This is due to the login page not working if it is iframed. https://ah-builtin-python-bundle-dot-latest-dot-info-most.appspot.com On 15 September 2011 07:52, rekby wrote: > I click button "Enable datastore admin" many days ago. Now I see white paper > instead datastore admin page. > My app-id: info-most. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/ZgP2Q2H0AaIJ. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] What happens to URLFetch Async Call that is left alone?
What happens when you start an RPC, but not wait for it to finish is not really defined. As you have may have noticed, it also works differently in production and in the dev appserver. Also note that since it is not defined, the behaviour may change at anytime. I would suggest that you should also wait for an rpc call to finish before returning from your handler. This will also allow you to determine if the rpc call succeeded or not. On 23 September 2011 12:52, keakon lolicon wrote: > The SDK will automatically create a RPC object for async function, or you > can create it by yourself. > Before the end of your response, the RPC object will block the respond and > wait until it's done, and no callback will be called. > However, it looks like the local dev server just ignores it. > > -- > keakon > My blog(Chinese): www.keakon.net > Blog source code: https://bitbucket.org/keakon/doodle/ > > > On Fri, Sep 23, 2011 at 8:17 AM, Albert wrote: >> >> Suppose I call URLFetch Async, and then I return a response to the >> user without calling get_result(). >> >> 1. Does a frontend instance wait for the response of the fetched URL, >> and consume instance time? Or does some background appengine server >> handle that for me, and then just discards the response of the url >> because I never called get_result()? >> >> 2. Is the behavior the same for datastore async, memcache async, and >> other async calls? >> >> Thanks! >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: What happens to URLFetch Async Call that is left alone?
It is undefined, and thus you should not do it. On 23 September 2011 14:35, Albert wrote: > Thanks! > > How about in the Production environment? What's the behavior there? > > On Sep 22, 10:52 pm, keakon lolicon wrote: >> The SDK will automatically create a RPC object for async function, or you >> can create it by yourself. >> Before the end of your response, the RPC object will block the respond and >> wait until it's done, and no callback will be called. >> However, it looks like the local dev server just ignores it. >> >> -- >> keakon >> >> My blog(Chinese):www.keakon.net >> Blog source code:https://bitbucket.org/keakon/doodle/ >> >> >> >> >> >> >> >> On Fri, Sep 23, 2011 at 8:17 AM, Albert wrote: >> > Suppose I call URLFetch Async, and then I return a response to the >> > user without calling get_result(). >> >> > 1. Does a frontend instance wait for the response of the fetched URL, >> > and consume instance time? Or does some background appengine server >> > handle that for me, and then just discards the response of the url >> > because I never called get_result()? >> >> > 2. Is the behavior the same for datastore async, memcache async, and >> > other async calls? >> >> > Thanks! >> >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "Google App Engine" group. >> > To post to this group, send email to google-appengine@googlegroups.com. >> > To unsubscribe from this group, send email to >> > google-appengine+unsubscr...@googlegroups.com. >> > For more options, visit this group at >> >http://groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Any plans to open TaskQueueFetchQueueStats?
If you would like this feature, please file feature request on the
issue tracker: http://code.google.com/p/googleappengine/issues/list.
You can currently get the same details by looking at the admin
console: https://appengine.google.com/
On 22 September 2011 05:26, Jason Collins wrote:
> There are API points to retrieve task queue stats. E.g.,
>
> request = taskqueue_service_pb.TaskQueueFetchQueueStatsRequest()
> response = taskqueue_service_pb.TaskQueueFetchQueueStatsResponse()
> e.MakeSyncCall('taskqueue', 'FetchQueueStats', request, response)
>
> However, these seem to return Application Error: 9 (permission
> denied).
>
> Any plans to simply flip this on so that we can have a peek at our
> queues?
> j
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: public, dynamic backends are magical (in a good way!!)
This should never occur. A cron task should only run on one instance. If you have specified the 'target' parameter then it should execute on that version/backend otherwise it will execute on the default version of your application. What is your app id so I can look into this? On 25 September 2011 17:49, pman wrote: > i found a new problem - the cron job run twice (frontend and > backend). > > how to specify frontend cron job only? > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Using other code in a backend
Yes it is quite easy to do. The easiest way is to create a separate directory containing the code for the backend, then just upload the backend (be sure to not accidentally upload the code over your default version). Though be careful with configuration that is not scoped to a specific version: indexes, queues, cron, dos. You should ensure that you specify the same information in the configuration for both languages. Testing a solution that uses multiple languages can be quite difficult (and is generally easiest to test the application by deploying to a test app-id). On 27 September 2011 04:54, Hans Then wrote: > Hi all, > > Is it possible to create a backend containing another program than the > main instance? Specifically, is it possible to create a backend using > a different programming language e.g. Java in the backend and Python > in the main instance? > > Regards, > > Hans Then > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Using other code in a backend
No, It is best to test the interaction of the two versions by deploying them. On 28 September 2011 16:17, Hans Then wrote: > Will this also work in the development server? I have not seen an > option to start the Java development server and the Python development > server with the same datastore. > > Regards, > > Hans Then > > On Sep 27, 5:53 am, "Greg Darke (Google)" > wrote: > > Yes it is quite easy to do. The easiest way is to create a separate > > directory containing the code for the backend, then just upload the > > backend (be sure to not accidentally upload the code over your default > > version). > > > > Though be careful with configuration that is not scoped to a specific > > version: indexes, queues, cron, dos. You should ensure that you > > specify the same information in the configuration for both languages. > > > > Testing a solution that uses multiple languages can be quite difficult > > (and is generally easiest to test the application by deploying to a > > test app-id). > > > > On 27 September 2011 04:54, Hans Then wrote: > > > > > > > > > > > > > > > > > Hi all, > > > > > Is it possible to create a backend containing another program than the > > > main instance? Specifically, is it possible to create a backend using > > > a different programming language e.g. Java in the backend and Python > > > in the main instance? > > > > > Regards, > > > > > Hans Then > > > > > -- > > > You received this message because you are subscribed to the Google > Groups "Google App Engine" group. > > > To post to this group, send email to google-appengine@googlegroups.com > . > > > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > > > For more options, visit this group athttp:// > groups.google.com/group/google-appengine?hl=en. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Python 2.7, threadsafe: true, and deferred.defer
Have you specified 'runtime: python' or 'runtime: python27'? If you are using 'runtime: python', then setting 'threadsafe: true' is not supported. On 14 October 2011 00:59, Emlyn wrote: > I am using the builtin handler in app.yaml > > On Oct 14, 2011 12:25 AM, "Greg Darke" wrote: >> >> On Fri, Oct 14, 2011 at 12:10:09AM +1030, Emlyn wrote: >> > I'm trying to give python 2.7 a go, with an app that uses >> > deferred.defer. I'm getting this error when I try to upload it: >> > >> > Error 400: --- begin server output --- >> > Error when loading application configuration: >> > Invalid object: >> > Threadsafe cannot be enabled with CGI handler: >> > $PYTHON_LIB/google/appengine/ext/deferred/handler.py >> > >> > Is there a workaround? Or should I even be using deferred.defer in this >> > context? >> >> The easiest way to solve this problem is to remove the explicit handler >> you are defining, and use the builtin directive as described in >> >> http://code.google.com/appengine/docs/python/config/appconfig.html#Builtin_Handlers >> >> If you are running the deferred handler at non-default url, you will >> need to specify the WSGI application in your app.yaml. This is >> google.appengine.ext.deferred.application >> > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] AssertionErrors from URLFetch RPCs
Can you please add the full stack trace? And if it is repeatable either a code snippet or an explanation of what your program was doing when the exception occured. On 14 November 2011 18:29, someone1 wrote: > I am continunously getting the following error for a segment of code > that has been working fine until just now: > > ... > result = rpc.get_result() > File "/base/python27_runtime/python27_lib/versions/1/google/ > appengine/api/apiproxy_stub_map.py", line 589, in get_result > self.check_success() > File "/base/python27_runtime/python27_lib/versions/1/google/ > appengine/api/apiproxy_stub_map.py", line 556, in check_success > self.wait() > File "/base/python27_runtime/python27_lib/versions/1/google/ > appengine/api/apiproxy_stub_map.py", line 530, in wait > assert self.__rpc.state != apiproxy_rpc.RPC.IDLE, repr(self.state) > AssertionError: 0 > > Is this an issue I can resolve or an AppEngine thing? My code runs in > a backend with python27 (threadsafe: false) > > Thanks, > Prateek > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Endless cron job
The datastore admin job does not use cron. Have you checked the cron tab in the admin-console to see if there are any cron tasks enabled in your app? Have you recently performed a migration from Master/Slave to High Replication and had cron enabled on the application? If so, you will need to disable the cron job on the old version of your application (or just delete the old version). On 20 December 2011 21:09, David Lindsey wrote: > Hi, > > There's a cron job that runs every minute spamming up my logs which I > think is an orphan of Datastore Admin delete job. That particular > delete request completed in that it deleted all the entities it was > supposed to but it said 2 jobs completed 1 active. It seemed hung like > that. Another post mentioned deleting the Datastore Admin entity so I > did, although I was skeptical, and that didn't have any effect. As far > as I know (I'm not the only person who can upload to the app) this app > has never had cron.yaml. I know it doesn't have one right now. > > Here's the Cron Jobs listing: > > /scheduled_process > Run processing of scheduled uploads every 1 minutes every 1 minutes > (UTC) > 2011/12/20 09:53:17 on time Success > > Here's what the logs show: > > 2011-12-20 02:05:23.477 /scheduled_process 200 31ms 3kb AppEngine- > Google; (+http://code.google.com/appengine) > 0.1.0.1 - - [20/Dec/2011:02:05:23 -0800] "GET /scheduled_process HTTP/ > 1.1" 200 3459 - "AppEngine-Google; (+http://code.google.com/ > appengine)" "" ms=32 cpu_ms=0 api_cpu_ms=0 > cpm_usd=0.000419 queue_name=__cron > task_name=0b7f457ee2ee7ab4fe9e71fcb34667fe > instance=00c61b117ccc5309d8c89143086451a9ded1d58f > > Does anyone know how to get rid of this? > > Thanks for reading, > David > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] IRC Office hours 20120321 9am PDT
Sorry for the times in the log, my computer is set to UTC+11. 02:44 <+proppy> hey there 02:48 < sec_> good holiday proppy :) 02:48 <+proppy> I'm visiting london :) 02:48 <+proppy> not really in holiday 02:49 * sec_ have read all GAE thing related (but slow) 02:54 <+proppy> how many people are there for the upcoming office hours? 02:55 < jwbnyc> ++ 02:55 < mbw> im around 02:56 < sec_> ++ 02:57 < jwbnyc> I'd like to know about the datastore admin tool: It appears that it's not useful when entities are under namespaces. 02:57 <+proppy> ok nice :) 02:57 < sec_> i have many newbie questions :( 02:57 <+proppy> anyone want's to do an hangout ? 02:57 <+proppy> or do you prefer IRC 02:58 < jwbnyc> No preference for me. 02:59 < jwbnyc> We're running with Java SDK 1.6.0; would the datastore admin function change if we deployed with a newer SDK? 03:02 <+proppy> let me start the hangout 03:03 <+proppy> https://plus.google.com/hangouts/9e7da0a525c0945ee8f7d6e641c44849153c90bd 03:03 < vlad___> good morning 03:03 <+proppy> here we are 03:04 < vlad___> do we chat here on on g+? 03:04 <+proppy> jwbnyc: which function are you referring to? 03:04 <+proppy> vlad___: we can do both :) 03:04 < jwbnyc> Would it be good to change the topic message so folks have a better chance of finding the hangout? 03:04 < vlad___> ok, is greg darke or someone working on TaskQueues around? 03:04 <+proppy> ok 03:04 <+proppy> forget the hangout :) 03:04 <+proppy> it is not working for me 03:05 <+proppy> let's do it here 03:05 <+proppy> jwbnyc: usually the datastore admin tool, get's updated with new SDK version 03:05 <+proppy> but the fact that you deploy (or not) your application doesn't affect the functionnality 03:07 < sec_> are gae python only for geek? i had read 1week on Docs but don't go to far.. any tip for newcomer to use GAE fast (template, webapp2, datastore). Thanks 03:08 < jwbnyc> I'm looking at an issue that was started March 1: http://code.google.com/p/googleappengine/issues/detail?id=3962 03:08 <+proppy> sec_: the python runtime is for everyone :) 03:08 < vlad___> Is greg darke or someone working on TaskQueues around? 03:08 <+proppy> sec_: it is actually the first runtime that was developped for App Engine 03:08 < vlad___> have a question on task scheduling 03:08 <+proppy> so there are many apps, and may resources about it 03:08 < dave_google> sec_: i've seen people have good luck by starting with the tutorial before diving deeper into the doc.s 03:08 <+proppy> vlad___: he will be there in a few minutes 03:09 <+proppy> sec_: I would recommend to try python27 tutorial, because it features webapp2, and jinja templates 03:09 < vlad___> thanks 03:09 <+proppy> sec_: http://code.google.com/appengine/docs/python/gettingstartedpython27/ 03:09 < jwbnyc> We poked at the admin tool and found no way to get to our data. 03:10 < sec_> thanks proppy dave_google 03:10 < jwbnyc> If our app deploys the 1.6.0 SDK does that mean we're lacking some enhancements to the admin tool in 1.6.3? 03:10 <+proppy> jwbnyc: will ask the eng team to followup 03:11 <+proppy> jwbnyc: nop, deploys should not affect the functionnality in the admin tool 03:11 < dave_google> jwbnyc: that'll be fixed in the 1.6.4 SDK 03:11 <+proppy> jwbnyc: here you go :) 03:12 < sec_> dave_google: where/how download sample code in tutorial? 03:12 < troberti> heh hangout plugin doesn't work 03:13 < troberti> ok chrome restart did the trick 03:13 < dave_google> sec_: you can copy/paste it directly from the tutorial pages. 03:13 < jwbnyc> (y) 03:14 < sec_> troberti: did you see bug in index.yaml? 03:14 <+proppy> troberti: I cancelled the hangout 03:14 <+proppy> troberti: I have issue with my webcam :( 03:14 < troberti> sec_: no I am working, not debugging other peoples tutorials :) 03:14 < troberti> proppy: aha 03:15 < dave_google> sec_: there's also http://googcloudlabs.appspot.com/which has a different tutorial, also put together by googlers. 03:15 < troberti> was just lurking there anyway :) 03:16 < sec_> dave_google: http://code.google.com/appengine/docs/python/gettingstartedpython27/usingdatastore.html|| i try that index.yaml on real server( not devserver in SDK), and got 500 error. what is it? 03:16 <+proppy> moar questions? 03:16 < dave_google> sec_: did you register an application and change the application: line? 03:17 < BlartShart> Is there a way to programatically invoke the admin console's datastore backup from a java app? I'd like to schedule that with a cron job 03:17 < sec_> dave_google: i did 03:18 < dave_google> sec_: oops, scratch that second tutorial. you're doing python. 03:18 < dave_google> sec_: does anything of use show up on the application logs (via the admin console)? 03:18 < jwbnyc> Any projection for (more) availability of SSL for custom domains? 03:18 < troberti> ok lazy question: Is it possible to pack all entities of a kind in a big blob and then transfer that to another app? 03:18 < sec_> dave_google: yes, let me login 03:19 < troberti> lazy because I coul
Re: [google-appengine] Re: Tasks and rate- throttled urls
Hi Justin, I checked the code to confirm this issue. If you would like this functionality in the development appserver, please create a bug on the issue tracker. On 17 February 2011 00:44, Justin wrote: > Ok thanks - does it say this in the docs somewhere ? or are you > looking through the source ? > > On Feb 16, 1:27 pm, Greg Darke wrote: >> Hi Justin, >> >> Currently dev_appserver.py does not respect the rate or bucket size >> (Only the eta). >> >> On 16 February 2011 21:15, Justin wrote: >> >> >> >> > Hi Nick, >> >> > I have; doesn't seem to work [all tasks are still executed in a >> > burst]; even if I set bucket size to 1 >> >> > Is it possible this is a dev_appserver.py issue ? [I haven't tried it >> > on the production servers yet] >> >> > Also it might be worth updating the docs to make it clear you can use >> > floats in the queue.yaml rate field [it wasn't clear to me you could >> > use '0.1/s'] >> >> > Thanks, >> >> > Justin >> >> > On Feb 15, 11:45 pm, "Nick Johnson (Google)" >> > wrote: >> >> Hi Justin, >> >> >> Have you tried 0.1/s as a rate? >> >> >> -Nick >> >> >> On Wed, Feb 16, 2011 at 10:40 AM, Justin wrote: >> >> > I have generated some tasks which call the same URL with different >> >> > parameters. Unfortunately the URL is rate throttled - I can only make >> >> > one call every 10 secs - so I want to configure queue.yaml to make one >> >> > call every 10 secs. >> >> >> > I started by setting the rate parameter at 6/m, but all the tasks were >> >> > executed immediately. On closer reading of the queue.yaml docs, I can >> >> > see there's nothing to stop app engine executing 6 tasks per minute in >> >> > a single burst (rather than every 10 secs). >> >> >> > The best I can do at the minute is to use a rate of 1/m, so a task is >> >> > executed every 60 secs; anything more than 1/m triggers the URL >> >> > throttle. >> >> >> > Is there some combination of queue.yaml params that can get the >> >> > interval down to 10 secs ? >> >> >> > -- >> >> > You received this message because you are subscribed to the Google >> >> > Groups >> >> > "Google App Engine" group. >> >> > To post to this group, send email to google-appengine@googlegroups.com. >> >> > To unsubscribe from this group, send email to >> >> > google-appengine+unsubscr...@googlegroups.com. >> >> > For more options, visit this group at >> >> >http://groups.google.com/group/google-appengine?hl=en. >> >> >> -- >> >> Nick Johnson, Developer Programs Engineer, App Engine >> >> Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number: >> >> 368047 >> >> > -- >> > You received this message because you are subscribed to the Google Groups >> > "Google App Engine" group. >> > To post to this group, send email to google-appengine@googlegroups.com. >> > To unsubscribe from this group, send email to >> > google-appengine+unsubscr...@googlegroups.com. >> > For more options, visit this group >> > athttp://groups.google.com/group/google-appengine?hl=en. >> >> -- >> Greg Darke > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Pass Params to Cron Jobs?
Hi Daniel, This will work, though you will need to put a '/' character at the start of the url. On 13 March 2011 07:30, Daniel wrote: > I want to do something like this.. will it work? > cron: > - description: Resign Games > url: rgame?daysOld=5 > schedule: every 2 hours synchronized > I'm not seeing examples in the docs of passing static params to a url but > feels like this should be possible? > Thanks > Daniel > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Unexpected errors importing Django 1.2
Hi Nathanael,
You should also put the following in your appengine_config.py file:
webapp_django_version = '1.2'
This will make webapp.template use the specified version of django.
This is needed since the dev admin-console, appstats and the
map-reduce framework all use webapp.template to render their output.
On 10 March 2011 09:52, Johan Euphrosine wrote:
> On Wed, Mar 9, 2011 at 11:41 PM, Nathanael Abbotts
> wrote:
>> Can anyone identify what is causing the following error?
>>
>
> Hi Nathanael,
>
> Be sure to insert use_library('django', '1.2') in every script handler
> that import django.
>
> References:
> http://code.google.com/appengine/docs/python/tools/libraries.html#Django
> http://stackoverflow.com/questions/4994913/app-engine-default-django-version-change
>
> Hope that helps.
> --
> Johan Euphrosine (proppy)
> Developer Programs Engineer
> Google Developer Relations
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] SDK 1.4.3 has been released!
On 31 March 2011 09:21, Robert Kluin wrote: > Cool; quite a few bug fixes. > > To specify a version for a task queue, do you use set a 'target' in > queue.yaml? Yes, that is correct. You also use the same parameter for cron tasks. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
