Re: [appengine-java] Setting Namespace for an entire Request
Namespace-api is set normally in a Servlet Filter, which is normally the beginning point for most application HttpRequests. This means, there is a request coming in, it is serviced by a single-thread. Namespace-api is threadsafe and the namespace is put in the threadlocal, and hence, the same namespace is available during the complete HttpRequest serviced by that thread. -Aswath On Fri, May 13, 2011 at 9:13 PM, mpire sebastian.wennin...@googlemail.comwrote: Hi! I'm trying to use the Namespace-API to build a multi-tenant app. Therefore i have defined a filter that sets the Namespace for each request regarding to the User making the request. However, if there's a new Request before the current request finishes its datastore-operation, the namespace would be overwritten by the new request and the current operation would have an incorrect namespace. I could also set the namespace before each datastore operation, but the problem would be the same.If another request overwrites the namespace before the operation is finished, the data in the database will be messed. Is there a way to set the namespace for an entire request? Or should i try to build some kind of Mutex around the NamespaceManager? Any suggestions to solve this problem are welcome :) Thx in advance -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] HTTP headers and protocol version not consistent on client and google app engine
I am trying to access http header Accept-Encoding and http protocol version on google app engine. But Accept-Encoding never reaches while protocol version is always 1.1 (even when client side shows HTTP/1.0). Am i missing some sort of setting in my application? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Object visibility between threads (running requests)
Maybe you could use the AppEngine cache? You have the guarantee that objects in cache are replicated across instances. However you need to be extra careful in your locking mechanism as Object.wait() and Object.notify() obviously won't work. I guess some sort of polling can do it, if you don't need extra reactivity. Afterthought: I'm wondering if cache object modification is seen immediately on other instance, or if there is some kind of transactional mode within a request, that makes the cache appear consistent for a whole request. You have to check that too, I guess. --Laurent On May 13, 7:34 pm, Samuel Erdtman sam...@erdtman.se wrote: That is true, I just want to know that it is by design and not because I have missed to do something. And the thing is that i can see the objects and that there is difference in visibility after the first thread returns. does this mean that objects are synchronized between data centers between request. I´ll see if i figure something out //Samuel -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] stderr: SystemId Unknown; Line #57; Column #31; Failed calling setMethod method
My log is full of this weird stderr message [vedmack/1.350345457568997204].stderr: SystemId Unknown; Line #57; Column #31; Failed calling setMethod method Its an Info message... any ideas to what can cause this? Thanks ahead! Daniel, -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: stderr: SystemId Unknown; Line #57; Column #31; Failed calling setMethod method
Hi, Look at http://groups.google.com/group/google-appengine-java/browse_thread/thread/f6f40411a076aa30/1b0f2adb19ce0997?lnk=gstq=Failed+calling+setMethod+method+#1b0f2adb19ce0997 At the very bottom, you may have the answer to you question regards didier On May 14, 11:25 am, Daniel vedm...@gmail.com wrote: My log is full of this weird stderr message [vedmack/1.350345457568997204].stderr: SystemId Unknown; Line #57; Column #31; Failed calling setMethod method Its an Info message... any ideas to what can cause this? Thanks ahead! Daniel, -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Backends.xml ERROR when deploying
I put that in my xml file : backends backend name=tastBackend classb1/class instances1/instances /backend /backends and i still get the same error when try to deploy. Something looks wrong in this xml? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Facebook Login integration with java appengine
Hi, I am using http://code.google.com/p/facebook-java-api/ to login using facebook. But unable to do so as documenttaion is not quite completed there and posted my problem on this site but long time no response. I am sure in this group many people must have done with their java app to login using facebook. Could you please share your code or give me some guidance. Requirement: I will be dispalying google/yahoo/facebook logo on my site and when user click on Facebook logo it will take it to facebook page to authenticate.Once authenticate it will call my callback url and i should be able to query logged in user's name etc using facebook api(java). I am sorry posting a facebook login question here, but the project it self seems inactive and no one is responding. Thanks in advance Ravi. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] How To: Persist an entity, if and only if, another successfully persists
Hey I have two entities I update and which to persist. These are in two different entity groups so I cannot update and persist them within a Transaction. I am using JDO pm.makePersistentAll and must ensure each entity only saves to the datastore if both can be written. What's the best way to do this? Thanks -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Vs: Re: Stripes + UrlBinding + Google App Engine
Hello
I had some time to try this again, but I still have no success.
I have an ActionBean with @UrlBinding(/home). The @DefaultHandler method
ForwardResolutions to index.jsp which only prints Hello:
@DefaultHandler
public Resolution home() {
return new ForwardResolution(index.jsp); // Also tried /index.jsp
}
In dev environment I access my app via http://localhost:/index.jsp and
http://localhost:/home. Both print Hello. That is, it works perfectly.
After I have deployed my app on appengine I access the application via
http://myapp.appspot.com/index.jsp. This prints Hello, but when I access
my app via http://myapp.appspot.com/home I get an infinite redirect loop.
I checked the app engine logs:
/home/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp
/home/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp
/home/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp
/home/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp
/home/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp
/home/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp
/home/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp
/home/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp
/home/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp
/home/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp
/home/index.jsp/index.jsp/index.jsp/index.jsp/index.jsp
/home/index.jsp/index.jsp/index.jsp/index.jsp
/home/index.jsp/index.jsp/index.jsp
/home/index.jsp/index.jsp
/home/index.jsp
/home/
I consider this more an app engine issue than Stripes issue because
everything works locally but not in app engine. Stripes uses a servlet
filter to handle the UrlBindings. The filter javadocs states:
This filter takes the following approach to determining when to dispatch an
ActionBeanhttp://stripes.sourceforge.net/docs/current/javadoc/net/sourceforge/stripes/action/ActionBean.html:
1. Allow the request to process normally, trapping any HTTP errors that
are returned.
2. If no error was returned then do nothing, allowing the request to
complete successfully. If any error other than 404 was returned then send
the error through. Otherwise ...
3. Check the
ActionResolverhttp://stripes.sourceforge.net/docs/current/javadoc/net/sourceforge/stripes/controller/ActionResolver.htmlto
see if an
ActionBeanhttp://stripes.sourceforge.net/docs/current/javadoc/net/sourceforge/stripes/action/ActionBean.htmlis
mapped to the URL. If not, then send the
404 error through. Otherwise...
4. Invoke
StripesFilterhttp://stripes.sourceforge.net/docs/current/javadoc/net/sourceforge/stripes/controller/StripesFilter.htmland
DispatcherServlethttp://stripes.sourceforge.net/docs/current/javadoc/net/sourceforge/stripes/controller/DispatcherServlet.html
So it expects a 404. This all seems like the app engine doesn't like this
behaviour, thus the infinte redirect loop.
I have tried to clean my configuration as much as possible:
- I only define the DynamicMappingFilter in my web.xml. No StripesFilter
anymore. According to the Stripes documentation this is perfectly ok.
- I also had Guice stuff in my web.xml but I removed them as well.
- I also configured Stripes to use my own empty implementation of the
MultipartWrapperFactory.
I am running out of ideas. Would be nice if could be confirmed that this
error indeed is reproducable and not just my misconfiguration.
Thanks
Pauli Savolainen
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Facebook Login integration with java appengine
We use restFB http://restfb.com/; however, we wrote our own authentication. Our application is a FB Canvas app. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Strange failure last night
Sometime between 10:35 and 10:40 last evening, our app became unable to find datastore entities. The app was pretty much inactive - there were two clients that were successful shortly before 10:35 and then a couple of clients that arrived shortly after 10:40 couldn't authenticate. The log entries reported not-found when retrieving the authentication entities from the datastore. Looking at the datastore viewer in the GAE console, those entities do appear. I deployed the same code to a new version and it's working OK, again. I know this isn't much useful information, but I wonder if any other apps have encountered this apparently transient type of failure. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Strange failure last night
To clarify - the failure began between 22:35PT and 22:40PT and persisted until we deployed the same code to a new version. (We didn't try deploying on top of the default version that was having the problem.) -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Facebook Login integration with java appengine
Don't waste your time with facebook-java-api. It's ancient. You're much better off with RestFB or BatchFB (disclosure: I develop BatchFB). Use the Javascript SDK for login. Don't try to do OAuth yourself - aside from periodic changes to the protocol, it's not nearly as smooth for users. BTW, sometime soon BatchFB is going to get parallel fetching on appengine - in combination with the batching mechanisms, this will be *the* fastest way to pull facebook data to GAE. Jeff On Sat, May 14, 2011 at 7:35 AM, Ravi Sharma ping2r...@gmail.com wrote: Hi, I am using http://code.google.com/p/facebook-java-api/ to login using facebook. But unable to do so as documenttaion is not quite completed there and posted my problem on this site but long time no response. I am sure in this group many people must have done with their java app to login using facebook. Could you please share your code or give me some guidance. Requirement: I will be dispalying google/yahoo/facebook logo on my site and when user click on Facebook logo it will take it to facebook page to authenticate.Once authenticate it will call my callback url and i should be able to query logged in user's name etc using facebook api(java). I am sorry posting a facebook login question here, but the project it self seems inactive and no one is responding. Thanks in advance Ravi. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: stderr: SystemId Unknown; Line #57; Column #31; Failed calling setMethod method
Hehe :) That's a thread I opened almost a year ago (on a completely diff issues) happens to be that in a log I posted back than the weird stderr was appearing too,.. On May 14, 12:46 pm, Didier Durand durand.did...@gmail.com wrote: Hi, Look athttp://groups.google.com/group/google-appengine-java/browse_thread/th... At the very bottom, you may have the answer to you question regards didier On May 14, 11:25 am, Daniel vedm...@gmail.com wrote: My log is full of this weird stderr message [vedmack/1.350345457568997204].stderr: SystemId Unknown; Line #57; Column #31; Failed calling setMethod method Its an Info message... any ideas to what can cause this? Thanks ahead! Daniel, -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Facebook Login integration with java appengine
Hi, I have open sourced a Java API which works on the latest FB Graph API. There's oAuth 2.0 in it. I have made it for our own app. If authentication is the only priority for you then you can definitely use it. https://github.com/nischal/face4j Let me know if you need any help integrating it. I've tried to make it as simple as possible. It's very weak on documentation though but here's an example for authentication - https://github.com/nischal/face4j/wiki/Facebook-oAuth-login-with-face4j -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Facebook Login integration with java appengine
And it works well on the Appengine! -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Facebook Login integration with java appengine
Thanks all for your replys. SOme how i made it work with facebook_java_api project and posted the tutorial here http://techsravi.blogspot.com/2011/05/facebook-login-on-google-app-enginejava.html face4j looks quite promising too...i will look into it too. Thanks Ravi. On Sat, May 14, 2011 at 7:15 PM, Nischal nischalshett...@gmail.com wrote: And it works well on the Appengine! -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] How To: Persist an entity, if and only if, another successfully persists
Depending on your application, you may be able to perform the 2nd entity write in a task. Enqueue the task transactionally with the first operation. You'll be guaranteed that if the first entity commits, the 2nd will *eventually* be written as well. It's not a perfect solution for all apps but it does cover a lot of practical use cases. Jeff On Sat, May 14, 2011 at 7:52 AM, mscwd01 mscw...@gmail.com wrote: Hey I have two entities I update and which to persist. These are in two different entity groups so I cannot update and persist them within a Transaction. I am using JDO pm.makePersistentAll and must ensure each entity only saves to the datastore if both can be written. What's the best way to do this? Thanks -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] JPA cross-reference table question
Hello gurus. I'm trying to design an application that allows users to rate
movies with a star system of 1-5 stars. For that I need a user table, a
movie table, and an X-ref table to link them as each user can rate multiple
movies and each movie can be rated by multiple users. I can't define this
relationship correctly with JPA and I need your help. Has anyone defined
this kind of relationship or know where a tutorial/example is? Here is what
I have which throws the following error:
javax.persistence.PersistenceException: App Engine ORM does not support
multiple parent key provider fields.
package com.app.movieRatings.model;
import java.util.ArrayList;
import java.util.List;
import javax.persistence.CascadeType;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.OneToMany;
import com.google.appengine.api.datastore.Key;
@Entity
public class User {
@Id
@GeneratedValue(strategy=GenerationType.IDENTITY)
private Key id;
private String userId;
@OneToMany(mappedBy = user, cascade = CascadeType.ALL)
private ListUserMovieXRef UserMovieXRefs = new
ArrayListUserMovieXRef();
// Getters and setters...
}
package com.app.movieRatings.model;
import java.util.ArrayList;
import java.util.List;
import javax.persistence.CascadeType;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.OneToMany;
import com.google.appengine.api.datastore.Key;
@Entity
public class Movie {
@Id
@GeneratedValue(strategy=GenerationType.IDENTITY)
private Key id;
private String name;
@OneToMany(mappedBy = movie, cascade = CascadeType.ALL)
private ListUserMovieXRef UserMovieXRefs = new
ArrayListUserMovieXRef();
// Getters and setters...
}
package com.app.movieRatings.model;
import javax.persistence.Entity;
import javax.persistence.FetchType;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.ManyToOne;
import com.google.appengine.api.datastore.Key;
@Entity
public class UserMovieXRef {
@Id
@GeneratedValue(strategy=GenerationType.IDENTITY)
private Key id;
private int starRating;
@ManyToOne(fetch = FetchType.LAZY)
private User user;
@ManyToOne(fetch = FetchType.LAZY)
private Movie movie;
// Getters and setters...
}
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Properties With Multiple Values Can Have Surprising Behaviors
at this link http://code.google.com/appengine/docs/java/datastore/queries.html#Restrictions_on_Queries i found this topic : Properties With Multiple Values Can Have Surprising Behaviors but i couldn't understand how can i solve this issue and why this problem occur ??? can anyone help me in this issue ? thanks.. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Facebook Login integration with java appengine
We used Janrain which has the added advantage of supporting N networks instead of just one. One caveat is that it makes the login slower because you go through Janrain redirects during login so test this to ensure it's fast enough for you if you try it. Steve On May 15, 12:35 am, Ravi Sharma ping2r...@gmail.com wrote: Hi, I am usinghttp://code.google.com/p/facebook-java-api/to login using facebook. But unable to do so as documenttaion is not quite completed there and posted my problem on this site but long time no response. I am sure in this group many people must have done with their java app to login using facebook. Could you please share your code or give me some guidance. Requirement: I will be dispalying google/yahoo/facebook logo on my site and when user click on Facebook logo it will take it to facebook page to authenticate.Once authenticate it will call my callback url and i should be able to query logged in user's name etc using facebook api(java). I am sorry posting a facebook login question here, but the project it self seems inactive and no one is responding. Thanks in advance Ravi. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
RE: [google-appengine] How to find IP address to add to blacklist
I lied, the stub code I start my projecta with has
logging.debug('User-Agent = %s, Referrer = %s',
self.request.user_agent,
self.request.referer)
Which will give you this
-Original Message-
From: google-appengine@googlegroups.com
[mailto:google-appengine@googlegroups.com] On Behalf Of Brandon Wirtz
Sent: Friday, May 13, 2011 6:37 PM
To: google-appengine@googlegroups.com
Subject: RE: [google-appengine] How to find IP address to add to blacklist
Logs should have the ip address as the first thing on the second line of
each entry
-Original Message-
From: google-appengine@googlegroups.com
[mailto:google-appengine@googlegroups.com] On Behalf Of Kwame
Sent: Friday, May 13, 2011 6:22 PM
To: Google App Engine
Subject: [google-appengine] How to find IP address to add to blacklist
Hi,
I need help tracking down a possible hacker trying to access my GAE website.
The person is sending multiple requests to phpMyAdmin/scripts/ setup.php I
guess in an attempt to bring down my site!!! Fortunately they don't know I
use GAE, and that GAE does not have this file.
I really need to blacklist this persons URI, but the only information
provided in Dashboard is the path listed above.
How can I find out their URI?
Thanks for your help.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] How to find IP address to add to blacklist
I don't do that but I can still see the IPs. May be your lie works in my
case :P
On 14 May 2011 13:09, Brandon Wirtz drak...@digerat.com wrote:
I lied, the stub code I start my projecta with has
logging.debug('User-Agent = %s, Referrer = %s',
self.request.user_agent,
self.request.referer)
Which will give you this
-Original Message-
From: google-appengine@googlegroups.com
[mailto:google-appengine@googlegroups.com] On Behalf Of Brandon Wirtz
Sent: Friday, May 13, 2011 6:37 PM
To: google-appengine@googlegroups.com
Subject: RE: [google-appengine] How to find IP address to add to blacklist
Logs should have the ip address as the first thing on the second line of
each entry
-Original Message-
From: google-appengine@googlegroups.com
[mailto:google-appengine@googlegroups.com] On Behalf Of Kwame
Sent: Friday, May 13, 2011 6:22 PM
To: Google App Engine
Subject: [google-appengine] How to find IP address to add to blacklist
Hi,
I need help tracking down a possible hacker trying to access my GAE
website.
The person is sending multiple requests to phpMyAdmin/scripts/ setup.php I
guess in an attempt to bring down my site!!! Fortunately they don't know I
use GAE, and that GAE does not have this file.
I really need to blacklist this persons URI, but the only information
provided in Dashboard is the path listed above.
How can I find out their URI?
Thanks for your help.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
-Nischal
+91-9920240474
twitter: NischalShetty http://twitter.com/nischalshetty
facebook: Nischal http://facebook.com/nischal
http://www.justunfollow.com http://www.buffr.com
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] How to find IP address to add to blacklist
I didn't try remming the line. I was just looking at my code and thinking .
Oh maybe people don't see what I see :-)
That code does the user agent which the other thread about blocking bots
needs. I'm being helpful, just on the wrong thread :-)
From: google-appengine@googlegroups.com
[mailto:google-appengine@googlegroups.com] On Behalf Of Nischal Shetty
Sent: Saturday, May 14, 2011 12:51 AM
To: google-appengine@googlegroups.com
Subject: Re: [google-appengine] How to find IP address to add to blacklist
I don't do that but I can still see the IPs. May be your lie works in my
case :P
On 14 May 2011 13:09, Brandon Wirtz drak...@digerat.com wrote:
I lied, the stub code I start my projecta with has
logging.debug('User-Agent = %s, Referrer = %s',
self.request.user_agent,
self.request.referer)
Which will give you this
-Original Message-
From: google-appengine@googlegroups.com
[mailto:google-appengine@googlegroups.com] On Behalf Of Brandon Wirtz
Sent: Friday, May 13, 2011 6:37 PM
To: google-appengine@googlegroups.com
Subject: RE: [google-appengine] How to find IP address to add to blacklist
Logs should have the ip address as the first thing on the second line of
each entry
-Original Message-
From: google-appengine@googlegroups.com
[mailto:google-appengine@googlegroups.com] On Behalf Of Kwame
Sent: Friday, May 13, 2011 6:22 PM
To: Google App Engine
Subject: [google-appengine] How to find IP address to add to blacklist
Hi,
I need help tracking down a possible hacker trying to access my GAE website.
The person is sending multiple requests to phpMyAdmin/scripts/ setup.php I
guess in an attempt to bring down my site!!! Fortunately they don't know I
use GAE, and that GAE does not have this file.
I really need to blacklist this persons URI, but the only information
provided in Dashboard is the path listed above.
How can I find out their URI?
Thanks for your help.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
mailto:google-appengine%2bunsubscr...@googlegroups.com .
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
mailto:google-appengine%2bunsubscr...@googlegroups.com .
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com
mailto:google-appengine%2bunsubscr...@googlegroups.com .
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
-Nischal
+91-9920240474
twitter: NischalShetty http://twitter.com/nischalshetty
facebook: Nischal http://facebook.com/nischal
http://www.justunfollow.com/ Description: Image removed by sender.
http://www.buffr.com/ Description: Image removed by sender.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
image001.jpgimage002.jpg
[google-appengine] Datastore cost in new price
I want make full backup. Do query SELECT * FROM MyTable bill as only one API call if I have some thoutands/millions records? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: 1.5 improvements Make me less scared of Pricing
I suspected that might be the case, but if I have a few minutes to spare sometime I might try it out. My startup and servicing costs is pretty minimal - I've got a one-page-webapp and most of my calls are just AJAX calls to load and update data, so it's typically just object to JSON and back again. So given the way the API calls are counted and how my traffic works, I might look at getting rid of using the datastore for fields and just store what was a collection of items as a single larger JSON text blob (and look at moving to the blobstore), and tune the caching in the client in terms of writing back changes. I suppose a dynamic API for the scheduler might make it a bit tricky for GAE to plan (at the small scale) how to schedule work on machines, but if it was more like a declaration of hints at startup about how long I'd prefer to hang around when idle before being killed etc - even a declaration in app.yaml - then I think it might go some way to allaying some of the concerns being voiced. -- T -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Re: 1.5 improvements Make me less scared of Pricing
I don't know what I want instances to do.. I am finishing up a complete re-write of my code so that if I have to move to Java or GO that my code is perfect in Python and I can pay for a port, not a re-write. On the App I'm baking the code to make sure I didn't blow anything up on, I'm now cruising at 16 instances and using .025 CPU seconds a second. So why I need 16 instances if I'm only doing 1/40th of a CPU worth of work is beyond me. That'd imply I've got 640x as much horse power as I need. Also there are a lot of things you can do much better than the example code would imply. I took request to first bit from 330ms down to 120ms. And I think I can get that to 70ms. I could probably be more aggressive if I didn't have error handling. Who needs that I never make errors. I tried several things with regards to how .py was loaded and the CGIhandler, I can't measure any difference. If anyone cares zipping data makes it faster out of datastore, a tie out of memcache, big (400k) wins big if the compression is good, and you lose on anything under 20k. Assumes 25% average compression. Considered zip based on size. (may do that) Anything worth writing to MemCache is probably worth writing to a scratchpad datastore that you access by keyname. The writes are cheap compared to a cache miss on memcache, or you wouldn't have memcached it would you? YMMV From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Tim Sent: Saturday, May 14, 2011 1:42 AM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] Re: 1.5 improvements Make me less scared of Pricing I suspected that might be the case, but if I have a few minutes to spare sometime I might try it out. My startup and servicing costs is pretty minimal - I've got a one-page-webapp and most of my calls are just AJAX calls to load and update data, so it's typically just object to JSON and back again. So given the way the API calls are counted and how my traffic works, I might look at getting rid of using the datastore for fields and just store what was a collection of items as a single larger JSON text blob (and look at moving to the blobstore), and tune the caching in the client in terms of writing back changes. I suppose a dynamic API for the scheduler might make it a bit tricky for GAE to plan (at the small scale) how to schedule work on machines, but if it was more like a declaration of hints at startup about how long I'd prefer to hang around when idle before being killed etc - even a declaration in app.yaml - then I think it might go some way to allaying some of the concerns being voiced. -- T -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Datastore cost in new price
It's undecided but it will probably be billed per RPC. (The query is divided into multiple RPCs under the covers.) On May 14, 11:34 am, rekby timofey.koo...@gmail.com wrote: I want make full backup. Do query SELECT * FROM MyTable bill as only one API call if I have some thoutands/millions records? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Lot's of requests per user, aggressive caching or what?
Thanks I'll check this out. Is there anything I can to do optimize bandwidth used. Each of those 50,000 requests are completely different entities and have different uris, (/api/articles/46452) On May 13, 10:10 pm, Gopal Patel patelgo...@gmail.com wrote: you can use in-instance caching. It may not be always up to date but will give you extreme speeds. http://appengine-cookbook.appspot.com/recipe/decorator-to-getset-from... checkout fifth one. On Sat, May 14, 2011 at 4:37 AM, Ricky Button m...@rickybutton.com wrote: At a peak, a user of my application can make around 30,000 - 50,000 requests a day that returns some xml data. The xml data itself is cached so the request is only doing two RPC calls (one for authentication, one for the xml string). However, this seems to use a lot of CPU time and outgoing bandwidth. I've tried using memcache for the xml data but it turns out that it is usually a low hit ratio so it is quite a bit slower. Is there any form of aggressive caching or method I can use to limit the resources used over millions of requests per day? P.S. I have looked at this article http://www.kyle-jensen.com/proxy-caching-on-google-appengine and figured that it would not work well in my scenario because I want to be able to authenticate on each request. Thanks, Ricky Button -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Google App Engine OAuth endpoints throwing 400 in production.
Thanks for the reply Robert.
I ended up spending several hours last night reading through the OAuth specs
and SHA1 specs (in particular how to properly sign a request). I now have a
much better understanding of the OAuth dance and the Google docs seem much
more clear.
My development partner and I tried out Singpost (for Android), which seems
to work.
Just to be clear, Oauth2 isn't for OAuth2.0, but rather just indicates it's
a second version of oauth?
On Sat, May 14, 2011 at 12:32 AM, Robert Kluin robert.kl...@gmail.comwrote:
Hey Will,
Responses inline.
On Fri, May 13, 2011 at 16:46, Will Merydith will.meryd...@gmail.com
wrote:
OK I have some new information and questions to add to this discussion:
1 - There is some confusion in the documentation over whether
registration
is required or not.
Yup I agree. But it solved some issue for me in the past, so I
thought I'd suggest it.
Both here and here, it says registration is optional. Regardless, I have
registered my app.
2 - When going through the process of registering your application with
Google, you're informed that a certificate is not necessary on Google App
Engine, and that the OAuth signature method is HMAC-SHA1. Can anyone
confirm that as the case for using the OAuth for Python?
If I'm not mistaken, if you're using HMAC-SHA1 you don't upload a
cert. It will instead give you a secret key to use when signing your
requests.
3- Testing this locally doesn't provide much feedback because no matter
what
you pass, it works. On appspot, it's the opposite problem because
there's
no information on what failed. I've tried the OAuth Playground, which is
great, but wanted to verify that it supports testing OAuth for python. I
am
using it to test my Request for a Token and it fails, reporting,
signature_invalid.
The documentation on signature construction is confusing, in some cases
it
indicates sending a base string to the server that is the request url
encoded, and in other places it seems to be saying that you pass the
string
directly as in:
oauth_signature=wOJIO9A2W5mFwDgiDvZbTSMK%2FPY%3D
Can anyone clarify?
Yes, you're going to find many things about the gdata APIs suck very
bad and give effectively worthless error feedback. The docs are a
frightening mess too. Make sure you read the protocol guides so you
can make sure the requests look right.
I'm using python-oauth2. Here is a snippet of code I use, maybe it
will help get you started:
params = {
'oauth_version': 1.0,
'oauth_nonce': oauth.generate_nonce(),
'oauth_timestamp': int(time.time()),
'oauth_callback': 'https://%s%s' % (SERVER,
URL_MAP['FinalizeAuth']),
'oauth_consumer_key': CONSUMER_KEY,
'oauth_signature_method': 'HMAC-SHA1',
'scope': service_scopes
}
url = OAUTH_URLS['OAuthGetRequestToken']
consumer = oauth.Consumer(key=CONSUMER_KEY,
secret=CONSUMER_SECRET)
request = oauth.Request(method='POST', url=url, parameters=params)
request.sign_request(oauth.SignatureMethod_HMAC_SHA1(),
consumer, None)
params['oauth_signature_method'] = request['oauth_signature_method']
params['oauth_signature'] = request['oauth_signature']
result = urlfetch.fetch(url, payload=urllib.urlencode(params),
method='POST')
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Do HR-enabled apps get more than one memcached instance?
So, I was watching the presentation on the HR datastore from the IO 2011 ( http://www.google.com/events/io/2011/sessions/more-9s-please-under-the-covers-of-the-high-replication-datastore.html ) and one thing caught my attention: the slides were showing the frontend instances running in more than one datacenter at the same time. So I understand that the memcached can lose data at any time and if the app is migrated between the datacenters all of the data in the cache are lost, however, running the app in two or more datacenters at a time each DC having a separate memcached cluster, that changes the properties quite significantly. For example let's consider an app that uses some memcached key to keep a cursor where to write to the datastore, atomically incrementing it after each write. Such an app can detect if such a key is not present in the cache and determine what the cursor is by doing a query on the database and then add it to the cache atomically. If we add the possibility that there's another DC running the app w/ an independent memcached instance, such an app would just corrupt its own data. I hope I explained my concern well enough, and I would love to hear an answer from someone on the App Engine team. Thank you. -Sergey -- http://self.maluke.com/ -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] What's going on with my CPU times
Can anyone help me understand these CPU times. I have an on-line handler function that basically does a get_by_id(), and sends out the bytes for a 150x100 pixel jpeg BLOB record value. A pretty straightforward image retrieval from the datastore. My CPU loads are all over the map. I am $9 paying for 3 instances to stay resident. An instance takes just less than 15 meg of RAM. Very low volumes right now, so maybe that affects this? Looking at AppStats, the function itself is consistently taking 200ms or less for every call. So how am I getting 13,000ms for this simple function? I've copied/pasted a series of calls for this image below. Here is an extract of the last 4 url calls: 242ms...80cpu 351ms...13,740cpu 466ms4,903cpu 182ms2,922cpu Looking at the full list below, I only occasionally get fast cpu times. This is the simplest function, can't imagine where the inefficiencies are. Is the HR datastore that slow at pushing a small payload of bytes out the door? Man oh man, averaging these cpu times for a 150x100 pixel jpg, I better switch my image saves/retrieves to S3 mucho soon. Thanks in advance, stevep 2011-05-14 07:20:10.949 /cardimage/59005_t.jpg 200 242ms 80cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-14 07:03:04.967 /cardimage/59005_t.jpg 200 351ms 13740cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 14:50:48.065 /cardimage/59005_t.jpg 200 466ms 4903cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 13:57:09.990 /cardimage/59005_t.jpg 200 182ms 2922cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 10:39:27.922 /cardimage/59005_t.jpg 200 196ms 1474cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 09:01:08.347 /cardimage/59005_t.jpg 200 494ms 777cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 08:13:33.672 /cardimage/59005_t.jpg 200 501ms 435cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 07:23:52.577 /cardimage/59005_t.jpg 200 371ms 117cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 07:06:09.403 /cardimage/59005_t.jpg 200 880ms 1585cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 05:39:36.415 /cardimage/59005_t.jpg 200 728ms 6460cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 03:17:36.456 /cardimage/59005_t.jpg 200 413ms 1252cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 00:55:52.879 /cardimage/59005_t.jpg 200 683ms 191cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-12 22:42:57.523 /cardimage/59005_t.jpg 200 342ms 796cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-12 22:06:37.960 /cardimage/59005_t.jpg 200 77ms 66cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-12 22:03:02.608 /cardimage/59005_t.jpg 200 301ms 99cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-12 22:00:49.586 /cardimage/59005_t.jpg 200 393ms 118cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-12 21:54:47.092 /cardimage/59005_t.jpg 200 220ms 45cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-12 21:54:45.633 /cardimage/59005_t.jpg 200 90ms 46cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Understanding Security
If I understand what you are saying, I could make every URL map to a servlet, and let the servlet return the specific static pages. Yes. I generate my static content from simple templates, store them in the Datastore + Memcache, and serve them through a Servlet. There's a blog post by Nick Johnson on this technique for Python (though i can find it now). Or can I just put my folder of HTML into the resources section, and not have to write another servlet? No. You still need a servlet or filter to serve the file out of the resources Cheers, Eric -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Understanding Security
Actually I was wrong I on the last point. If you use appengine_web.xml [1] to configure your static files as excluded then appengine won't move them to the static server. Therefore, you can use a statandard filtering mechanism for those files. [1] http://code.google.com/appengine/docs/java/config/appconfig.html#Including_and_Excluding_Files -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Tasks are not executing!
I've been noticing this as well. Behavior is very similar to what I saw and reported on January 5, and which was subsequently resolved. http://groups.google.com/group/google-appengine/browse_thread/thread/7628e7474f792170/6ca1eb5f4058a9e8?lnk=gst On May 6, 8:51 pm, Nischal Shetty nischalshett...@gmail.com wrote: Yeah, tasks have been behaving strange from the last couple of days. I hope someone looks into this. On 6 May 2011 19:23, Steven Cains ste...@cains.me wrote: Tasks have been slow here. Our default queue is supposed to run @ 60/ m, it's slowed to 6/m for no reason. On May 5, 8:44 pm, David Mora dla.m...@gmail.com wrote: We are on master/slave: 18k per minutes as we speak On 5 May 2011 13:37, Stephen Johnson onepagewo...@gmail.com wrote: Sorry but that's what I'm seeing. Anyone else? On Thu, May 5, 2011 at 12:35 PM, Nischal Shetty nischalshett...@gmail.com wrote: I'm on Master/Slave. Don't tell me Task Queue's would also behave different ways for different data stores :( On 6 May 2011 01:01, Stephen Johnson onepagewo...@gmail.com wrote: Just to add some more insight hopefully. I'm still developing my application and have moved to HR datastore but kept my Master/Slave test running. Past couple of days Master/Slave has exhausted Task Queue of 100,000 but HR is working fine. Typical Task Queue would probably be 5000 for the day. Already Master/Slave has exhausted quota. For those having task queue problems are you on HR or Master/Slave? On Thu, May 5, 2011 at 12:21 PM, David Mora dla.m...@gmail.com wrote: hey guys, We are not experiencing this problem and we execute like 15k+ task per minute probably. One tip tho, might worth to mention: The ones staying in the queue have 0 retry attempts? Retries uses a push back mechanism so if they fail they are kept on the queue if your configuration is not suitable to handle this case On 5 May 2011 13:12, nischalshetty nischalshett...@gmail.com wrote: Tasks are not executing again! Close to 30K pending tasks in my queue, help! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- http://about.me/david.mora -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- -Nischal twitter: NischalShetty http://twitter.com/nischalshetty facebook: Nischal http://facebook.com/nischal -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. --http://about.me/david.mora -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more
[google-appengine] Re: Do HR-enabled apps get more than one memcached instance?
I thing HR-applications have more than one syncronised memcache - by me test save in HR-memcache in 3 times slower, than Master/Slave application. On May 14, 6:56 pm, Sergey Schetinin ser...@maluke.com wrote: So, I was watching the presentation on the HR datastore from the IO 2011 (http://www.google.com/events/io/2011/sessions/more-9s-please-under-th... ) and one thing caught my attention: the slides were showing the frontend instances running in more than one datacenter at the same time. So I understand that the memcached can lose data at any time and if the app is migrated between the datacenters all of the data in the cache are lost, however, running the app in two or more datacenters at a time each DC having a separate memcached cluster, that changes the properties quite significantly. For example let's consider an app that uses some memcached key to keep a cursor where to write to the datastore, atomically incrementing it after each write. Such an app can detect if such a key is not present in the cache and determine what the cursor is by doing a query on the database and then add it to the cache atomically. If we add the possibility that there's another DC running the app w/ an independent memcached instance, such an app would just corrupt its own data. I hope I explained my concern well enough, and I would love to hear an answer from someone on the App Engine team. Thank you. -Sergey --http://self.maluke.com/ -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Do HR-enabled apps get more than one memcached instance?
I'm pretty sure the memcached clusters (if there are more than one) are not synchronized. First of all, that would be way too slow. Second, the talk I referenced specifically mentions that when the apps are being migrated from a DC, the memcache writes return success but are in fact noop, because synchronizing memcache data does not make sense. So I would expect that there's no synchronization going on during regular operation as well. Anyway, I really hope that there's only one memcached cluster active at a time. On 14 May 2011 22:05, rekby timofey.koo...@gmail.com wrote: I thing HR-applications have more than one syncronised memcache - by me test save in HR-memcache in 3 times slower, than Master/Slave application. On May 14, 6:56 pm, Sergey Schetinin ser...@maluke.com wrote: So, I was watching the presentation on the HR datastore from the IO 2011 (http://www.google.com/events/io/2011/sessions/more-9s-please-under-th... ) and one thing caught my attention: the slides were showing the frontend instances running in more than one datacenter at the same time. So I understand that the memcached can lose data at any time and if the app is migrated between the datacenters all of the data in the cache are lost, however, running the app in two or more datacenters at a time each DC having a separate memcached cluster, that changes the properties quite significantly. For example let's consider an app that uses some memcached key to keep a cursor where to write to the datastore, atomically incrementing it after each write. Such an app can detect if such a key is not present in the cache and determine what the cursor is by doing a query on the database and then add it to the cache atomically. If we add the possibility that there's another DC running the app w/ an independent memcached instance, such an app would just corrupt its own data. I hope I explained my concern well enough, and I would love to hear an answer from someone on the App Engine team. Thank you. -Sergey --http://self.maluke.com/ -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- http://self.maluke.com/ -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] What's going on with my CPU times
Put it not in a blog. Consider using Static (depending what else you are doing) With out code it is hard to tell what you are doing but you may be using API cpu and not CPU Cpu Most of my CPU bill is API. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of stevep Sent: Saturday, May 14, 2011 8:04 AM To: Google App Engine Subject: [google-appengine] What's going on with my CPU times Can anyone help me understand these CPU times. I have an on-line handler function that basically does a get_by_id(), and sends out the bytes for a 150x100 pixel jpeg BLOB record value. A pretty straightforward image retrieval from the datastore. My CPU loads are all over the map. I am $9 paying for 3 instances to stay resident. An instance takes just less than 15 meg of RAM. Very low volumes right now, so maybe that affects this? Looking at AppStats, the function itself is consistently taking 200ms or less for every call. So how am I getting 13,000ms for this simple function? I've copied/pasted a series of calls for this image below. Here is an extract of the last 4 url calls: 242ms...80cpu 351ms...13,740cpu 466ms4,903cpu 182ms2,922cpu Looking at the full list below, I only occasionally get fast cpu times. This is the simplest function, can't imagine where the inefficiencies are. Is the HR datastore that slow at pushing a small payload of bytes out the door? Man oh man, averaging these cpu times for a 150x100 pixel jpg, I better switch my image saves/retrieves to S3 mucho soon. Thanks in advance, stevep 2011-05-14 07:20:10.949 /cardimage/59005_t.jpg 200 242ms 80cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-14 07:03:04.967 /cardimage/59005_t.jpg 200 351ms 13740cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 14:50:48.065 /cardimage/59005_t.jpg 200 466ms 4903cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 13:57:09.990 /cardimage/59005_t.jpg 200 182ms 2922cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 10:39:27.922 /cardimage/59005_t.jpg 200 196ms 1474cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 09:01:08.347 /cardimage/59005_t.jpg 200 494ms 777cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 08:13:33.672 /cardimage/59005_t.jpg 200 501ms 435cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 07:23:52.577 /cardimage/59005_t.jpg 200 371ms 117cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 07:06:09.403 /cardimage/59005_t.jpg 200 880ms 1585cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 05:39:36.415 /cardimage/59005_t.jpg 200 728ms 6460cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 03:17:36.456 /cardimage/59005_t.jpg 200 413ms 1252cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 00:55:52.879 /cardimage/59005_t.jpg 200 683ms 191cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-12 22:42:57.523 /cardimage/59005_t.jpg 200 342ms 796cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-12 22:06:37.960 /cardimage/59005_t.jpg 200 77ms 66cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-12 22:03:02.608 /cardimage/59005_t.jpg 200 301ms 99cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-12 22:00:49.586 /cardimage/59005_t.jpg 200 393ms 118cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-12 21:54:47.092 /cardimage/59005_t.jpg 200 220ms 45cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-12 21:54:45.633 /cardimage/59005_t.jpg 200 90ms 46cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at
Re: [google-appengine] Re: 1.5 improvements Make me less scared of Pricing
If the data doesn't get updated, using the blobstore might be a good idea. As I recall, you can't update a blob. So if the data changes it may not be the best idea. Of course, it really all depends on the app / usage. I store serialized dicts / lists in text / blob properties quite often. It works well for me. Robert On Sat, May 14, 2011 at 04:41, Tim meer...@gmail.com wrote: I suspected that might be the case, but if I have a few minutes to spare sometime I might try it out. My startup and servicing costs is pretty minimal - I've got a one-page-webapp and most of my calls are just AJAX calls to load and update data, so it's typically just object to JSON and back again. So given the way the API calls are counted and how my traffic works, I might look at getting rid of using the datastore for fields and just store what was a collection of items as a single larger JSON text blob (and look at moving to the blobstore), and tune the caching in the client in terms of writing back changes. I suppose a dynamic API for the scheduler might make it a bit tricky for GAE to plan (at the small scale) how to schedule work on machines, but if it was more like a declaration of hints at startup about how long I'd prefer to hang around when idle before being killed etc - even a declaration in app.yaml - then I think it might go some way to allaying some of the concerns being voiced. -- T -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Possible to Defer Tasks to Backend
Hi, The way my application is set up involves deferring a lot of long- running tasks, which I now would like to move to a backend. Is it possible to defer a task so that it is processed in a backend? Or, do I have to create handlers for every deferred task in order to take advantage of this feature? Best, Aaron P.S. Thanks so much for the hard work! The new features are very exciting :). -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Do HR-enabled apps get more than one memcached instance?
Your app actually only runs out of one data center at once. The diagram was a little misleading. If a data center goes down then your app is served out of the other with a different memcache (empty) On May 14, 3:26 pm, Sergey Schetinin ser...@maluke.com wrote: I'm pretty sure the memcached clusters (if there are more than one) are not synchronized. First of all, that would be way too slow. Second, the talk I referenced specifically mentions that when the apps are being migrated from a DC, the memcache writes return success but are in fact noop, because synchronizing memcache data does not make sense. So I would expect that there's no synchronization going on during regular operation as well. Anyway, I really hope that there's only one memcached cluster active at a time. On 14 May 2011 22:05, rekby timofey.koo...@gmail.com wrote: I thing HR-applications have more than one syncronised memcache - by me test save in HR-memcache in 3 times slower, than Master/Slave application. On May 14, 6:56 pm, Sergey Schetinin ser...@maluke.com wrote: So, I was watching the presentation on the HR datastore from the IO 2011 (http://www.google.com/events/io/2011/sessions/more-9s-please-under-th... ) and one thing caught my attention: the slides were showing the frontend instances running in more than one datacenter at the same time. So I understand that the memcached can lose data at any time and if the app is migrated between the datacenters all of the data in the cache are lost, however, running the app in two or more datacenters at a time each DC having a separate memcached cluster, that changes the properties quite significantly. For example let's consider an app that uses some memcached key to keep a cursor where to write to the datastore, atomically incrementing it after each write. Such an app can detect if such a key is not present in the cache and determine what the cursor is by doing a query on the database and then add it to the cache atomically. If we add the possibility that there's another DC running the app w/ an independent memcached instance, such an app would just corrupt its own data. I hope I explained my concern well enough, and I would love to hear an answer from someone on the App Engine team. Thank you. -Sergey --http://self.maluke.com/ -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. --http://self.maluke.com/ -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Do HR-enabled apps get more than one memcached instance?
Thanks for the clarification. On 15 May 2011 00:47, Mike Wesner mike.wes...@webfilings.com wrote: Your app actually only runs out of one data center at once. The diagram was a little misleading. If a data center goes down then your app is served out of the other with a different memcache (empty) On May 14, 3:26 pm, Sergey Schetinin ser...@maluke.com wrote: I'm pretty sure the memcached clusters (if there are more than one) are not synchronized. First of all, that would be way too slow. Second, the talk I referenced specifically mentions that when the apps are being migrated from a DC, the memcache writes return success but are in fact noop, because synchronizing memcache data does not make sense. So I would expect that there's no synchronization going on during regular operation as well. Anyway, I really hope that there's only one memcached cluster active at a time. On 14 May 2011 22:05, rekby timofey.koo...@gmail.com wrote: I thing HR-applications have more than one syncronised memcache - by me test save in HR-memcache in 3 times slower, than Master/Slave application. On May 14, 6:56 pm, Sergey Schetinin ser...@maluke.com wrote: So, I was watching the presentation on the HR datastore from the IO 2011 (http://www.google.com/events/io/2011/sessions/more-9s-please-under-th... ) and one thing caught my attention: the slides were showing the frontend instances running in more than one datacenter at the same time. So I understand that the memcached can lose data at any time and if the app is migrated between the datacenters all of the data in the cache are lost, however, running the app in two or more datacenters at a time each DC having a separate memcached cluster, that changes the properties quite significantly. For example let's consider an app that uses some memcached key to keep a cursor where to write to the datastore, atomically incrementing it after each write. Such an app can detect if such a key is not present in the cache and determine what the cursor is by doing a query on the database and then add it to the cache atomically. If we add the possibility that there's another DC running the app w/ an independent memcached instance, such an app would just corrupt its own data. I hope I explained my concern well enough, and I would love to hear an answer from someone on the App Engine team. Thank you. -Sergey --http://self.maluke.com/ -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. --http://self.maluke.com/ -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- http://self.maluke.com/ -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: What's going on with my CPU times
Thanks Brandon. Every image is unique, and is generated by users from the client. So statics limitations make it infeasible. Right now I think this has to do with the on/off again hits. I went into my Dashboard after a few individual calls, and saw I had gone from 3 resident instances: 3 resident and 3 dynamic. Although this was serving a query rate of 0.018 per second. My average instance latency was 65ms I think. Have no idea what the resident instances are doing. But guess will wait and see. Cheers, stevep On May 14, 12:27 pm, Brandon Wirtz drak...@digerat.com wrote: Put it not in a blog. Consider using Static (depending what else you are doing) With out code it is hard to tell what you are doing but you may be using API cpu and not CPU Cpu Most of my CPU bill is API. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of stevep Sent: Saturday, May 14, 2011 8:04 AM To: Google App Engine Subject: [google-appengine] What's going on with my CPU times Can anyone help me understand these CPU times. I have an on-line handler function that basically does a get_by_id(), and sends out the bytes for a 150x100 pixel jpeg BLOB record value. A pretty straightforward image retrieval from the datastore. My CPU loads are all over the map. I am $9 paying for 3 instances to stay resident. An instance takes just less than 15 meg of RAM. Very low volumes right now, so maybe that affects this? Looking at AppStats, the function itself is consistently taking 200ms or less for every call. So how am I getting 13,000ms for this simple function? I've copied/pasted a series of calls for this image below. Here is an extract of the last 4 url calls: 242ms...80cpu 351ms...13,740cpu 466ms4,903cpu 182ms2,922cpu Looking at the full list below, I only occasionally get fast cpu times. This is the simplest function, can't imagine where the inefficiencies are. Is the HR datastore that slow at pushing a small payload of bytes out the door? Man oh man, averaging these cpu times for a 150x100 pixel jpg, I better switch my image saves/retrieves to S3 mucho soon. Thanks in advance, stevep 2011-05-14 07:20:10.949 /cardimage/59005_t.jpg 200 242ms 80cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-14 07:03:04.967 /cardimage/59005_t.jpg 200 351ms 13740cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 14:50:48.065 /cardimage/59005_t.jpg 200 466ms 4903cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 13:57:09.990 /cardimage/59005_t.jpg 200 182ms 2922cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 10:39:27.922 /cardimage/59005_t.jpg 200 196ms 1474cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 09:01:08.347 /cardimage/59005_t.jpg 200 494ms 777cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 08:13:33.672 /cardimage/59005_t.jpg 200 501ms 435cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 07:23:52.577 /cardimage/59005_t.jpg 200 371ms 117cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 07:06:09.403 /cardimage/59005_t.jpg 200 880ms 1585cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 05:39:36.415 /cardimage/59005_t.jpg 200 728ms 6460cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 03:17:36.456 /cardimage/59005_t.jpg 200 413ms 1252cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 00:55:52.879 /cardimage/59005_t.jpg 200 683ms 191cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-12 22:42:57.523 /cardimage/59005_t.jpg 200 342ms 796cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-12 22:06:37.960 /cardimage/59005_t.jpg 200 77ms 66cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-12 22:03:02.608 /cardimage/59005_t.jpg 200 301ms 99cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-12 22:00:49.586 /cardimage/59005_t.jpg 200 393ms 118cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-12
RE: [google-appengine] Re: What's going on with my CPU times
Well Duh, You are generating a JPG, you said send the bytes but you are modifying, rendering, and compressing a JPG. That takes some CPU. My app sends jpgs and uses no CPU as it were because it really is do I have this? Oh I do? Great here it is Your app is Hey Draw me a pretty picture Um... Ok let me do that for youYou probably want that as a JPG not a BMP let me compress that Let me save it to a blob incase you need it later... here you go It's the difference between having an intern running a photo copy and asking Me to paint you a picture of you with my cat. And now I'm certain you are using APIs which have a minimum CPU per action cost. Likely if these are one time use images, you should just write them, no blob. These are small, so likely if you need to keep them around you should write them to data store If you only kind of need them sometimes, consider memcache so you don't have to clean up in a week when you have lots of these. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of stevep Sent: Saturday, May 14, 2011 3:06 PM To: Google App Engine Subject: [google-appengine] Re: What's going on with my CPU times Thanks Brandon. Every image is unique, and is generated by users from the client. So statics limitations make it infeasible. Right now I think this has to do with the on/off again hits. I went into my Dashboard after a few individual calls, and saw I had gone from 3 resident instances: 3 resident and 3 dynamic. Although this was serving a query rate of 0.018 per second. My average instance latency was 65ms I think. Have no idea what the resident instances are doing. But guess will wait and see. Cheers, stevep On May 14, 12:27 pm, Brandon Wirtz drak...@digerat.com wrote: Put it not in a blog. Consider using Static (depending what else you are doing) With out code it is hard to tell what you are doing but you may be using API cpu and not CPU Cpu Most of my CPU bill is API. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of stevep Sent: Saturday, May 14, 2011 8:04 AM To: Google App Engine Subject: [google-appengine] What's going on with my CPU times Can anyone help me understand these CPU times. I have an on-line handler function that basically does a get_by_id(), and sends out the bytes for a 150x100 pixel jpeg BLOB record value. A pretty straightforward image retrieval from the datastore. My CPU loads are all over the map. I am $9 paying for 3 instances to stay resident. An instance takes just less than 15 meg of RAM. Very low volumes right now, so maybe that affects this? Looking at AppStats, the function itself is consistently taking 200ms or less for every call. So how am I getting 13,000ms for this simple function? I've copied/pasted a series of calls for this image below. Here is an extract of the last 4 url calls: 242ms...80cpu 351ms...13,740cpu 466ms4,903cpu 182ms2,922cpu Looking at the full list below, I only occasionally get fast cpu times. This is the simplest function, can't imagine where the inefficiencies are. Is the HR datastore that slow at pushing a small payload of bytes out the door? Man oh man, averaging these cpu times for a 150x100 pixel jpg, I better switch my image saves/retrieves to S3 mucho soon. Thanks in advance, stevep 2011-05-14 07:20:10.949 /cardimage/59005_t.jpg 200 242ms 80cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-14 07:03:04.967 /cardimage/59005_t.jpg 200 351ms 13740cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 14:50:48.065 /cardimage/59005_t.jpg 200 466ms 4903cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 13:57:09.990 /cardimage/59005_t.jpg 200 182ms 2922cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 10:39:27.922 /cardimage/59005_t.jpg 200 196ms 1474cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 09:01:08.347 /cardimage/59005_t.jpg 200 494ms 777cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 08:13:33.672 /cardimage/59005_t.jpg 200 501ms 435cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 07:23:52.577 /cardimage/59005_t.jpg 200 371ms 117cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http:// developers.facebook.com),gzip(gfe),gzip(gfe),gzip(gfe) 2011-05-13 07:06:09.403 /cardimage/59005_t.jpg 200 880ms 1585cpu_ms 8api_cpu_ms 35kb facebookplatform/1.0 (+http://
[google-appengine] Re: Possible to Defer Tasks to Backend
A follow up to this question is if we can use the new pipeline API with backends. If we start a pipeline in a backend, will all future pipeline tasks be processed by the backend? On May 14, 2:41 pm, Aaron aaron.t.che...@gmail.com wrote: Hi, The way my application is set up involves deferring a lot of long- running tasks, which I now would like to move to a backend. Is it possible to defer a task so that it is processed in a backend? Or, do I have to create handlers for every deferred task in order to take advantage of this feature? Best, Aaron P.S. Thanks so much for the hard work! The new features are very exciting :). -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Caching GeoModel requests?
Hi all, Our project http://safesharps.org.au uses GeoModel (http://code.google.com/p/geomodel/). To boost performance we have been caching many of our query results using memcache, this is generally done by keeping an index of the search terms. Our app has Web / Mobile Web and upcoming mobile applications for iOS and Android. Geo is a little tricky to cache, since using the Lat/Lng is not exactly efficient. For example, a search query by people within 10 meters of each other would result in the same results on our database but would have different Geo locations. Any ideas on caching / performance enhancement when using Geo queries? Thanks. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Caching GeoModel requests?
Round. http://maps.google.com/?ie=UTF8ll=47.595,-122.36spn=.873914,.712402t=hz= 15 3 decimals of accuracy puts the boat near the center, but would you change results if you Were closer to the peer? http://maps.google.com/?ie=UTF8ll=47.589,-122.37spn=.873914,.712402t=hz= 15 Essentially, 47.6 , -122.35 is the same as any of the above values. Pick the level of accuracy you need and round to that. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Devraj Mukherjee Sent: Saturday, May 14, 2011 5:10 PM To: google-appengine@googlegroups.com Subject: [google-appengine] Caching GeoModel requests? Hi all, Our project http://safesharps.org.au uses GeoModel (http://code.google.com/p/geomodel/). To boost performance we have been caching many of our query results using memcache, this is generally done by keeping an index of the search terms. Our app has Web / Mobile Web and upcoming mobile applications for iOS and Android. Geo is a little tricky to cache, since using the Lat/Lng is not exactly efficient. For example, a search query by people within 10 meters of each other would result in the same results on our database but would have different Geo locations. Any ideas on caching / performance enhancement when using Geo queries? Thanks. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Caching GeoModel requests?
you can shard by the biggest geocell and cache based on the smallest geocell. at then end GeoModel creates a hash based on geoboxes that you can easily play with On 14 May 2011 18:10, Devraj Mukherjee dev...@gmail.com wrote: Hi all, Our project http://safesharps.org.au uses GeoModel (http://code.google.com/p/geomodel/). To boost performance we have been caching many of our query results using memcache, this is generally done by keeping an index of the search terms. Our app has Web / Mobile Web and upcoming mobile applications for iOS and Android. Geo is a little tricky to cache, since using the Lat/Lng is not exactly efficient. For example, a search query by people within 10 meters of each other would result in the same results on our database but would have different Geo locations. Any ideas on caching / performance enhancement when using Geo queries? Thanks. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- http://about.me/david.mora -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Caching GeoModel requests?
Appreciate that Brandon. On Sun, May 15, 2011 at 10:24 AM, Brandon Wirtz drak...@digerat.com wrote: Round. http://maps.google.com/?ie=UTF8ll=47.595,-122.36spn=.873914,.712402t=hz= 15 3 decimals of accuracy puts the boat near the center, but would you change results if you Were closer to the peer? http://maps.google.com/?ie=UTF8ll=47.589,-122.37spn=.873914,.712402t=hz= 15 Essentially, 47.6 , -122.35 is the same as any of the above values. Pick the level of accuracy you need and round to that. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Possible to Defer Tasks to Backend
You should be able to send a deferred task to a backend. Some points to remember though: * You should ensure that any classes/etc you reference from the frontend must exist in the backend. * The deferred function should pass the 'target' parameter on to taskqueue * You must enable the deferred handler on the backend (just make sure it is enabled in app.yaml when you upload the backend). On 14 May 2011 14:41, Aaron aaron.t.che...@gmail.com wrote: Hi, The way my application is set up involves deferring a lot of long- running tasks, which I now would like to move to a backend. Is it possible to defer a task so that it is processed in a backend? Or, do I have to create handlers for every deferred task in order to take advantage of this feature? Best, Aaron P.S. Thanks so much for the hard work! The new features are very exciting :). -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Possible to Defer Tasks to Backend
That is correct, all tasks get the same host header as the request that created them. This means that if you access the url http://0.backend-name.app-id.appspot.com/ and it inserts a task, the task will execute against instance 0 of 'backend-name'. If you access the url http://backend-name.app-id.appspot.com/ and it inserts a task, the task will execute against any instance of 'backend-name'. On 14 May 2011 15:31, Aaron aaron.t.che...@gmail.com wrote: A follow up to this question is if we can use the new pipeline API with backends. If we start a pipeline in a backend, will all future pipeline tasks be processed by the backend? On May 14, 2:41 pm, Aaron aaron.t.che...@gmail.com wrote: Hi, The way my application is set up involves deferring a lot of long- running tasks, which I now would like to move to a backend. Is it possible to defer a task so that it is processed in a backend? Or, do I have to create handlers for every deferred task in order to take advantage of this feature? Best, Aaron P.S. Thanks so much for the hard work! The new features are very exciting :). -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] HR applications and Cache Control/Pragma/Expires headers
Can I bribe somebody to do my apps? (I even moved to 1.5 hoping that'd auto fix them) My apps all return this: HTTP/1.1.200.OK(CR)(LF) Cache-Control:.no-cache,.must-revalidate(CR)(LF) Content-Type:.text/html;.charset=UTF-8(CR)(LF) Set-Cookie:.S=apphosting=JPio4yliHelJf10OZQQWDw;.path=/(CR)(LF) Date:.Sun,.15.May.2011.00:43:02.GMT(CR)(LF) Pragma:.no-cache(CR)(LF) Expires:.Fri,.01.Jan.1990.00:00:00.GMT(CR)(LF) Server:.Google.Frontend(CR)(LF) And I noticed that SOME Company I won't name, names.. Every Time they Run an Ad on one of my pages they seem to think they need to send the AD bot, and the Index Bot (twice) to the page, and they always do it with in 3 seconds of the page being requested, so my bill could be like 1/3 what it is if this was fixed. Or maybe that's the plan.. Are you guys in the GAE team giving the Adsense and Search Teams a Kick back the more they hit my site? Not to mention the messages the Webmasters team constantly send saying you have really short expirations on your CSS and Images speed up your site by setting longer cache lengths for your style sheets, javascript, and images And those people at THAT OTHER search engine the one that kind of rhymes with what 95% of the people who try out for American Idol can't do. They just sent me a message that Setting No Cache in headers is interpreted in a way not dissimilar to 'no-index'. No-cache implies that the information is for onetime use and that the user or bot should not view this information as static. If information doesn't have at least some shelf life there is no reason to index it as it may not be there when future users arrive at the site. You have to give them credit on the amount of logic they put in to their failure to understand that maybe I just want the user to not cache the page so I can track them via the server rather than via javascript analytics. (I don't think they liked or believed my answer). From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Gregory D'alesandre Sent: Wednesday, May 11, 2011 11:10 AM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] HR applications and Cache Control/Pragma/Expires headers If this was an existing app prior to 1.5.0 then it is not fixed yet, 1.5.0 put the fix in but we still need to clean up apps that were created prior to 1.5.0. Hope that helps, Greg On Wed, May 11, 2011 at 10:54 AM, Stephen Johnson onepagewo...@gmail.com wrote: I'm confused. I have an HR app and this is what I see when I look at the headers using Firebug for one of my images. You're saying this should be different? On Wed, May 11, 2011 at 9:36 AM, Francois Masurel f.masu...@gmail.com wrote: What is rather strange is that cache headers work perfectly fine with MS applications. I hope it's not a way to increase billing ;-) I'm personaly concerned as I'm serving most of my image and file contents dynamically from datastore or memcache. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com mailto:google-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com mailto:google-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Understanding Security
Woo-Hoo! Thanks, I can't wait to try that, but I'm don't working for the day. Cheers, Eric On 2011-05-14 10:15 AM, Erick Fleming wrote: Actually I was wrong I on the last point. If you use appengine_web.xml [1] to configure your static files as excluded then appengine won't move them to the static server. Therefore, you can use a statandard filtering mechanism for those files. [1] http://code.google.com/appengine/docs/java/config/appconfig.html#Including_and_Excluding_Files -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Possible to Defer Tasks to Backend
Since backends and frontends are updated separately, the code in each may be different. Updating a frontend does not change the code that is running in a backend/vice versa. As for the syntax, I believe it should be: deferred.defer(deferred_fn_name, param1, param2, _target='backend-name') On 14 May 2011 18:22, Aaron aaron.t.che...@gmail.com wrote: Hi Greg, Thanks for the response. Just to be clear, the syntax to defer a task to a backend is the following? deferred.defer(deferred_fn_name, param1, param2, _target='http:// backend-name.app-id.appspot.com') Also, you mentioned that we should be sure that the backend has the same classes and configurations as the frontend? As far as I have read, the frontend and backend are using the same code base, so is this implied already, or is there something extra I am missing? Best, Aaron On May 14, 5:56 pm, Greg Darke (Google) darke+goo...@google.com wrote: You should be able to send a deferred task to a backend. Some points to remember though: * You should ensure that any classes/etc you reference from the frontend must exist in the backend. * The deferred function should pass the 'target' parameter on to taskqueue * You must enable the deferred handler on the backend (just make sure it is enabled in app.yaml when you upload the backend). On 14 May 2011 14:41, Aaron aaron.t.che...@gmail.com wrote: Hi, The way my application is set up involves deferring a lot of long- running tasks, which I now would like to move to a backend. Is it possible to defer a task so that it is processed in a backend? Or, do I have to create handlers for every deferred task in order to take advantage of this feature? Best, Aaron P.S. Thanks so much for the hard work! The new features are very exciting :). -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Numbers that don't quite add Up?
This isn't a big deal, but the numbers don't sum what I'd have expected. Both of these are about 2.5x in billing what I think I am doing in storage. Where does the Discrepancy come from? From the Dashboard:290 MB on one app 1.56GB on the other High Replication Data $0.008/ GByte-day Description: 58% 58% 0.29 of 0.50 GBytes $0.00 / $0.00 High Replication Data $0.008/ GByte-day Description: 3% 3% 1.56 of 50.50 GBytes $0.01 / $0.40 From the DataStore Admin: 115MB and 600MB Entity Kind # Entities Avg. Size/Entity Total Size Kind1 149,947 342 Bytes 49 MBytes Kind2 11,885 6 KBytes 66 MBytes Config 1 244 Bytes 244 Bytes Kind1 633,362 317 Bytes 192 MBytes Kind2 84,211 5 KBytes 408 MBytes Config 1 257 Bytes 257 Bytes -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. image003.pngimage004.png
Re: [google-appengine] Numbers that don't quite add Up?
Well, the Datastore Statistics don't add in any indexing, etc. whereas the quota details does add that in. On Sat, May 14, 2011 at 9:37 PM, Brandon Wirtz drak...@digerat.com wrote: This isn’t a big deal, but the numbers don’t sum what I’d have expected… Both of these are about 2.5x in billing what I think I am doing in storage. Where does the Discrepancy come from? From the Dashboard:290 MB on one app 1.56GB on the other High Replication Data $0.008/ GByte-day [image: Description: 58%] 58% 0.29 of 0.50 GBytes $0.00 / $0.00 High Replication Data $0.008/ GByte-day [image: Description: 3%] 3% 1.56 of 50.50 GBytes $0.01 / $0.40 From the DataStore Admin: 115MB and 600MB ** *Entity Kind* *# Entities* *Avg. Size/Entity* *Total Size* Kind1 149,947 342 Bytes 49 MBytes Kind2 11,885 6 KBytes 66 MBytes Config 1 244 Bytes 244 Bytes Kind1 633,362 317 Bytes 192 MBytes Kind2 84,211 5 KBytes 408 MBytes Config 1 257 Bytes 257 Bytes -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. image003.pngimage004.png
