[appengine-java] embedding parent object in parent-child relation with jdo
I am using the app engine SDK 1.3.0 with the latest Google Eclipse
plugin.
Currently, in order to be able to run queries on attributes of the
parent object I have to embed parent object into a child. So, I have
created the following classes
@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class Parent {
@PrimaryKey @Persistent String parentName;
@Persistent(mappedBy = "parent") List children = new
ArrayList();
public Parent(String parentName) { this.parentName = parentName; }
public void setParentName(String parentName) { this.parentName =
parentName; }
public String getParentName() { return parentName; }
public List getChildren() { return children; }
public void addChild(Child child) {
child.setParent(this);
children.add(child);
}
}
@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class Child {
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) Key
childKey;
@Persistent String childName;
@Embedded(
members = { @Persistent(name = "parentName", column =
"childParentName") },
ownerMember = "children")
@Persistent Parent parent;
public Child(String childName) { this.childName = childName; }
public void setChildName(String childName) { this.childName =
childName; }
public String getChildName() { return childName; }
public void setParent(Parent parent) { this.parent = parent; }
public Parent getParent() { return parent; }
}
and then I created a simple test to verify if I can persist and run
queries on those objects:
public class ParentTest {
public static void main(String[] args) {
Parent parent1 = new Parent("parent1");
parent1.addChild(new Child("child1"));
parent1.addChild(new Child("child2"));
Parent parent2 = new Parent("parent2");
parent2.addChild(new Child("child3"));
ApiProxy.setEnvironmentForCurrentThread(new TestEnvironment());
ApiProxy.setDelegate(new ApiProxyLocalImpl(new File(".")) { });
ApiProxyLocalImpl proxy = (ApiProxyLocalImpl) ApiProxy.getDelegate
();
proxy.setProperty(LocalDatastoreService.NO_STORAGE_PROPERTY,
Boolean.TRUE.toString());
PersistenceManagerFactory pmf =
JDOHelper.getPersistenceManagerFactory("transactions-optional");
PersistenceManager pm = pmf.getPersistenceManager();
Transaction tx = pm.currentTransaction();
try {
tx.begin();
pm.makePersistent(parent1);
tx.commit();
tx.begin();
pm.makePersistent(parent2);
tx.commit();
Query query2 = pm.newQuery(Child.class);
query2.setFilter("parent.parentName==parentNameParam");
query2.declareParameters("java.lang.String parentNameParam");
List results2 = (List) query2.executeWithArray
("parent1");
System.err.println(results2);
} finally {
if (tx.isActive()) {
tx.rollback();
}
pm.close();
}
}
}
The DataNucleus Enhancer runs ok on the above code, but when I run
my test app, I am getting an IndexOutOfBoundException from somewhere
in the DataNucleus internals:
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 0
at org.datanucleus.store.appengine.DatastoreTable.initializeNonPK
(DatastoreTable.java:356)
at org.datanucleus.store.appengine.DatastoreTable.buildMapping
(DatastoreTable.java:285)
at org.datanucleus.store.appengine.DatastoreManager.buildStoreData
(DatastoreManager.java:405)
at org.datanucleus.store.appengine.DatastoreManager.newStoreData
(DatastoreManager.java:363)
at org.datanucleus.store.AbstractStoreManager.addClasses
(AbstractStoreManager.java:788)
at org.datanucleus.store.AbstractStoreManager.addClass
(AbstractStoreManager.java:759)
at org.datanucleus.store.mapped.MappedStoreManager.getDatastoreClass
(MappedStoreManager.java:358)
at org.datanucleus.store.appengine.DatastoreManager.getDatastoreClass
(DatastoreManager.java:631)
at
org.datanucleus.store.appengine.DatastoreFieldManager.buildMappingConsumer
(DatastoreFieldManager.java:1008)
at
org.datanucleus.store.appengine.DatastoreFieldManager.buildMappingConsumer
(DatastoreFieldManager.java:998)
at org.datanucleus.store.appengine.DatastoreFieldManager.
(DatastoreFieldManager.java:133)
at org.datanucleus.store.appengine.DatastoreFieldManager.
(DatastoreFieldManager.java:167)
at
org.datanucleus.store.appengine.DatastorePersistenceHandler.insertPreProcess
(DatastorePersistenceHandler.java:316)
at
org.datanucleus.store.appengine.DatastorePersistenceHandler.insertObjects
(DatastorePersistenceHandler.java:236)
at
org.datanucleus.store.appengine.DatastorePersistenceHandler.insertObject
(DatastorePersistenceHandler.java:225)
at org.datanucleus.state.JDOStateManagerImpl.internalMakePersistent
(JDOStateManagerImpl.java:3185)
at org.datanucleus.state.JDOStateManagerImpl.makePersistent
(JDOStateManagerImpl.java:3161)
at org.datanucleus.ObjectManagerIm
[appengine-java] Re: embedding parent object in parent-child relation with jdo
It shouldn't be needed, since according to the docs [1], the Key
type is only required to record parent relation in the child.
But I tried it anyways and got pretty much the same exception:
Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 0
at org.datanucleus.store.appengine.DatastoreTable.initializeNonPK
(DatastoreTable.java:356)
at org.datanucleus.store.appengine.DatastoreTable.buildMapping
(DatastoreTable.java:285)
at org.datanucleus.store.appengine.DatastoreManager.buildStoreData
(DatastoreManager.java:405)
...
regards,
Eugene
[1]
http://code.google.com/appengine/docs/java/datastore/creatinggettinganddeletingdata.html#Keys
On Dec 18, 2:33 pm, Rusty Wright wrote:
> Try making the parent's primary key be Key instead of String.
>
> Eugene Kuleshov wrote:
> > I am using the app engine SDK 1.3.0 with the latest Google Eclipse
> > plugin.
>
> > Currently, in order to be able to run queries on attributes of the
> > parent object I have to embed parent object into a child. So, I have
> > created the following classes
>
> > @PersistenceCapable(identityType = IdentityType.APPLICATION)
> > public class Parent {
> > @PrimaryKey @Persistent String parentName;
> > @Persistent(mappedBy = "parent") List children = new
> > ArrayList();
>
> > public Parent(String parentName) { this.parentName = parentName; }
> > public void setParentName(String parentName) { this.parentName =
> > parentName; }
> > public String getParentName() { return parentName; }
> > public List getChildren() { return children; }
> > public void addChild(Child child) {
> > child.setParent(this);
> > children.add(child);
> > }
> > }
>
> > @PersistenceCapable(identityType = IdentityType.APPLICATION)
> > public class Child {
> > @PrimaryKey
> > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) Key
> > childKey;
> > @Persistent String childName;
>
> > @Embedded(
> > members = { @Persistent(name = "parentName", column =
> > "childParentName") },
> > ownerMember = "children")
> > @Persistent Parent parent;
>
> > public Child(String childName) { this.childName = childName; }
> > public void setChildName(String childName) { this.childName =
> > childName; }
> > public String getChildName() { return childName; }
> > public void setParent(Parent parent) { this.parent = parent; }
> > public Parent getParent() { return parent; }
> > }
> > and then I created a simple test to verify if I can persist and run
> > queries on those objects:
>
> > public class ParentTest {
> > public static void main(String[] args) {
> > Parent parent1 = new Parent("parent1");
> > parent1.addChild(new Child("child1"));
> > parent1.addChild(new Child("child2"));
>
> > Parent parent2 = new Parent("parent2");
> > parent2.addChild(new Child("child3"));
>
> > ApiProxy.setEnvironmentForCurrentThread(new TestEnvironment());
> > ApiProxy.setDelegate(new ApiProxyLocalImpl(new File(".")) { });
>
> > ApiProxyLocalImpl proxy = (ApiProxyLocalImpl) ApiProxy.getDelegate
> > ();
> > proxy.setProperty(LocalDatastoreService.NO_STORAGE_PROPERTY,
> > Boolean.TRUE.toString());
>
> > PersistenceManagerFactory pmf =
> > JDOHelper.getPersistenceManagerFactory("transactions-optional");
> > PersistenceManager pm = pmf.getPersistenceManager();
> > Transaction tx = pm.currentTransaction();
> > try {
> > tx.begin();
> > pm.makePersistent(parent1);
> > tx.commit();
>
> > tx.begin();
> > pm.makePersistent(parent2);
> > tx.commit();
>
> > Query query2 = pm.newQuery(Child.class);
> > query2.setFilter("parent.parentName==parentNameParam");
> > query2.declareParameters("java.lang.String parentNameParam");
> > List results2 = (List) query2.executeWithArray
> > ("parent1");
> > System.err.println(results2);
> > } finally {
> > if (tx.isActive()) {
> > tx.rollback();
> > }
> > pm.close();
> > }
> > }
> > }
>
> > The DataNucleus Enhancer runs ok on the above code, but when I run
> > my test app, I am getting an IndexOutOfBoundException from somewhere
> > in the DataNucleus internals:
>
> > Exception in thread "main" java.lang.Arra
[appengine-java] Re: embedding parent object in parent-child relation with jdo
On Dec 19, 2:56 am, jd wrote:
> Could you not split your query into two? The first to find the parent
> and the second to return all children with an ancestor query.
In my case the main filtering criteria is actually by one of the
child's attributes, but then I need to sort the results and present
them using some of the parent's attributes. For example, with Order
and OrderItem model, find all OrderItems that include the given part
number and then sort the result according to the Order's date. So, I
do need to preserve parent-child relationship, but don't want to
execute N+1 queries (one for each parent referenced from the found
children).
Also, I would like to maintain parent-child relationship, so entity
groups are handled automatically (so I won't have to manually create
foreign keys), i.e. I can process all parent-child chunks in one
transaction.
Thanks
Eugene
> On Dec 19, 3:58 am, Eugene Kuleshov wrote:
>
> > It shouldn't be needed, since according to the docs [1], the Key
> > type is only required to record parent relation in the child.
> > But I tried it anyways and got pretty much the same exception:
>
> > Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 0
> > at org.datanucleus.store.appengine.DatastoreTable.initializeNonPK
> > (DatastoreTable.java:356)
> > at org.datanucleus.store.appengine.DatastoreTable.buildMapping
> > (DatastoreTable.java:285)
> > at org.datanucleus.store.appengine.DatastoreManager.buildStoreData
> > (DatastoreManager.java:405)
> > ...
>
> > regards,
> > Eugene
>
> > [1]http://code.google.com/appengine/docs/java/datastore/creatinggettinga...
>
> > On Dec 18, 2:33 pm, Rusty Wright wrote:
>
> > > Try making the parent's primary key be Key instead of String.
>
> > > Eugene Kuleshov wrote:
> > > > I am using the app engine SDK 1.3.0 with the latest Google Eclipse
> > > > plugin.
>
> > > > Currently, in order to be able to run queries on attributes of the
> > > > parent object I have to embed parent object into a child. So, I have
> > > > created the following classes
>
> > > > @PersistenceCapable(identityType = IdentityType.APPLICATION)
> > > > public class Parent {
> > > > @PrimaryKey @Persistent String parentName;
> > > > @Persistent(mappedBy = "parent") List children = new
> > > > ArrayList();
>
> > > > public Parent(String parentName) { this.parentName = parentName; }
> > > > public void setParentName(String parentName) { this.parentName =
> > > > parentName; }
> > > > public String getParentName() { return parentName; }
> > > > public List getChildren() { return children; }
> > > > public void addChild(Child child) {
> > > > child.setParent(this);
> > > > children.add(child);
> > > > }
> > > > }
>
> > > > @PersistenceCapable(identityType = IdentityType.APPLICATION)
> > > > public class Child {
> > > > @PrimaryKey
> > > > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) Key
> > > > childKey;
> > > > @Persistent String childName;
>
> > > > @Embedded(
> > > > members = { @Persistent(name = "parentName", column =
> > > > "childParentName") },
> > > > ownerMember = "children")
> > > > @Persistent Parent parent;
>
> > > > public Child(String childName) { this.childName = childName; }
> > > > public void setChildName(String childName) { this.childName =
> > > > childName; }
> > > > public String getChildName() { return childName; }
> > > > public void setParent(Parent parent) { this.parent = parent; }
> > > > public Parent getParent() { return parent; }
> > > > }
> > > > and then I created a simple test to verify if I can persist and run
> > > > queries on those objects:
>
> > > > public class ParentTest {
> > > > public static void main(String[] args) {
> > > > Parent parent1 = new Parent("parent1");
> > > > parent1.addChild(new Child("child1"));
> > > > parent1.addChild(new Child("child2"));
>
> > > > Parent parent2 = new Parent("parent2");
> > > > parent2.addChild(new Child("child3"));
>
> > > > ApiProxy.setEnvironmentForCurrentThread(new TestEnvironment
[appengine-java] Re: embedding parent object in parent-child relation with jdo
John, I still hope to find a plain JDO solution that won't tie my application to GAE-specific model and helper libraries that working around GAE restrictions. Especially because it is not mentioned in the docs that it is not allowed to use parent-shild relation and have parent embedded into the child. More over, ArrayIndexOutOfBound exception look rather suspicious and may actually mean that there is rather issue with configuration or even with jdo runtime implementation. It would be great to hear back from the DataNucleus developers on that. Also, I would prefer to not manually inlinine parent's data into the child because it a) cause data duplication (e.g. I have set the data in all places upon creation); b) it makes model even worse when parent has more then one child type (e.g. order/orderItem, order/ orderDelivery, etc). I should mention that I used an order model only as an example and my model is a bit more complex then that. Thanks Eugene On Dec 21, 11:53 pm, John Patterson wrote: > In that case, do you need the child to be an Entity at all? Perhaps you > could simply embed it in the parent. > > Not too sure about the JDO case, I think it might be impossible to embed a > collection unless it is of "native" values. But with Twig you can embed an > entire collection of child objects and they can be automatically encoded as > multi-value properties that you can then query on. It does this using the > order of multi valued properties. For example: > > name="Order name" > date="1 Dec 2009" > items.name=["An item", "Another Item", "Yet another item"] > items.price=[3.45, 1.67, 8.98] > > You can now filter (or sort) by a child property and partent property. You > cannot find a parent with a certain price AND a name but your case sounds > fine. Don't worry, Twig encodes and decodes the objects. This only works > for one level of collection; if an item had a collection of parts Twig can > still encode it but not as ordered multi-value properties. > > John > > 2009/12/22 Eugene Kuleshov > > > On Dec 19, 2:56 am, jd wrote: > > > > Could you not split your query into two? The first to find the parent > > > and the second to return all children with an ancestor query. > > > In my case the main filtering criteria is actually by one of the > > child's attributes, but then I need to sort the results and present > > them using some of the parent's attributes. For example, with Order > > and OrderItem model, find all OrderItems that include the given part > > number and then sort the result according to the Order's date. So, I > > do need to preserve parent-child relationship, but don't want to > > execute N+1 queries (one for each parent referenced from the found > > children). > > > Also, I would like to maintain parent-child relationship, so entity > > groups are handled automatically (so I won't have to manually create > > foreign keys), i.e. I can process all parent-child chunks in one > > transaction. > > > Thanks > > > Eugene > > > > On Dec 19, 3:58 am, Eugene Kuleshov wrote: > > > > > It shouldn't be needed, since according to the docs [1], the Key > > > > type is only required to record parent relation in the child. > > > > But I tried it anyways and got pretty much the same exception: > > > > > Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: 0 > > > > at > > org.datanucleus.store.appengine.DatastoreTable.initializeNonPK > > > > (DatastoreTable.java:356) > > > > at org.datanucleus.store.appengine.DatastoreTable.buildMapping > > > > (DatastoreTable.java:285) > > > > at > > org.datanucleus.store.appengine.DatastoreManager.buildStoreData > > > > (DatastoreManager.java:405) > > > > ... > > > > > regards, > > > > Eugene > > > > > [1] > >http://code.google.com/appengine/docs/java/datastore/creatinggettinga... > > > > > On Dec 18, 2:33 pm, Rusty Wright wrote: > > > > > > Try making the parent's primary key be Key instead of String. > > > > > > Eugene Kuleshov wrote: > > > > > > I am using the app engine SDK 1.3.0 with the latest Google > > Eclipse > > > > > > plugin. > > > > > > > Currently, in order to be able to run queries on attributes of > > the > > > > > > parent object I have to embed parent object into a child. So, I > > have > > > > > > created
[appengine-java] Re: Error creating EntityManagerFactory bean occasionally
I am also seeing that every request hits "Initializing Spring FrameworkServlet", which takes good chunk of CPU and significantly slows down the response, ut to a level when web application in not useable. It been mentioned before that there is some ongoing work to reduce the startup time and I wonder if there is any opdate on that yet? Thanks Eugene On Jan 8, 10:34 am, Wong wrote: > I suspect the cause of the problem is the ping cron every 3 minutes. I > also get the following warning: > > "This request used a high amount of CPU, and was roughly 1.2 times > over the average request CPU limit. High CPU requests have a small > quota, and if you exceed this quota, your app will be temporarily > disabled." > > The ping cron is very simple task. It accesses an URL which output a > short text > > Full stack trace: > # > > 0.1.0.1 - - [08/Jan/2010:07:22:05 -0800] "GET /ping HTTP/1.1" 200 45 - > - "mobisociety.appspot.com" > > # > I 01-08 07:21AM 43.585 > > javax.servlet.ServletContext log:InitializingSpringroot > WebApplicationContext > > # > W 01-08 07:21AM 47.796 > > org.springframework.security.config.method.GlobalMethodSecurityBeanDefinitionParser > parse: Expressions were enabled for method security but no > SecurityExpressionHandler was configured. All hasPermision() > expressions will evaluate to false. > > # > I 01-08 07:22AM 03.596 > > javax.servlet.ServletContext log: > org.tuckey.web.filters.urlrewrite.UrlRewriteFilter INFO: loaded (conf > ok) > > # > I 01-08 07:22AM 03.689 > > javax.servlet.ServletContext log:InitializingSpringFrameworkServlet > 'dispatcher' > > # > E 01-08 07:22AM 05.397 > > org.apache.jasper.runtime.JspFactoryImpl internalGetPageContext: > Exceptioninitializingpage context > java.lang.ExceptionInInitializerError > at com.google.appengine.api.memcache.MemcacheServicePb > $MemcacheSetResponse.(MemcacheServicePb.java:2649) > at com.google.appengine.api.memcache.MemcacheServiceImpl.put > (MemcacheServiceImpl.java:315) > at com.google.appengine.api.memcache.MemcacheServiceImpl.put > (MemcacheServiceImpl.java:376) > at com.google.apphosting.runtime.jetty.SessionManager.createSession > (SessionManager.java:334) > at com.google.apphosting.runtime.jetty.SessionManager > $AppEngineSession.(SessionManager.java:135) > at com.google.apphosting.runtime.jetty.SessionManager.newSession > (SessionManager.java:265) > at com.google.apphosting.runtime.jetty.SessionManager.newSession > (SessionManager.java:50) > at org.mortbay.jetty.servlet.AbstractSessionManager.newHttpSession > (AbstractSessionManager.java:413) > at org.mortbay.jetty.Request.getSession(Request.java:1005) > at org.mortbay.jetty.Request.getSession(Request.java:977) > at org.apache.jasper.runtime.PageContextImpl._initialize > (PageContextImpl.java:147) > at org.apache.jasper.runtime.PageContextImpl.initialize > (PageContextImpl.java:122) > at org.apache.jasper.runtime.JspFactoryImpl.internalGetPageContext > (JspFactoryImpl.java:104) > at org.apache.jasper.runtime.JspFactoryImpl.access$000 > (JspFactoryImpl.java:37) > at org.apache.jasper.runtime.JspFactoryImpl > $PrivilegedGetPageContext.run(JspFactoryImpl.java:151) > at > com.google.apphosting.runtime.security.shared.intercept.java.security.AccessController_.doPrivileged > (AccessController_.java:34) > at org.apache.jasper.runtime.JspFactoryImpl.getPageContext > (JspFactoryImpl.java:59) > at org.apache.jsp.ping_jsp._jspService(ping_jsp.java:33) > at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:94) > at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) > at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java: > 487) > at org.mortbay.jetty.servlet.ServletHandler.handle > (ServletHandler.java:362) > at org.mortbay.jetty.security.SecurityHandler.handle > (SecurityHandler.java:216) > at org.mortbay.jetty.servlet.SessionHandler.handle > (SessionHandler.java:181) > at org.mortbay.jetty.handler.ContextHandler.handle > (ContextHandler.java:712) > at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java: > 405) > at org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:268) > at org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:126) > at org.tuckey.web.filters.urlrewrite.NormalRewrittenUrl.doRewrite > (NormalRewrittenUrl.java:195) > at org.tuckey.web.filters.urlrewrite.RuleChain.handleRewrite > (RuleChain.java:159) > at org.tuckey.web.filters.urlrewrite.RuleChain.doRules(RuleChain.java: > 141) > at org.tuckey.web.filters.urlrewrite.UrlRewriter.processRequest > (UrlRewriter.java:90) > at org.tuckey.web.filters.urlrewrite.UrlRewriteFilter.doFilter > (UrlRewriteFilter.java:417) > at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter > (
[appengine-java] Scheduled Tasks aborted early
I have Java application deployed on the appengine and it has several job definitions. However I see that Scheduled Tasks are aborted within 10 seconds from the start, even so corresponding FAQ entry states 30 seconds. http://code.google.com/appengine/docs/java/runtime.html#Quotas_and_Limits This is a big problem for any applications using Springframework, which has startup time about 20 seconds. Here is corresponding entry from the log file for one of the executions of such scheduled task. --- 1. 02-28 02:45PM 39.147 /fetch.htm?... 500 10012ms 0cpu_ms 0kb See details 0.1.0.1 - - [28/Feb/2010:14:45:49 -0800] "GET /fetch.htm?... HTTP/1.1" 500 0 - - "ipsc-stats.appspot.com" 2. W 02-28 02:45PM 49.160 Request was aborted after waiting too long to attempt to service your request. Most likely, this indicates that you have reached your simultaneous dynamic request limit. This is almost always due to excessively high latency in your app. Please see http://code.google.com/appengine/docs/quotas.html for more details. --- Can you please advise what could be done in such and if I am missing anything obvious? Thanks Eugene -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Scheduled Tasks aborted early
John, My application is hardly used right now, so there is certainly NO other requests coming in. So, I still don't understand why execution is cut after 10 seconds instead of promised 30 seconds. With such limitation GAE for Java is practically making impossible to use the most popular web framework (i.e. Spring MVC + Spring Security + Spring Core), but I'd be interested to hear how you can replace such stack with something based on Guice. regards, Eugene On Feb 28, 11:55 pm, John Patterson wrote: > You get this message when you app is still starting while other > requests come in. The only current solution is to reduce start up > time - pinging is not a satisfactory solution because you still get > frequent "loading requests". With Guice I was able to reduce star up > by using a non AOP version which did no bytecode enhancement. Also, > try delaying initialising components until they are needed. > > On 1 Mar 2010, at 08:45, Eugene Kuleshov wrote: > > > > > I have Java application deployed on the appengine and it has several > > job definitions. However I see that Scheduled Tasks are aborted within > > 10 seconds from the start, even so corresponding FAQ entry states 30 > > seconds.http://code.google.com/appengine/docs/java/runtime.html#Quotas_and_Li... > > > This is a big problem for any applications using Springframework, > > which has startup time about 20 seconds. > > > Here is corresponding entry from the log file for one of the > > executions of such scheduled task. > > > --- > > 1. 02-28 02:45PM 39.147 /fetch.htm?... 500 10012ms 0cpu_ms 0kb > > See details > > > 0.1.0.1 - - [28/Feb/2010:14:45:49 -0800] "GET /fetch.htm?... > > HTTP/1.1" 500 0 - - "ipsc-stats.appspot.com" > > > 2. W 02-28 02:45PM 49.160 > > > Request was aborted after waiting too long to attempt to service > > your request. Most likely, this indicates that you have reached your > > simultaneous dynamic request limit. This is almost always due to > > excessively high latency in your app. Please see > >http://code.google.com/appengine/docs/quotas.htmlfor more details. > > --- > > > Can you please advise what could be done in such and if I am missing > > anything obvious? > > > Thanks > > > Eugene > > > -- > > You received this message because you are subscribed to the Google > > Groups "Google App Engine for Java" group. > > To post to this group, send email to google-appengine-java@googlegroups.com > > . > > To unsubscribe from this group, send email to > > google-appengine-java+unsubscr...@googlegroups.com > > . > > For more options, visit this group > > athttp://groups.google.com/group/google-appengine-java?hl=en > > . -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Scheduled Tasks aborted early
Also, that this 10 seconds issue actually appear mostly when my scheduled tasks are kicked, so in result they never start, even so user requests from the regular UI are coming trough. It seem like this problem been introduced recently Thanks Eugene On Mar 1, 11:35 am, Eugene Kuleshov wrote: > John, > > My application is hardly used right now, so there is certainly NO > other requests coming in. So, I still don't understand why execution > is cut after 10 seconds instead of promised 30 seconds. > > With such limitation GAE for Java is practically making impossible > to use the most popular web framework (i.e. Spring MVC + Spring > Security + Spring Core), but I'd be interested to hear how you can > replace such stack with something based on Guice. > > regards, > Eugene > > On Feb 28, 11:55 pm, John Patterson wrote: > > > You get this message when you app is still starting while other > > requests come in. The only current solution is to reduce start up > > time - pinging is not a satisfactory solution because you still get > > frequent "loading requests". With Guice I was able to reduce star up > > by using a non AOP version which did no bytecode enhancement. Also, > > try delaying initialising components until they are needed. > > > On 1 Mar 2010, at 08:45, Eugene Kuleshov wrote: > > > > I have Java application deployed on the appengine and it has several > > > job definitions. However I see that Scheduled Tasks are aborted within > > > 10 seconds from the start, even so corresponding FAQ entry states 30 > > > seconds.http://code.google.com/appengine/docs/java/runtime.html#Quotas_and_Li... > > > > This is a big problem for any applications using Springframework, > > > which has startup time about 20 seconds. > > > > Here is corresponding entry from the log file for one of the > > > executions of such scheduled task. > > > > --- > > > 1. 02-28 02:45PM 39.147 /fetch.htm?... 500 10012ms 0cpu_ms 0kb > > > See details > > > > 0.1.0.1 - - [28/Feb/2010:14:45:49 -0800] "GET /fetch.htm?... > > > HTTP/1.1" 500 0 - - "ipsc-stats.appspot.com" > > > > 2. W 02-28 02:45PM 49.160 > > > > Request was aborted after waiting too long to attempt to service > > > your request. Most likely, this indicates that you have reached your > > > simultaneous dynamic request limit. This is almost always due to > > > excessively high latency in your app. Please see > > >http://code.google.com/appengine/docs/quotas.htmlformore details. > > > --- > > > > Can you please advise what could be done in such and if I am missing > > > anything obvious? > > > > Thanks > > > > Eugene > > > > -- > > > You received this message because you are subscribed to the Google > > > Groups "Google App Engine for Java" group. > > > To post to this group, send email to > > > google-appengine-java@googlegroups.com > > > . > > > To unsubscribe from this group, send email to > > > google-appengine-java+unsubscr...@googlegroups.com > > > . > > > For more options, visit this group > > > athttp://groups.google.com/group/google-appengine-java?hl=en > > > . -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Scheduled Tasks aborted early
John, I hear you, but I am quite certain that there is no other requests. It happen in a middle of the night when there is no users and the failing request is initiated by the GAE's own cron, so there is no refresh of the pages, nor any requests for css. With Spring framework, the most of the initialization happens in either startup servlet or context listener, so splitting application won't address the issue. Another problem is that request is terminated without providing any stack trace, so it is rather hard to debug issue on a live system. regards, Eugene On Mar 1, 12:10 pm, John Patterson wrote: > Hi Eugene, > > All it takes are two requests to see that error - the first to start > the application loading and then any other request (even a request for > a css file or refreshing the page) will wait 10 seconds and then throw > that exception. It is rare that a single page does not need more than > one request. > > In my opinion this loading request problem is the biggest hidden > gotcha that should be written in big red letters on the introduction > page. It could end up requiring you to discard the frameworks you are > accustomed to using. > > Something which helped me was to split my app into an "admin" version > and a user facing "client" version. That reduced the loading time of > the client version significantly. > > On 1 Mar 2010, at 23:35, Eugene Kuleshov wrote: > > > John, > > > My application is hardly used right now, so there is certainly NO > > other requests coming in. So, I still don't understand why execution > > is cut after 10 seconds instead of promised 30 seconds. > > > With such limitation GAE for Java is practically making impossible > > to use the most popular web framework (i.e. Spring MVC + Spring > > Security + Spring Core), but I'd be interested to hear how you can > > replace such stack with something based on Guice. > > > regards, > > Eugene > > > On Feb 28, 11:55 pm, John Patterson wrote: > >> You get this message when you app is still starting while other > >> requests come in. The only current solution is to reduce start up > >> time - pinging is not a satisfactory solution because you still get > >> frequent "loading requests". With Guice I was able to reduce star up > >> by using a non AOP version which did no bytecode enhancement. Also, > >> try delaying initialising components until they are needed. > > >> On 1 Mar 2010, at 08:45, Eugene Kuleshov wrote: > > >>> I have Java application deployed on the appengine and it has > >>> several > >>> job definitions. However I see that Scheduled Tasks are aborted > >>> within > >>> 10 seconds from the start, even so corresponding FAQ entry states 30 > >>> seconds.http://code.google.com/appengine/docs/java/runtime.html#Quotas_and_Li > >>> ... > > >>> This is a big problem for any applications using Springframework, > >>> which has startup time about 20 seconds. > > >>> Here is corresponding entry from the log file for one of the > >>> executions of such scheduled task. > > >>> --- > >>> 1. 02-28 02:45PM 39.147 /fetch.htm?... 500 10012ms 0cpu_ms 0kb > >>> See details > > >>> 0.1.0.1 - - [28/Feb/2010:14:45:49 -0800] "GET /fetch.htm?... > >>> HTTP/1.1" 500 0 - - "ipsc-stats.appspot.com" > > >>> 2. W 02-28 02:45PM 49.160 > > >>> Request was aborted after waiting too long to attempt to > >>> service > >>> your request. Most likely, this indicates that you have reached your > >>> simultaneous dynamic request limit. This is almost always due to > >>> excessively high latency in your app. Please see > >>>http://code.google.com/appengine/docs/quotas.htmlformore details. > >>> --- > > >>> Can you please advise what could be done in such and if I am > >>> missing > >>> anything obvious? > > >>> Thanks > > >>> Eugene > > >>> -- > >>> You received this message because you are subscribed to the Google > >>> Groups "Google App Engine for Java" group. > >>> To post to this group, send email to > >>> google-appengine-java@googlegroups.com > >>> . > >>> To unsubscribe from this group, send email to > >>> google-appengine-java+unsubscr...@googlegroups.com > >>> . > >>> For more options, visit this
[appengine-java] Re: Scheduled Tasks aborted early
John, I've posted the pretty much complete log in my very first message. If I read it correctly the request took 10012ms and it haven't used any CPU (0cpu_ms) and there was NO other requests within 30 minutes around that time. More over, if I kick the very same url as defined in cron.xml manually from the browser, it works. So, the issue seem only affects requests coming from the cron.xml The only thing I could do to to avoid execution of context listener (besides of dropping Spring MVC and Spring Security) is split application into a two separate apps, but then I won't be able to access the same data storage. regards, Eugene On Mar 1, 12:41 pm, John Patterson wrote: > When you get the "Request was aborted after waiting too long ..." > message there will be no stack trace because your code is never run. > Rest assured you can run tasks for 30 seconds - I do it myself. > > Context listeners are run on every loading request so why do you say > that splitting your code will not reduce start up time? Surely if > there is less code to initialise it will take less time no? > > Could you post your log file of requests before and after the error? > > On 2 Mar 2010, at 00:17, Eugene Kuleshov wrote: > > > > > John, I hear you, but I am quite certain that there is no other > > requests. It happen in a middle of the night when there is no users > > and the failing request is initiated by the GAE's own cron, so there > > is no refresh of the pages, nor any requests for css. > > > With Spring framework, the most of the initialization happens in > > either startup servlet or context listener, so splitting application > > won't address the issue. > > > Another problem is that request is terminated without providing any > > stack trace, so it is rather hard to debug issue on a live system. > > > regards, > > Eugene > > > On Mar 1, 12:10 pm, John Patterson wrote: > >> Hi Eugene, > > >> All it takes are two requests to see that error - the first to start > >> the application loading and then any other request (even a request > >> for > >> a css file or refreshing the page) will wait 10 seconds and then > >> throw > >> that exception. It is rare that a single page does not need more > >> than > >> one request. > > >> In my opinion this loading request problem is the biggest hidden > >> gotcha that should be written in big red letters on the introduction > >> page. It could end up requiring you to discard the frameworks you > >> are > >> accustomed to using. > > >> Something which helped me was to split my app into an "admin" version > >> and a user facing "client" version. That reduced the loading time of > >> the client version significantly. > > >> On 1 Mar 2010, at 23:35, Eugene Kuleshov wrote: > > >>> John, > > >>> My application is hardly used right now, so there is certainly NO > >>> other requests coming in. So, I still don't understand why execution > >>> is cut after 10 seconds instead of promised 30 seconds. > > >>> With such limitation GAE for Java is practically making impossible > >>> to use the most popular web framework (i.e. Spring MVC + Spring > >>> Security + Spring Core), but I'd be interested to hear how you can > >>> replace such stack with something based on Guice. > > >>> regards, > >>> Eugene > > >>> On Feb 28, 11:55 pm, John Patterson wrote: > >>>> You get this message when you app is still starting while other > >>>> requests come in. The only current solution is to reduce start up > >>>> time - pinging is not a satisfactory solution because you still get > >>>> frequent "loading requests". With Guice I was able to reduce > >>>> star up > >>>> by using a non AOP version which did no bytecode enhancement. > >>>> Also, > >>>> try delaying initialising components until they are needed. > > >>>> On 1 Mar 2010, at 08:45, Eugene Kuleshov wrote: > > >>>>> I have Java application deployed on the appengine and it has > >>>>> several > >>>>> job definitions. However I see that Scheduled Tasks are aborted > >>>>> within > >>>>> 10 seconds from the start, even so corresponding FAQ entry > >>>>> states 30 > >>>>> seconds.http://code.google.com/appengine/docs/java/runtime.html#Quot
[appengine-java] Re: Scheduled Tasks aborted early
On Mar 1, 1:19 pm, John Patterson wrote: > The pasted log is a bit hard to read... but it looks to me like two > requests. One at "02-28 02:45PM 39" and another at "02-28 02:45PM > 49". Or is that just the way it printed? Just one request. That is how it looks in the log browser at the web UI. > The idea is to split the app into different _versions_ not different > apps. Then they use the same datastore but apart from that act almost > like separate apps. In my situation my admin app takes care of all > the loading and processing data - kicking off tasks etc. I see, it is an interesting, yet quite weird workaround. regards, Eugene -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Scheduled Tasks aborted early
John, There was no requests more then 30 minutes before that one and more then 30 minutes after that. So, I don't think those requests would matter. Take care Eugene On Mon, Mar 1, 2010 at 2:24 PM, John Patterson wrote: > > On 2 Mar 2010, at 02:18, Eugene Kuleshov wrote: > > Just one request. That is how it looks in the log browser at the web > UI. > > OK then could you post the logs surrounding that one request? Expanded. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to google-appengine-j...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine-java+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] performance of Task Queue Java API
My application need to create bunch of tasks to do some data processing. I've tried to prototype a small application that spawns 5000 tasks from the process initiated by cron job, but it seem like I am hitting some wall, because my test can't spawn more then 1000 tasks and it is terminated by GAE runtime after 30 seconds. Is there any known performance limitations of the Task Queue Java API or any best practices on how to spawn large number of tasks? Thanks Eugene -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
