Do you have 4000 entities total? Or 4000 child entities for the model? On Wed, Jan 20, 2010 at 2:04 PM, Lucian Baciu <lucianba...@gmail.com> wrote:
> This is what an insert request looks like now: 10659ms 44687cpu_ms
> 40040api_cpu_ms for about 4000 entities in the model. I would say its
> increases by 10 seconds per 1000 entities.
>
>
> On Jan 21, 12:01 am, "Ikai L (Google)" <ika...@google.com> wrote:
> > How much does the CPU time increase by?
> >
> >
> >
> >
> >
> > On Wed, Jan 20, 2010 at 1:59 PM, Lucian Baciu <lucianba...@gmail.com>
> wrote:
> > > In total my model has about 4000 entities. The shape of my model is
> > > like this: a root entity that has many owned one to many
> > > relationships
> > > (about 10), and each children collection can have hundreds or
> > > thousands of child entities in them, something like:
> >
> > > public class Account{
> > > @Persistent(mappedBy = "account")
> > > @Order(extensions = @Extension(vendorName="datanucleus",
> > > key="list-
> > > ordering", value="key asc"))
> > > private List<Child1> children1;
> > > @Persistent(mappedBy = "account")
> > > @Order(extensions = @Extension(vendorName="datanucleus",
> > > key="list-
> > > ordering", value="key asc"))
> > > private List<Child2> children2;
> > > //....
> > > @Persistent(mappedBy = "account")
> > > @Order(extensions = @Extension(vendorName="datanucleus",
> > > key="list-
> > > ordering", value="key asc"))
> > > private List<Child10> children10;
> > > }
> >
> > > Yes my inserts are in transaction. I've marked all entity properties
> > > not used in custom indexes as unindexed, but I still see the same
> > > problem. The CPU time continues to increase for the same insert
> > > operation of one entity as the group gets larger. Also, I have no
> > > custom indexes with two list properties, so I don't think I have an
> > > exploding index problem.
> >
> > > On Jan 20, 11:45 pm, "Ikai L (Google)" <ika...@google.com> wrote:
> > > > How many child entities does a model have? Is it in a transaction?
> Index
> > > > updates are synchronous, though a simple update to the index should
> not
> > > > dramatically increase the insert time.
> >
> > > > In addition to reducing the indexed properties, you may also want to
> look
> > > at
> > > > inserting entities asynchronously via the Task Queue. It won't reduce
> the
> > > > insert time, but it will reduce the amount of time your application
> > > spends
> > > > serving a response to a user.
> >
> > > > On Mon, Jan 18, 2010 at 11:30 AM, Lucian Baciu <
> lucianba...@gmail.com
> > > >wrote:
> >
> > > > > That's a great video, but I don't think it can help me. I think my
> > > > > model is right, it is basically 1 to many owned relationship and as
> > > > > the number of child entities grows so does the insert operations
> time.
> > > > > I can't figure out why! Any idea why???
> >
> > > > > Thanks,
> > > > > Lucian
> >
> > > > > On Jan 15, 8:55 pm, "Ikai L (Google)" <ika...@google.com> wrote:
> > > > > > Brett Slakin on the App Engine team gave a great talk about a
> problem
> > > > > > similar to what you are describing at last year's Google I/O and
> how
> > > to
> > > > > > solve it. You may want to check this video out:
> >
> > >http://code.google.com/events/io/2009/sessions/BuildingScalableComple.
> ..
> >
> > > > > > <
> > >http://code.google.com/events/io/2009/sessions/BuildingScalableComple..
> > > > > .>How
> > > > > > are your models current set up?
> >
> > > > > > On Fri, Jan 15, 2010 at 1:10 AM, Lucian Baciu <
> lucianba...@gmail.com
> >
> > > > > wrote:
> > > > > > > I have a root User entity and a child Note entity. Each User
> can
> > > have
> > > > > > > many Note entities. The Note entity has four indices.
> >
> > > > > > > I've noticed that as the number of Note entities for a User
> > > increases
> > > > > > > so does the request time/latency and CPU used on Note insert
> > > > > > > operations. When the number on Note entities for a user is
> about
> > > 1000,
> > > > > > > an insert Note request takes about 10 seconds in request time
> and
> > > > > > > about 40 seconds of CPU, which is very high and as it continues
> to
> > > > > > > increase it makes my app unusable.
> >
> > > > > > > Is there something I can do to optimize this, or is this an app
> > > engine
> > > > > > > issue?
> >
> > > > > > > Thanks,
> > > > > > > Lucian
> >
> > > > > > > --
> > > > > > > You received this message because you are subscribed to the
> Google
> > > > > Groups
> > > > > > > "Google App Engine for Java" group.
> > > > > > > To post to this group, send email to
> > > > > > > google-appengine-j...@googlegroups.com.
> > > > > > > To unsubscribe from this group, send email to
> > > > > > > google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com><google-appengine-java%2B
> unsubscr...@googlegroups.com><google-appengine-java%2B
> > > unsubscr...@googlegroups.com><google-appengine-java%2B
> > > > > unsubscr...@googlegroups.com>
> > > > > > > .
> > > > > > > For more options, visit this group at
> > > > > > >http://groups.google.com/group/google-appengine-java?hl=en.
> >
> > > > > > --
> > > > > > Ikai Lan
> > > > > > Developer Programs Engineer, Google App Engine
> >
> > > > > --
> > > > > You received this message because you are subscribed to the Google
> > > Groups
> > > > > "Google App Engine for Java" group.
> > > > > To post to this group, send email to
> > > > > google-appengine-j...@googlegroups.com.
> > > > > To unsubscribe from this group, send email to
> > > > > google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com><google-appengine-java%2B
> unsubscr...@googlegroups.com><google-appengine-java%2B
> > > unsubscr...@googlegroups.com>
> > > > > .
> > > > > For more options, visit this group at
> > > > >http://groups.google.com/group/google-appengine-java?hl=en.
> >
> > > > --
> > > > Ikai Lan
> > > > Developer Programs Engineer, Google App Engine
> >
> > > --
> > > You received this message because you are subscribed to the Google
> Groups
> > > "Google App Engine for Java" group.
> > > To post to this group, send email to
> > > google-appengine-j...@googlegroups.com.
> > > To unsubscribe from this group, send email to
> > > google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com><google-appengine-java%2B
> unsubscr...@googlegroups.com>
> > > .
> > > For more options, visit this group at
> > >http://groups.google.com/group/google-appengine-java?hl=en.
> >
> > --
> > Ikai Lan
> > Developer Programs Engineer, Google App Engine
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to
> google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com>
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>
>
>
>
--
Ikai Lan
Developer Programs Engineer, Google App Engine
-- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
