[appengine-java] Re: Put (insert) data store operations use increasingly high amount of CPU as the number of entities increases

[Lucian Baciu]

Sorry, I have 4000 entities in the entity group (child entities).

On Jan 21, 12:21 am, "Ikai L (Google)"  wrote:
> Do you have 4000 entities total? Or 4000 child entities for the model?
>
>
>
>
>
> On Wed, Jan 20, 2010 at 2:04 PM, Lucian Baciu  wrote:
> > This is what an insert request looks like now: 10659ms 44687cpu_ms
> > 40040api_cpu_ms for about 4000 entities in the model. I would say its
> > increases by 10 seconds per 1000 entities.
>
> > On Jan 21, 12:01 am, "Ikai L (Google)"  wrote:
> > > How much does the CPU time increase by?
>
> > > On Wed, Jan 20, 2010 at 1:59 PM, Lucian Baciu 
> > wrote:
> > > > In total my model has about 4000 entities. The shape of my model is
> > > > like this: a root entity that has many owned one to many
> > > > relationships
> > > > (about 10), and each children collection can have hundreds or
> > > > thousands of child entities in them, something like:
>
> > > > public class Account{
> > > >         @Persistent(mappedBy = "account")
> > > >       �...@order(extensions = @Extension(vendorName="datanucleus",
> > > > key="list-
> > > > ordering", value="key asc"))
> > > >         private List children1;
> > > >         @Persistent(mappedBy = "account")
> > > >       �...@order(extensions = @Extension(vendorName="datanucleus",
> > > > key="list-
> > > > ordering", value="key asc"))
> > > >         private List children2;
> > > >        //
> > > >         @Persistent(mappedBy = "account")
> > > >       �...@order(extensions = @Extension(vendorName="datanucleus",
> > > > key="list-
> > > > ordering", value="key asc"))
> > > >         private List children10;
> > > > }
>
> > > > Yes my inserts are in transaction. I've marked all entity properties
> > > > not used in custom indexes as unindexed, but I still see the same
> > > > problem. The CPU time continues to increase for the same insert
> > > > operation of one entity as the group gets larger. Also, I have no
> > > > custom indexes with two list properties, so I don't think I have an
> > > > exploding index problem.
>
> > > > On Jan 20, 11:45 pm, "Ikai L (Google)"  wrote:
> > > > > How many child entities does a model have? Is it in a transaction?
> > Index
> > > > > updates are synchronous, though a simple update to the index should
> > not
> > > > > dramatically increase the insert time.
>
> > > > > In addition to reducing the indexed properties, you may also want to
> > look
> > > > at
> > > > > inserting entities asynchronously via the Task Queue. It won't reduce
> > the
> > > > > insert time, but it will reduce the amount of time your application
> > > > spends
> > > > > serving a response to a user.
>
> > > > > On Mon, Jan 18, 2010 at 11:30 AM, Lucian Baciu <
> > lucianba...@gmail.com
> > > > >wrote:
>
> > > > > > That's a great video, but I don't think it can help me. I think my
> > > > > > model is right, it is basically 1 to many owned relationship and as
> > > > > > the number of child entities grows so does the insert operations
> > time.
> > > > > > I can't figure out why! Any idea why???
>
> > > > > > Thanks,
> > > > > > Lucian
>
> > > > > > On Jan 15, 8:55 pm, "Ikai L (Google)"  wrote:
> > > > > > > Brett Slakin on the App Engine team gave a great talk about a
> > problem
> > > > > > > similar to what you are describing at last year's Google I/O and
> > how
> > > > to
> > > > > > > solve it. You may want to check this video out:
>
> > > >http://code.google.com/events/io/2009/sessions/BuildingScalableComple.
> > ..
>
> > > > > > > <
> > > >http://code.google.com/events/io/2009/sessions/BuildingScalableComple..
> > > > > > .>How
> > > > > > > are your models current set up?
>
> > > > > > > On Fri, Jan 15, 2010 at 1:10 AM, Lucian Baciu <
> > lucianba...@gmail.com
>
> > > > > > wrote:
> > > > > > > > I have a root User entity and a child Note entity. Each User
> > can
> > > > have
> > > > > > > > many Note entities. The Note entity has four indices.
>
> > > > > > > > I've noticed that as the number of Note entities for a User
> > > > increases
> > > > > > > > so does the request time/latency and CPU used on Note insert
> > > > > > > > operations. When the number on Note entities for a user is
> > about
> > > > 1000,
> > > > > > > > an insert Note request takes about 10 seconds in request time
> > and
> > > > > > > > about 40 seconds of CPU, which is very high and as it continues
> > to
> > > > > > > > increase it makes my app unusable.
>
> > > > > > > > Is there something I can do to optimize this, or is this an app
> > > > engine
> > > > > > > > issue?
>
> > > > > > > > Thanks,
> > > > > > > > Lucian
>
> > > > > > > > --
> > > > > > > > You received this message because you are subscribed to the
> > Google
> > > > > > Groups
> > > > > > > > "Google App Engine for Java" group.
> > > > > > > > To post to this group, send email to
> > > > > > > > google-appengine-j...@googlegroups.com.
> > > > > > > > To unsubscribe from this group, send email to
> > > > > > > > google-appengine-java+unsubscr...@googlegroups.c

Re: [appengine-java] Re: Put (insert) data store operations use increasingly high amount of CPU as the number of entities increases

[Ikai L (Google)]

Do you have 4000 entities total? Or 4000 child entities for the model?

On Wed, Jan 20, 2010 at 2:04 PM, Lucian Baciu  wrote:

> This is what an insert request looks like now: 10659ms 44687cpu_ms
> 40040api_cpu_ms for about 4000 entities in the model. I would say its
> increases by 10 seconds per 1000 entities.
>
>
> On Jan 21, 12:01 am, "Ikai L (Google)"  wrote:
> > How much does the CPU time increase by?
> >
> >
> >
> >
> >
> > On Wed, Jan 20, 2010 at 1:59 PM, Lucian Baciu 
> wrote:
> > > In total my model has about 4000 entities. The shape of my model is
> > > like this: a root entity that has many owned one to many
> > > relationships
> > > (about 10), and each children collection can have hundreds or
> > > thousands of child entities in them, something like:
> >
> > > public class Account{
> > > @Persistent(mappedBy = "account")
> > >@Order(extensions = @Extension(vendorName="datanucleus",
> > > key="list-
> > > ordering", value="key asc"))
> > > private List children1;
> > > @Persistent(mappedBy = "account")
> > >@Order(extensions = @Extension(vendorName="datanucleus",
> > > key="list-
> > > ordering", value="key asc"))
> > > private List children2;
> > >//
> > > @Persistent(mappedBy = "account")
> > >@Order(extensions = @Extension(vendorName="datanucleus",
> > > key="list-
> > > ordering", value="key asc"))
> > > private List children10;
> > > }
> >
> > > Yes my inserts are in transaction. I've marked all entity properties
> > > not used in custom indexes as unindexed, but I still see the same
> > > problem. The CPU time continues to increase for the same insert
> > > operation of one entity as the group gets larger. Also, I have no
> > > custom indexes with two list properties, so I don't think I have an
> > > exploding index problem.
> >
> > > On Jan 20, 11:45 pm, "Ikai L (Google)"  wrote:
> > > > How many child entities does a model have? Is it in a transaction?
> Index
> > > > updates are synchronous, though a simple update to the index should
> not
> > > > dramatically increase the insert time.
> >
> > > > In addition to reducing the indexed properties, you may also want to
> look
> > > at
> > > > inserting entities asynchronously via the Task Queue. It won't reduce
> the
> > > > insert time, but it will reduce the amount of time your application
> > > spends
> > > > serving a response to a user.
> >
> > > > On Mon, Jan 18, 2010 at 11:30 AM, Lucian Baciu <
> lucianba...@gmail.com
> > > >wrote:
> >
> > > > > That's a great video, but I don't think it can help me. I think my
> > > > > model is right, it is basically 1 to many owned relationship and as
> > > > > the number of child entities grows so does the insert operations
> time.
> > > > > I can't figure out why! Any idea why???
> >
> > > > > Thanks,
> > > > > Lucian
> >
> > > > > On Jan 15, 8:55 pm, "Ikai L (Google)"  wrote:
> > > > > > Brett Slakin on the App Engine team gave a great talk about a
> problem
> > > > > > similar to what you are describing at last year's Google I/O and
> how
> > > to
> > > > > > solve it. You may want to check this video out:
> >
> > >http://code.google.com/events/io/2009/sessions/BuildingScalableComple.
> ..
> >
> > > > > > <
> > >http://code.google.com/events/io/2009/sessions/BuildingScalableComple..
> > > > > .>How
> > > > > > are your models current set up?
> >
> > > > > > On Fri, Jan 15, 2010 at 1:10 AM, Lucian Baciu <
> lucianba...@gmail.com
> >
> > > > > wrote:
> > > > > > > I have a root User entity and a child Note entity. Each User
> can
> > > have
> > > > > > > many Note entities. The Note entity has four indices.
> >
> > > > > > > I've noticed that as the number of Note entities for a User
> > > increases
> > > > > > > so does the request time/latency and CPU used on Note insert
> > > > > > > operations. When the number on Note entities for a user is
> about
> > > 1000,
> > > > > > > an insert Note request takes about 10 seconds in request time
> and
> > > > > > > about 40 seconds of CPU, which is very high and as it continues
> to
> > > > > > > increase it makes my app unusable.
> >
> > > > > > > Is there something I can do to optimize this, or is this an app
> > > engine
> > > > > > > issue?
> >
> > > > > > > Thanks,
> > > > > > > Lucian
> >
> > > > > > > --
> > > > > > > You received this message because you are subscribed to the
> Google
> > > > > Groups
> > > > > > > "Google App Engine for Java" group.
> > > > > > > To post to this group, send email to
> > > > > > > google-appengine-j...@googlegroups.com.
> > > > > > > To unsubscribe from this group, send email to
> > > > > > > google-appengine-java+unsubscr...@googlegroups.com unsubscr...@googlegroups.com> > > unsubscr...@googlegroups.com> > > > > unsubscr...@googlegroups.com>
> > > > > > > .
> > > > > > > For more options, visit this group at
> > > > > > >http://groups.google.com/group/google-appengine-java?hl=en.
> >
> > > > > > --
> > > > > > Ikai Lan
> > > >

[appengine-java] Re: Put (insert) data store operations use increasingly high amount of CPU as the number of entities increases

[Lucian Baciu]

This is what an insert request looks like now: 10659ms 44687cpu_ms
40040api_cpu_ms for about 4000 entities in the model. I would say its
increases by 10 seconds per 1000 entities.


On Jan 21, 12:01 am, "Ikai L (Google)"  wrote:
> How much does the CPU time increase by?
>
>
>
>
>
> On Wed, Jan 20, 2010 at 1:59 PM, Lucian Baciu  wrote:
> > In total my model has about 4000 entities. The shape of my model is
> > like this: a root entity that has many owned one to many
> > relationships
> > (about 10), and each children collection can have hundreds or
> > thousands of child entities in them, something like:
>
> > public class Account{
> >         @Persistent(mappedBy = "account")
> >       �...@order(extensions = @Extension(vendorName="datanucleus",
> > key="list-
> > ordering", value="key asc"))
> >         private List children1;
> >         @Persistent(mappedBy = "account")
> >       �...@order(extensions = @Extension(vendorName="datanucleus",
> > key="list-
> > ordering", value="key asc"))
> >         private List children2;
> >        //
> >         @Persistent(mappedBy = "account")
> >       �...@order(extensions = @Extension(vendorName="datanucleus",
> > key="list-
> > ordering", value="key asc"))
> >         private List children10;
> > }
>
> > Yes my inserts are in transaction. I've marked all entity properties
> > not used in custom indexes as unindexed, but I still see the same
> > problem. The CPU time continues to increase for the same insert
> > operation of one entity as the group gets larger. Also, I have no
> > custom indexes with two list properties, so I don't think I have an
> > exploding index problem.
>
> > On Jan 20, 11:45 pm, "Ikai L (Google)"  wrote:
> > > How many child entities does a model have? Is it in a transaction? Index
> > > updates are synchronous, though a simple update to the index should not
> > > dramatically increase the insert time.
>
> > > In addition to reducing the indexed properties, you may also want to look
> > at
> > > inserting entities asynchronously via the Task Queue. It won't reduce the
> > > insert time, but it will reduce the amount of time your application
> > spends
> > > serving a response to a user.
>
> > > On Mon, Jan 18, 2010 at 11:30 AM, Lucian Baciu  > >wrote:
>
> > > > That's a great video, but I don't think it can help me. I think my
> > > > model is right, it is basically 1 to many owned relationship and as
> > > > the number of child entities grows so does the insert operations time.
> > > > I can't figure out why! Any idea why???
>
> > > > Thanks,
> > > > Lucian
>
> > > > On Jan 15, 8:55 pm, "Ikai L (Google)"  wrote:
> > > > > Brett Slakin on the App Engine team gave a great talk about a problem
> > > > > similar to what you are describing at last year's Google I/O and how
> > to
> > > > > solve it. You may want to check this video out:
>
> >http://code.google.com/events/io/2009/sessions/BuildingScalableComple...
>
> > > > > <
> >http://code.google.com/events/io/2009/sessions/BuildingScalableComple..
> > > > .>How
> > > > > are your models current set up?
>
> > > > > On Fri, Jan 15, 2010 at 1:10 AM, Lucian Baciu 
> > > > wrote:
> > > > > > I have a root User entity and a child Note entity. Each User can
> > have
> > > > > > many Note entities. The Note entity has four indices.
>
> > > > > > I've noticed that as the number of Note entities for a User
> > increases
> > > > > > so does the request time/latency and CPU used on Note insert
> > > > > > operations. When the number on Note entities for a user is about
> > 1000,
> > > > > > an insert Note request takes about 10 seconds in request time and
> > > > > > about 40 seconds of CPU, which is very high and as it continues to
> > > > > > increase it makes my app unusable.
>
> > > > > > Is there something I can do to optimize this, or is this an app
> > engine
> > > > > > issue?
>
> > > > > > Thanks,
> > > > > > Lucian
>
> > > > > > --
> > > > > > You received this message because you are subscribed to the Google
> > > > Groups
> > > > > > "Google App Engine for Java" group.
> > > > > > To post to this group, send email to
> > > > > > google-appengine-j...@googlegroups.com.
> > > > > > To unsubscribe from this group, send email to
> > > > > > google-appengine-java+unsubscr...@googlegroups.com > > > > >  unsubscr...@googlegroups.com> > unsubscr...@googlegroups.com> > > > unsubscr...@googlegroups.com>
> > > > > > .
> > > > > > For more options, visit this group at
> > > > > >http://groups.google.com/group/google-appengine-java?hl=en.
>
> > > > > --
> > > > > Ikai Lan
> > > > > Developer Programs Engineer, Google App Engine
>
> > > > --
> > > > You received this message because you are subscribed to the Google
> > Groups
> > > > "Google App Engine for Java" group.
> > > > To post to this group, send email to
> > > > google-appengine-j...@googlegroups.com.
> > > > To unsubscribe from this group, send email to
> > > > google-appengine-java+unsubscr...@googlegroups.com > > >  unsubscr...@googlegrou

Re: [appengine-java] Re: Put (insert) data store operations use increasingly high amount of CPU as the number of entities increases

[Ikai L (Google)]

How much does the CPU time increase by?

On Wed, Jan 20, 2010 at 1:59 PM, Lucian Baciu  wrote:

> In total my model has about 4000 entities. The shape of my model is
> like this: a root entity that has many owned one to many
> relationships
> (about 10), and each children collection can have hundreds or
> thousands of child entities in them, something like:
>
> public class Account{
> @Persistent(mappedBy = "account")
>@Order(extensions = @Extension(vendorName="datanucleus",
> key="list-
> ordering", value="key asc"))
> private List children1;
> @Persistent(mappedBy = "account")
>@Order(extensions = @Extension(vendorName="datanucleus",
> key="list-
> ordering", value="key asc"))
> private List children2;
>//
> @Persistent(mappedBy = "account")
>@Order(extensions = @Extension(vendorName="datanucleus",
> key="list-
> ordering", value="key asc"))
> private List children10;
> }
>
> Yes my inserts are in transaction. I've marked all entity properties
> not used in custom indexes as unindexed, but I still see the same
> problem. The CPU time continues to increase for the same insert
> operation of one entity as the group gets larger. Also, I have no
> custom indexes with two list properties, so I don't think I have an
> exploding index problem.
>
>
> On Jan 20, 11:45 pm, "Ikai L (Google)"  wrote:
> > How many child entities does a model have? Is it in a transaction? Index
> > updates are synchronous, though a simple update to the index should not
> > dramatically increase the insert time.
> >
> > In addition to reducing the indexed properties, you may also want to look
> at
> > inserting entities asynchronously via the Task Queue. It won't reduce the
> > insert time, but it will reduce the amount of time your application
> spends
> > serving a response to a user.
> >
> > On Mon, Jan 18, 2010 at 11:30 AM, Lucian Baciu  >wrote:
> >
> >
> >
> >
> >
> > > That's a great video, but I don't think it can help me. I think my
> > > model is right, it is basically 1 to many owned relationship and as
> > > the number of child entities grows so does the insert operations time.
> > > I can't figure out why! Any idea why???
> >
> > > Thanks,
> > > Lucian
> >
> > > On Jan 15, 8:55 pm, "Ikai L (Google)"  wrote:
> > > > Brett Slakin on the App Engine team gave a great talk about a problem
> > > > similar to what you are describing at last year's Google I/O and how
> to
> > > > solve it. You may want to check this video out:
> >
> > > >
> http://code.google.com/events/io/2009/sessions/BuildingScalableComple...
> >
> > > > <
> http://code.google.com/events/io/2009/sessions/BuildingScalableComple..
> > > .>How
> > > > are your models current set up?
> >
> > > > On Fri, Jan 15, 2010 at 1:10 AM, Lucian Baciu  >
> > > wrote:
> > > > > I have a root User entity and a child Note entity. Each User can
> have
> > > > > many Note entities. The Note entity has four indices.
> >
> > > > > I've noticed that as the number of Note entities for a User
> increases
> > > > > so does the request time/latency and CPU used on Note insert
> > > > > operations. When the number on Note entities for a user is about
> 1000,
> > > > > an insert Note request takes about 10 seconds in request time and
> > > > > about 40 seconds of CPU, which is very high and as it continues to
> > > > > increase it makes my app unusable.
> >
> > > > > Is there something I can do to optimize this, or is this an app
> engine
> > > > > issue?
> >
> > > > > Thanks,
> > > > > Lucian
> >
> > > > > --
> > > > > You received this message because you are subscribed to the Google
> > > Groups
> > > > > "Google App Engine for Java" group.
> > > > > To post to this group, send email to
> > > > > google-appengine-j...@googlegroups.com.
> > > > > To unsubscribe from this group, send email to
> > > > > google-appengine-java+unsubscr...@googlegroups.com unsubscr...@googlegroups.com> > > unsubscr...@googlegroups.com>
> > > > > .
> > > > > For more options, visit this group at
> > > > >http://groups.google.com/group/google-appengine-java?hl=en.
> >
> > > > --
> > > > Ikai Lan
> > > > Developer Programs Engineer, Google App Engine
> >
> > > --
> > > You received this message because you are subscribed to the Google
> Groups
> > > "Google App Engine for Java" group.
> > > To post to this group, send email to
> > > google-appengine-j...@googlegroups.com.
> > > To unsubscribe from this group, send email to
> > > google-appengine-java+unsubscr...@googlegroups.com unsubscr...@googlegroups.com>
> > > .
> > > For more options, visit this group at
> > >http://groups.google.com/group/google-appengine-java?hl=en.
> >
> > --
> > Ikai Lan
> > Developer Programs Engineer, Google App Engine
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to
> google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, se

[appengine-java] Re: Put (insert) data store operations use increasingly high amount of CPU as the number of entities increases

[Lucian Baciu]

In total my model has about 4000 entities. The shape of my model is
like this: a root entity that has many owned one to many
relationships
(about 10), and each children collection can have hundreds or
thousands of child entities in them, something like:

public class Account{
@Persistent(mappedBy = "account")
@Order(extensions = @Extension(vendorName="datanucleus",
key="list-
ordering", value="key asc"))
private List children1;
@Persistent(mappedBy = "account")
@Order(extensions = @Extension(vendorName="datanucleus",
key="list-
ordering", value="key asc"))
private List children2;
//
@Persistent(mappedBy = "account")
@Order(extensions = @Extension(vendorName="datanucleus",
key="list-
ordering", value="key asc"))
private List children10;
}

Yes my inserts are in transaction. I've marked all entity properties
not used in custom indexes as unindexed, but I still see the same
problem. The CPU time continues to increase for the same insert
operation of one entity as the group gets larger. Also, I have no
custom indexes with two list properties, so I don't think I have an
exploding index problem.


On Jan 20, 11:45 pm, "Ikai L (Google)"  wrote:
> How many child entities does a model have? Is it in a transaction? Index
> updates are synchronous, though a simple update to the index should not
> dramatically increase the insert time.
>
> In addition to reducing the indexed properties, you may also want to look at
> inserting entities asynchronously via the Task Queue. It won't reduce the
> insert time, but it will reduce the amount of time your application spends
> serving a response to a user.
>
> On Mon, Jan 18, 2010 at 11:30 AM, Lucian Baciu wrote:
>
>
>
>
>
> > That's a great video, but I don't think it can help me. I think my
> > model is right, it is basically 1 to many owned relationship and as
> > the number of child entities grows so does the insert operations time.
> > I can't figure out why! Any idea why???
>
> > Thanks,
> > Lucian
>
> > On Jan 15, 8:55 pm, "Ikai L (Google)"  wrote:
> > > Brett Slakin on the App Engine team gave a great talk about a problem
> > > similar to what you are describing at last year's Google I/O and how to
> > > solve it. You may want to check this video out:
>
> > >http://code.google.com/events/io/2009/sessions/BuildingScalableComple...
>
> > >  > .>How
> > > are your models current set up?
>
> > > On Fri, Jan 15, 2010 at 1:10 AM, Lucian Baciu 
> > wrote:
> > > > I have a root User entity and a child Note entity. Each User can have
> > > > many Note entities. The Note entity has four indices.
>
> > > > I've noticed that as the number of Note entities for a User increases
> > > > so does the request time/latency and CPU used on Note insert
> > > > operations. When the number on Note entities for a user is about 1000,
> > > > an insert Note request takes about 10 seconds in request time and
> > > > about 40 seconds of CPU, which is very high and as it continues to
> > > > increase it makes my app unusable.
>
> > > > Is there something I can do to optimize this, or is this an app engine
> > > > issue?
>
> > > > Thanks,
> > > > Lucian
>
> > > > --
> > > > You received this message because you are subscribed to the Google
> > Groups
> > > > "Google App Engine for Java" group.
> > > > To post to this group, send email to
> > > > google-appengine-j...@googlegroups.com.
> > > > To unsubscribe from this group, send email to
> > > > google-appengine-java+unsubscr...@googlegroups.com > > >  unsubscr...@googlegroups.com> > unsubscr...@googlegroups.com>
> > > > .
> > > > For more options, visit this group at
> > > >http://groups.google.com/group/google-appengine-java?hl=en.
>
> > > --
> > > Ikai Lan
> > > Developer Programs Engineer, Google App Engine
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine for Java" group.
> > To post to this group, send email to
> > google-appengine-j...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine-java+unsubscr...@googlegroups.com > unsubscr...@googlegroups.com>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine-java?hl=en.
>
> --
> Ikai Lan
> Developer Programs Engineer, Google App Engine
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

Re: [appengine-java] Re: Put (insert) data store operations use increasingly high amount of CPU as the number of entities increases

[Ikai L (Google)]

How many child entities does a model have? Is it in a transaction? Index
updates are synchronous, though a simple update to the index should not
dramatically increase the insert time.

In addition to reducing the indexed properties, you may also want to look at
inserting entities asynchronously via the Task Queue. It won't reduce the
insert time, but it will reduce the amount of time your application spends
serving a response to a user.

On Mon, Jan 18, 2010 at 11:30 AM, Lucian Baciu wrote:

> That's a great video, but I don't think it can help me. I think my
> model is right, it is basically 1 to many owned relationship and as
> the number of child entities grows so does the insert operations time.
> I can't figure out why! Any idea why???
>
> Thanks,
> Lucian
>
> On Jan 15, 8:55 pm, "Ikai L (Google)"  wrote:
> > Brett Slakin on the App Engine team gave a great talk about a problem
> > similar to what you are describing at last year's Google I/O and how to
> > solve it. You may want to check this video out:
> >
> > http://code.google.com/events/io/2009/sessions/BuildingScalableComple...
> >
> >  .>How
> > are your models current set up?
> >
> >
> >
> >
> >
> > On Fri, Jan 15, 2010 at 1:10 AM, Lucian Baciu 
> wrote:
> > > I have a root User entity and a child Note entity. Each User can have
> > > many Note entities. The Note entity has four indices.
> >
> > > I've noticed that as the number of Note entities for a User increases
> > > so does the request time/latency and CPU used on Note insert
> > > operations. When the number on Note entities for a user is about 1000,
> > > an insert Note request takes about 10 seconds in request time and
> > > about 40 seconds of CPU, which is very high and as it continues to
> > > increase it makes my app unusable.
> >
> > > Is there something I can do to optimize this, or is this an app engine
> > > issue?
> >
> > > Thanks,
> > > Lucian
> >
> > > --
> > > You received this message because you are subscribed to the Google
> Groups
> > > "Google App Engine for Java" group.
> > > To post to this group, send email to
> > > google-appengine-j...@googlegroups.com.
> > > To unsubscribe from this group, send email to
> > > google-appengine-java+unsubscr...@googlegroups.com unsubscr...@googlegroups.com>
> > > .
> > > For more options, visit this group at
> > >http://groups.google.com/group/google-appengine-java?hl=en.
> >
> > --
> > Ikai Lan
> > Developer Programs Engineer, Google App Engine
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine for Java" group.
> To post to this group, send email to
> google-appengine-j...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine-java+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine-java?hl=en.
>
>
>
>


-- 
Ikai Lan
Developer Programs Engineer, Google App Engine
-- 

You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group.

To post to this group, send email to google-appengine-j...@googlegroups.com.

To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: Put (insert) data store operations use increasingly high amount of CPU as the number of entities increases

[Lucian Baciu]

That's a great video, but I don't think it can help me. I think my
model is right, it is basically 1 to many owned relationship and as
the number of child entities grows so does the insert operations time.
I can't figure out why! Any idea why???

Thanks,
Lucian

On Jan 15, 8:55 pm, "Ikai L (Google)"  wrote:
> Brett Slakin on the App Engine team gave a great talk about a problem
> similar to what you are describing at last year's Google I/O and how to
> solve it. You may want to check this video out:
>
> http://code.google.com/events/io/2009/sessions/BuildingScalableComple...
>
> How
> are your models current set up?
>
>
>
>
>
> On Fri, Jan 15, 2010 at 1:10 AM, Lucian Baciu  wrote:
> > I have a root User entity and a child Note entity. Each User can have
> > many Note entities. The Note entity has four indices.
>
> > I've noticed that as the number of Note entities for a User increases
> > so does the request time/latency and CPU used on Note insert
> > operations. When the number on Note entities for a user is about 1000,
> > an insert Note request takes about 10 seconds in request time and
> > about 40 seconds of CPU, which is very high and as it continues to
> > increase it makes my app unusable.
>
> > Is there something I can do to optimize this, or is this an app engine
> > issue?
>
> > Thanks,
> > Lucian
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine for Java" group.
> > To post to this group, send email to
> > google-appengine-j...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine-java+unsubscr...@googlegroups.com > unsubscr...@googlegroups.com>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine-java?hl=en.
>
> --
> Ikai Lan
> Developer Programs Engineer, Google App Engine
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.

[appengine-java] Re: Put (insert) data store operations use increasingly high amount of CPU as the number of entities increases

[Lucian Baciu]

I have a root Account entity:

@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class Account implements Serializable{
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Long id;

@Persistent(mappedBy = "account")
@Order(extensions = @Extension(vendorName="datanucleus", key="list-
ordering", value="key asc"))
private List notes;

//

}

and a child Note:

@PersistenceCapable(identityType = IdentityType.APPLICATION)
public class Note{
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Key key;

@Persistent
private Text note;

@Persistent
private List words;

@Persistent
private Key contact;

@Persistent
private Key project;

@Persistent
private Key createdByContact;

@Persistent
private Date date;

@Persistent
private Account account;

@Persistent
@Extension(vendorName="datanucleus", key="gae.parent-pk",
value="true")
private Key accountKey;
}

and these indices on Note:

contact ▲ , date ▼
date ▼
project ▲ , date ▼
words ▲ , date ▼

I'll look more closer into the suggestions in the video and see if
they can help me.

Lucian

On Jan 15, 8:55 pm, "Ikai L (Google)"  wrote:
> Brett Slakin on the App Engine team gave a great talk about a problem
> similar to what you are describing at last year's Google I/O and how to
> solve it. You may want to check this video out:
>
> http://code.google.com/events/io/2009/sessions/BuildingScalableComple...
>
> How
> are your models current set up?
>
>
>
>
>
> On Fri, Jan 15, 2010 at 1:10 AM, Lucian Baciu  wrote:
> > I have a root User entity and a child Note entity. Each User can have
> > many Note entities. The Note entity has four indices.
>
> > I've noticed that as the number of Note entities for a User increases
> > so does the request time/latency and CPU used on Note insert
> > operations. When the number on Note entities for a user is about 1000,
> > an insert Note request takes about 10 seconds in request time and
> > about 40 seconds of CPU, which is very high and as it continues to
> > increase it makes my app unusable.
>
> > Is there something I can do to optimize this, or is this an app engine
> > issue?
>
> > Thanks,
> > Lucian
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine for Java" group.
> > To post to this group, send email to
> > google-appengine-j...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine-java+unsubscr...@googlegroups.com > unsubscr...@googlegroups.com>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine-java?hl=en.
>
> --
> Ikai Lan
> Developer Programs Engineer, Google App Engine
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.