[appengine-java] Re: Entity relationship designing - best practice
I have the same question but this is leading to much bigger
relationship model.
As students, they may not have many exams, but I am working on an app
for financial institute so a banking client has more than 100,000,000
transactions for different account. Therefore, should I group all
transactions into the same entity group of one account or should I
still leave it separate and let an account object contain 100,000,000
keys?
Thanks in advance
WillSpecht wrote:
http://www.google.com/events/io/2009/sessions/BuildingScalableComplexApps.html
this is a pretty good discussion on building apps that have lots of
child objects. I also think you are prematurely optimizing. How many
exams is a user going to have? 100, 200? This is still a tiny number
and if you went with RB's method or the one in the link I posted, you
have to run a separate query each time, which has its own costs. I
think either way is fine until you start having users that have
thousands of exams.
On Dec 21, 3:05 am, Richard Berger richardlan...@gmail.com wrote:
Har_Shan:
I have been thinking about this topic also (but sadly, am not much of
an expert). I am using Objectify as a layer on top of the Datastore.
Their website has a lot of good information on structuring data in the
Datastore -
seehttp://code.google.com/p/objectify-appengine/wiki/Concepts?tm=6
).
There is also a good post on the various ways of handling 1 to many
relationships using Objectify. What appeared to be something that
would work in your situation (which is like my situation) is what I
think of as the database approach. What I mean by that is, if you
had a 1 to many relationship, how would you model that in a database?
Specifically if 1 User can have Many Exams, you would have...
User table
* userId
* Other user fields
Exam table
* examId
* userId
* Other exam fields
So, turning this into GWT/Objectify, it would be
public class User {
@Id private Long id;
// Other user fields
}
public class Exam{
@Id private Long id;
private KeyUser userKey;
// Other exam fields
}
The way to find all the exams for a particular user (represented by
this) is:
Objectify ofy = ObjectifyService.begin();
KeyUser userKey = new KeyUser(User.class, this.id);
ListExam result = ofy.query(Exam.class).filter(userKey,
userKey).list();
This is explained (much better than I could do)
at:http://code.google.com/p/objectify-appengine/wiki/IntroductionToObjec...
This may not solve all (or any) of your use cases, but it has worked
for me in my admittedly very limited usage.
Also check
out:http://groups.google.com/group/objectify-appengine/browse_frm/thread/...
Enjoy,
RB
On Dec 19, 12:17 am, Matej matej.crepin...@gmail.com wrote:
I am not expert in dataStore this is just what I would do:
Add new entity with fields:
ID OrderID UserID ExamID PageID QuestionID AnswerID
Yes it is a lot of redundant data, but simple.
When user starts new Exam, all data are created with AnswerID set to
general answer unAnswer. After that you just update AnswerID.
What are obstacle in design like this?
Best, M
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Entity relationship designing - best practice
Thanks all for comments. It really helped. I settled in this relationship which seems solve all problems, atleast as of now. Entity Group (EG) #1 User - User related CRUD can be done EG#2 Exam -Exam related CRUD can be done EG#3 PageIndex - Contains all (keys of) Pages under Exam, in order. Page CR and D, (re)ordering of Pages can be done simply maintaining data integrity. Page - Page CRUD can be done. Deletion will just happen in PageIndex first and then here. Can also done in transaction if needed as we are in same EG QuestionIndex - Contains all (keys of) Questions under a Page in order. Question (re)ordering can be done. Note Question can be reordered within a Page or from 1 page to another and can be done as all QuestionIndex are in same EG. Also Question CR, D can be done. EG#4 Question- Actual Question CRUD. Like Page, deletion of Question will happen in QuestionIndex (yes only its key) and lated propagated to actual entity here. Its ok if deletion of key of Question happens in QuestionIndex passes and it fails when actual Question is deleted, as we can try again later and the single source of truth would be QuestionIndex Answer - Answer CRUD Hope this also helps someone. I would be happy if someone points out flaws in my modeling. Andy, Am not clear about yr question. May be if you could show what are the relevant entities and what exactly should be the relationship btw them, i can share my 2 cents. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Entity relationship designing - best practice
On Dec 17, 8:43 pm, har_shan harsha...@gmail.com wrote: i want to hear the best practice on establishing entity relationships in my case- Does this help? http://code.google.com/p/gwt-gae-book/wiki/StoringData -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Entity relationship designing - best practice
http://www.google.com/events/io/2009/sessions/BuildingScalableComplexApps.html
this is a pretty good discussion on building apps that have lots of
child objects. I also think you are prematurely optimizing. How many
exams is a user going to have? 100, 200? This is still a tiny number
and if you went with RB's method or the one in the link I posted, you
have to run a separate query each time, which has its own costs. I
think either way is fine until you start having users that have
thousands of exams.
On Dec 21, 3:05 am, Richard Berger richardlan...@gmail.com wrote:
Har_Shan:
I have been thinking about this topic also (but sadly, am not much of
an expert). I am using Objectify as a layer on top of the Datastore.
Their website has a lot of good information on structuring data in the
Datastore - seehttp://code.google.com/p/objectify-appengine/wiki/Concepts?tm=6
).
There is also a good post on the various ways of handling 1 to many
relationships using Objectify. What appeared to be something that
would work in your situation (which is like my situation) is what I
think of as the database approach. What I mean by that is, if you
had a 1 to many relationship, how would you model that in a database?
Specifically if 1 User can have Many Exams, you would have...
User table
* userId
* Other user fields
Exam table
* examId
* userId
* Other exam fields
So, turning this into GWT/Objectify, it would be
public class User {
@Id private Long id;
// Other user fields
}
public class Exam{
@Id private Long id;
private KeyUser userKey;
// Other exam fields
}
The way to find all the exams for a particular user (represented by
this) is:
Objectify ofy = ObjectifyService.begin();
KeyUser userKey = new KeyUser(User.class, this.id);
ListExam result = ofy.query(Exam.class).filter(userKey,
userKey).list();
This is explained (much better than I could do)
at:http://code.google.com/p/objectify-appengine/wiki/IntroductionToObjec...
This may not solve all (or any) of your use cases, but it has worked
for me in my admittedly very limited usage.
Also check
out:http://groups.google.com/group/objectify-appengine/browse_frm/thread/...
Enjoy,
RB
On Dec 19, 12:17 am, Matej matej.crepin...@gmail.com wrote:
I am not expert in dataStore this is just what I would do:
Add new entity with fields:
ID OrderID UserID ExamID PageID QuestionID AnswerID
Yes it is a lot of redundant data, but simple.
When user starts new Exam, all data are created with AnswerID set to
general answer unAnswer. After that you just update AnswerID.
What are obstacle in design like this?
Best, M
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Entity relationship designing - best practice
Har_Shan:
I have been thinking about this topic also (but sadly, am not much of
an expert). I am using Objectify as a layer on top of the Datastore.
Their website has a lot of good information on structuring data in the
Datastore - see http://code.google.com/p/objectify-appengine/wiki/Concepts?tm=6
).
There is also a good post on the various ways of handling 1 to many
relationships using Objectify. What appeared to be something that
would work in your situation (which is like my situation) is what I
think of as the database approach. What I mean by that is, if you
had a 1 to many relationship, how would you model that in a database?
Specifically if 1 User can have Many Exams, you would have...
User table
* userId
* Other user fields
Exam table
* examId
* userId
* Other exam fields
So, turning this into GWT/Objectify, it would be
public class User {
@Id private Long id;
// Other user fields
}
public class Exam{
@Id private Long id;
private KeyUser userKey;
// Other exam fields
}
The way to find all the exams for a particular user (represented by
this) is:
Objectify ofy = ObjectifyService.begin();
KeyUser userKey = new KeyUser(User.class, this.id);
ListExam result = ofy.query(Exam.class).filter(userKey,
userKey).list();
This is explained (much better than I could do) at:
http://code.google.com/p/objectify-appengine/wiki/IntroductionToObjectify#Relationships
This may not solve all (or any) of your use cases, but it has worked
for me in my admittedly very limited usage.
Also check out:
http://groups.google.com/group/objectify-appengine/browse_frm/thread/102ede7022c7d7cf/30e6070900d5d523?lnk=gstq=relationships#30e6070900d5d523
Enjoy,
RB
On Dec 19, 12:17 am, Matej matej.crepin...@gmail.com wrote:
I am not expert in dataStore this is just what I would do:
Add new entity with fields:
ID OrderID UserID ExamID PageID QuestionID AnswerID
Yes it is a lot of redundant data, but simple.
When user starts new Exam, all data are created with AnswerID set to
general answer unAnswer. After that you just update AnswerID.
What are obstacle in design like this?
Best, M
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Entity relationship designing - best practice
I am not expert in dataStore this is just what I would do: Add new entity with fields: ID OrderID UserID ExamID PageID QuestionID AnswerID Yes it is a lot of redundant data, but simple. When user starts new Exam, all data are created with AnswerID set to general answer unAnswer. After that you just update AnswerID. What are obstacle in design like this? Best, M -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Entity relationship designing - best practice
Yes, understood. Thanks. Am just trying my best to keep entities in
isolation unless am really forced to group them.
There is this use case in my example:
User while creating a Exam, will create, edit, delete any Page,
Question, Answer as they wish.
At regular intervals, say 1 min, i will persist all changes. At this
time, i can be successful in persisting a Page but unsuccessful in
some Question/Answer (say)
So i will just retry failed ones at next save.
I just don't want to put all Page, Question, Answer is same entity
group and fail the save altogether if a particular entity save is
failed - just to maintain integrity of each save.
Makes sense - atleast for my example?
Also what about my relationship mentioned in my prev post?
Thanks again.
On Dec 18, 11:53 am, Didier Durand durand.did...@gmail.com wrote:
Hi,
The most important thing to remember to design your objects and their
relationship is the entity group: you cannot touch (update, delete,
create) entities belonging to more than 1 group in a single
transaction (unless you then use transaction-bound tasks)
So, keep this point in min when you maximize the number of entity
groups by keeping entities in separate groups: it will make your
transactional code a little bit harder to develop if you want to
maintain database integrity.
regards
didier
On Dec 17, 7:43 pm, har_shan harsha...@gmail.com wrote:
i want to hear the best practice on establishing entity relationships in my
case-
Example, for sake of discussion:-
User
Exam
Page
Question
Answer
As you can see, from top, each entity has 1 - many relationship with entity
immediately beneath it.
Interesting case is a User can have any number of Exam and it can *scale* in
whatever manner, but a Exam usually will have few Pages,
a Page with few Questions, Questions with few Answers
I saw this interesting post
best practice to persist medium-large
datahttp://www.mail-archive.com/google-appengine-java@googlegroups.com/ms...
(which is also a answer to this ques) and it made me to design like this:
Also i plan to keep each entity in its own entity group, as recommended by
AppEngine - totally eliminating Parent relationship, as far as possible
class User{
String name;
// ListKeyExam exams; NOT a good idea, as this will unnecessarily load
bulk amt of Exam keys when i load User
...
}
So i would use a query to get all Exams created by any User. Fine. But
class Exam{
String subject;
// With this i can *quickly* retrieve all Pages with keys, but as Exam and
Page are not in same Entity Group,
// how do i maintain *consistency* when, say, i delete a Page and
accordingly remove that Key entry in Exam.
// Also as said earlier, no. of pages (hence keys) shouldn't be too high, so
dont need to worry abt size
ListKeyPage pages;
}
class Page{
String pageNo.
KeyExam exam; // with this i can *only(?)* use query to get all Pages
under a Exam, so *unnecessary index scan*[1]
ListKeyQuestion questions;
}
The same extends to Page - Question - Answer
[1] Retrieving via Keys are faster than
queryinghttp://www.mail-archive.com/google-appengine@googlegroups.com/msg3384...
So what is your recommendation and why?
I can suspect that it should be a trade-off between unnecessary loading of
keys/extra scan of indices but want to get some thoughts and what about
consistency?
Thanks much in advance.
p.s. though not relevant, most likely i will use low level framework, mainly
Objectify..
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Entity relationship designing - best practice
Ah, i missed an important piece.
You are right, Didier. Making all entities to be isolated doesn't seem
to work out, atleast because of this important use case, in my
example:
All Pages will be in particular *order* under any Exam and similarly,
all Questions will be in some *order* as well under any Page, same for
Answers under Question.
So i might do this with a pageNo field and to maintain consistency
while updating this field, all Pages should be in entity group of
Exam, all Questions should be same group of corresponding Page - so
that i can update all in a single transaction!
This means, Exam, Page, Question and Answer should all be in same
entity group?!
Should be ok, as this entity group will not be too big - atleast its a
a User data, as per entity group thumb rule?
Or am missing something.
Thanks,
On Dec 18, 1:22 pm, har_shan harsha...@gmail.com wrote:
Yes, understood. Thanks. Am just trying my best to keep entities in
isolation unless am really forced to group them.
There is this use case in my example:
User while creating a Exam, will create, edit, delete any Page,
Question, Answer as they wish.
At regular intervals, say 1 min, i will persist all changes. At this
time, i can be successful in persisting a Page but unsuccessful in
some Question/Answer (say)
So i will just retry failed ones at next save.
I just don't want to put all Page, Question, Answer is same entity
group and fail the save altogether if a particular entity save is
failed - just to maintain integrity of each save.
Makes sense - atleast for my example?
Also what about my relationship mentioned in my prev post?
Thanks again.
On Dec 18, 11:53 am, Didier Durand durand.did...@gmail.com wrote:
Hi,
The most important thing to remember to design your objects and their
relationship is the entity group: you cannot touch (update, delete,
create) entities belonging to more than 1 group in a single
transaction (unless you then use transaction-bound tasks)
So, keep this point in min when you maximize the number of entity
groups by keeping entities in separate groups: it will make your
transactional code a little bit harder to develop if you want to
maintain database integrity.
regards
didier
On Dec 17, 7:43 pm, har_shan harsha...@gmail.com wrote:
i want to hear the best practice on establishing entity relationships in
my
case-
Example, for sake of discussion:-
User
Exam
Page
Question
Answer
As you can see, from top, each entity has 1 - many relationship with
entity
immediately beneath it.
Interesting case is a User can have any number of Exam and it can *scale*
in
whatever manner, but a Exam usually will have few Pages,
a Page with few Questions, Questions with few Answers
I saw this interesting post
best practice to persist medium-large
datahttp://www.mail-archive.com/google-appengine-java@googlegroups.com/ms...
(which is also a answer to this ques) and it made me to design like this:
Also i plan to keep each entity in its own entity group, as recommended by
AppEngine - totally eliminating Parent relationship, as far as possible
class User{
String name;
// ListKeyExam exams; NOT a good idea, as this will unnecessarily load
bulk amt of Exam keys when i load User
...
}
So i would use a query to get all Exams created by any User. Fine. But
class Exam{
String subject;
// With this i can *quickly* retrieve all Pages with keys, but as Exam and
Page are not in same Entity Group,
// how do i maintain *consistency* when, say, i delete a Page and
accordingly remove that Key entry in Exam.
// Also as said earlier, no. of pages (hence keys) shouldn't be too high,
so
dont need to worry abt size
ListKeyPage pages;
}
class Page{
String pageNo.
KeyExam exam; // with this i can *only(?)* use query to get all Pages
under a Exam, so *unnecessary index scan*[1]
ListKeyQuestion questions;
}
The same extends to Page - Question - Answer
[1] Retrieving via Keys are faster than
queryinghttp://www.mail-archive.com/google-appengine@googlegroups.com/msg3384...
So what is your recommendation and why?
I can suspect that it should be a trade-off between unnecessary loading of
keys/extra scan of indices but want to get some thoughts and what about
consistency?
Thanks much in advance.
p.s. though not relevant, most likely i will use low level framework,
mainly
Objectify..
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Entity relationship designing - best practice
Sorry again, another use case:
a Question can be moved to any Page while creating any Exam - so if
have Parent-Child relation between Page-Question,
i might need to delete Question and recreate it with new Parent - BAD
IDEA as told by Jeff in post that i mentioned.
i gave up!
On Dec 18, 1:22 pm, har_shan harsha...@gmail.com wrote:
Yes, understood. Thanks. Am just trying my best to keep entities in
isolation unless am really forced to group them.
There is this use case in my example:
User while creating a Exam, will create, edit, delete any Page,
Question, Answer as they wish.
At regular intervals, say 1 min, i will persist all changes. At this
time, i can be successful in persisting a Page but unsuccessful in
some Question/Answer (say)
So i will just retry failed ones at next save.
I just don't want to put all Page, Question, Answer is same entity
group and fail the save altogether if a particular entity save is
failed - just to maintain integrity of each save.
Makes sense - atleast for my example?
Also what about my relationship mentioned in my prev post?
Thanks again.
On Dec 18, 11:53 am, Didier Durand durand.did...@gmail.com wrote:
Hi,
The most important thing to remember to design your objects and their
relationship is the entity group: you cannot touch (update, delete,
create) entities belonging to more than 1 group in a single
transaction (unless you then use transaction-bound tasks)
So, keep this point in min when you maximize the number of entity
groups by keeping entities in separate groups: it will make your
transactional code a little bit harder to develop if you want to
maintain database integrity.
regards
didier
On Dec 17, 7:43 pm, har_shan harsha...@gmail.com wrote:
i want to hear the best practice on establishing entity relationships in
my
case-
Example, for sake of discussion:-
User
Exam
Page
Question
Answer
As you can see, from top, each entity has 1 - many relationship with
entity
immediately beneath it.
Interesting case is a User can have any number of Exam and it can *scale*
in
whatever manner, but a Exam usually will have few Pages,
a Page with few Questions, Questions with few Answers
I saw this interesting post
best practice to persist medium-large
datahttp://www.mail-archive.com/google-appengine-java@googlegroups.com/ms...
(which is also a answer to this ques) and it made me to design like this:
Also i plan to keep each entity in its own entity group, as recommended by
AppEngine - totally eliminating Parent relationship, as far as possible
class User{
String name;
// ListKeyExam exams; NOT a good idea, as this will unnecessarily load
bulk amt of Exam keys when i load User
...
}
So i would use a query to get all Exams created by any User. Fine. But
class Exam{
String subject;
// With this i can *quickly* retrieve all Pages with keys, but as Exam and
Page are not in same Entity Group,
// how do i maintain *consistency* when, say, i delete a Page and
accordingly remove that Key entry in Exam.
// Also as said earlier, no. of pages (hence keys) shouldn't be too high,
so
dont need to worry abt size
ListKeyPage pages;
}
class Page{
String pageNo.
KeyExam exam; // with this i can *only(?)* use query to get all Pages
under a Exam, so *unnecessary index scan*[1]
ListKeyQuestion questions;
}
The same extends to Page - Question - Answer
[1] Retrieving via Keys are faster than
queryinghttp://www.mail-archive.com/google-appengine@googlegroups.com/msg3384...
So what is your recommendation and why?
I can suspect that it should be a trade-off between unnecessary loading of
keys/extra scan of indices but want to get some thoughts and what about
consistency?
Thanks much in advance.
p.s. though not relevant, most likely i will use low level framework,
mainly
Objectify..
--
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
