Re: [google-appengine] Re: My master/slave to high replication datastore migration experience
OK, they are just creating roll forward transactions, in multiple entity groups the same as this article by Nick Johnson. http://blog.notdot.net/2009/9/Distributed-Transactions-on-App-Engine As to the statement "you can avoid all this parenting nonsense." thats what slim3 is doing under the hood for you any way. So one way or another it has to happen. Rgds Tim -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/manImcytK_oJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: My master/slave to high replication datastore migration experience
tcp guarantees delivery built upon udp which doesn't, so it's not too hard to believe you can build something more robust on top of a limited implementation http://lmgtfy.com/?q=slim3+global+transaction it's open source so you can view the implementation for yourself to certify it's sound. I'd ask the slim3 user group. they're friendly and very responsive. good luck Mike Lawrence On Aug 15, 2011, at 10:08 PM, Tim Hoffman wrote: > HI Mike. > > You said "slim3 allows you to update two of the same kind in a single > transaction!" > > How does slim3 achieve this as the underlying datastore doesn't "currently" > support multiple entity group transactions. > > It would seem this would be a slim3 transaction (I know nothing about slim3 > by the way) and not a datastore operation, > which would suggest that the transaction isn't reliable. > > But would love to be enligntened on this topic. > > Rgds > > Tim > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/weML1S1dK6EJ. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: My master/slave to high replication datastore migration experience
HI Mike. You said "slim3 allows you to update two of the same kind in a single transaction!" How does slim3 achieve this as the underlying datastore doesn't "currently" support multiple entity group transactions. It would seem this would be a slim3 transaction (I know nothing about slim3 by the way) and not a datastore operation, which would suggest that the transaction isn't reliable. But would love to be enligntened on this topic. Rgds Tim -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/weML1S1dK6EJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: My master/slave to high replication datastore migration experience
if you use slim3 for your data store operations you can avoid all this parenting nonsense. slim3 allows you to update two of the same kind in a single transaction! why should we be required to place entities that are not logically related into a parenting relationship just to get transaction support? I have a game where a user domain object has a list of friends (users). Try modeling that as a parent relationship. It's self-referential. A user is not a parent of another user. Then try adding a list of games domain objects, and add a new friend to a new game in a single transaction. With stand-alone domain objects it's simple. Back to the original post I'm finding the latency on the master/slave datastore is too unpredictable for my game. Hopefully migrating will eliminate some of the long latency, and dynamic instance start problems i'm seeing. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/1zQr36sv23sJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: My master/slave to high replication datastore migration experience
+1 to @Waleed and @Joshua. Personally i dont really have the resources and time to deal with migration. Just switch my datastore to HR. I am willing to foot the bill for the higher pricing, but cannot take downtime and all related issues. Besides that, looking at the AppEngine group, it feels like i need to make following the group as a full time job. There are so many changes going on that it is really really hard to keep track. On Jul 13, 10:17 pm, Waleed Abdulla wrote: > I agree with Joshua. That's a very bad experience to force us to go through > to migrate. My migration is probably going to be much worse because I have > a lot of data, which means that it'll take a very long time (days?) to copy > the data across, and by that time the data would have changed on the > original app. I cannot bring the app down for extended periods of time while > I do this. > > Not to mention how much of a hassle it is to change app IDs. > > I posted an issue here, star it if you agree that this process should be > automated: > > http://code.google.com/p/googleappengine/issues/detail?id=5250 > > > > > > > > On Wed, Jul 13, 2011 at 6:41 PM, Greg wrote: > > On Jul 13, 1:05 am, Joshua Smith wrote: > > > Frankly, I've never completely groked what an entity group maps to in my > > (30+ years of) programming experience. It's certainly not a parent/child > > relationship in the usual sense. > > > If you figure it out, let me know! I have a app that needs to update > > two distinct entity types at the same time. Because they are distinct, > > I can't do both in a transaction. Even if it was only one entity type, > > I don't want to lock all of them when writing to one, which I believe > > is a consequence of making them a group. > > > A good article called "entity groups for dummies" seems warranted! > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to google-appengine@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group at > >http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: My master/slave to high replication datastore migration experience
Hey Greg and Joshua, Entity groups really aren't that hard, once you play with them and do some testing, I promise ;). First thing to note is that entity groups have effectively nothing to do with the entity's kind. It has to do with the entity's key; or, more precisely, its path. Entity's in the same group have the same path prefix and so are stored "together," and they can be operated on transactionally. The relationship is parent-child in the same sense that a file has a parent directory. You could think of them almost as a sub-database. There are limits, however, to the frequency you can write entities in the same entity group (I think 2 or 3 per second is stated in the docs). Also, because the entity group is stored in the key it can not be changed once the entity has been saved. Hopefully that is not more confusing. :) Robert On Wed, Jul 13, 2011 at 20:41, Greg wrote: > On Jul 13, 1:05 am, Joshua Smith wrote: >> Frankly, I've never completely groked what an entity group maps to in my >> (30+ years of) programming experience. It's certainly not a parent/child >> relationship in the usual sense. > > If you figure it out, let me know! I have a app that needs to update > two distinct entity types at the same time. Because they are distinct, > I can't do both in a transaction. Even if it was only one entity type, > I don't want to lock all of them when writing to one, which I believe > is a consequence of making them a group. > > A good article called "entity groups for dummies" seems warranted! > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: My master/slave to high replication datastore migration experience
I agree with Joshua. That's a very bad experience to force us to go through to migrate. My migration is probably going to be much worse because I have a lot of data, which means that it'll take a very long time (days?) to copy the data across, and by that time the data would have changed on the original app. I cannot bring the app down for extended periods of time while I do this. Not to mention how much of a hassle it is to change app IDs. I posted an issue here, star it if you agree that this process should be automated: http://code.google.com/p/googleappengine/issues/detail?id=5250 On Wed, Jul 13, 2011 at 6:41 PM, Greg wrote: > On Jul 13, 1:05 am, Joshua Smith wrote: > > Frankly, I've never completely groked what an entity group maps to in my > (30+ years of) programming experience. It's certainly not a parent/child > relationship in the usual sense. > > If you figure it out, let me know! I have a app that needs to update > two distinct entity types at the same time. Because they are distinct, > I can't do both in a transaction. Even if it was only one entity type, > I don't want to lock all of them when writing to one, which I believe > is a consequence of making them a group. > > A good article called "entity groups for dummies" seems warranted! > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: My master/slave to high replication datastore migration experience
On Jul 13, 1:05 am, Joshua Smith wrote: > Frankly, I've never completely groked what an entity group maps to in my (30+ > years of) programming experience. It's certainly not a parent/child > relationship in the usual sense. If you figure it out, let me know! I have a app that needs to update two distinct entity types at the same time. Because they are distinct, I can't do both in a transaction. Even if it was only one entity type, I don't want to lock all of them when writing to one, which I believe is a consequence of making them a group. A good article called "entity groups for dummies" seems warranted! -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
