Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Also to understand index creation speed in HR datastore I have tested in same request standard (non-ancestor) datastore query immediately after 4 datastore put operation and I was able to get 3 out of 4 newly stored entites, so currently index creation in HR datastore is pretty fast. These entities don't have much indexes and they aren't big, but this gives us some HR datastore architecture performance indicator. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
M/S: 2011-03-06 23:26:21.462 200 388ms 1690cpu_ms 1430api_cpu_ms 2011-03-06 23:25:06.242 200 155ms 1529cpu_ms 1409api_cpu_ms HR: 2011-03-06 23:26:24.337 200 148ms 1476cpu_ms 1396api_cpu_ms 2011-03-06 23:25:02.859 200 113ms 1456cpu_ms 1396api_cpu_ms Two parallel apps, no cold start, two entities store in same entity group, one task enqueu, few datastore get and memchache hits, a lot of index creations. ;) -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
As far as eventual consistency goes, our users that are on high replication have not seen any issues. We are migrating to HR datastore. I was skeptical about eventual consistency, but when finally we tried HR datastore, we couldn't get it. I know that it is there somewhere, but it is so unnoticeable. My only concern is will it be noticeable when more applications migrate, but it is small price to pay for great availability. Great job. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Here's a question for Google: Is there any chance that sometime in the future we can have a geographically remote datacenter, say in Europe or Asia, that operates on the same datastore? I wouldn't mind a few hundred additional ms of eventual consistency in exchange for a few hundred ms less latency to the client... I guess it would bring up memcache consistency issues. Hmmm. Jeff On Sat, Mar 5, 2011 at 3:30 AM, Matija matija.jerko...@gmail.com wrote: As far as eventual consistency goes, our users that are on high replication have not seen any issues. We are migrating to HR datastore. I was skeptical about eventual consistency, but when finally we tried HR datastore, we couldn't get it. I know that it is there somewhere, but it is so unnoticeable. My only concern is will it be noticeable when more applications migrate, but it is small price to pay for great availability. Great job. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
If you schedule writes it is not a big deal. I have several apps (admittedly not yet ported to GAE) that I only update the data from time to time. Recipe Databases, little league stats. In those cases you just delay the write, no big deal. This doesn't work for everything, but there are definitely cases where M/S makes sense From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of vlad Sent: Saturday, March 05, 2011 2:43 PM To: google-appengine@googlegroups.com Subject: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost? Mission Critical is just PR packaging around a simple fact that Master/Slave has serious performance issues and team is giving up on it. That was my read on the situation and now Ikal is confirming it. Ikal, thank you - it takes balls to admit a failure. This is a dev forum so no need to sugar coat things too much. I too feel like Darien mostly because 3x pricing bump for HR. I am not a all concerned about eventual consistency issues. Just wondering if you take a regular car with 4 wheels (M/S) and add 8 more wheels does it make it better? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
+1 2011/3/4 stevep prosse...@gmail.com Hi Ikai, Don't usually comment on threads going in this direction. But noted that you seem on edge a bit. First wanted to say how important your contributions are. I'm sure it is hard to be in meetings and then try to provide insights to the forum in a manner that keeps Legal happy. Myself, I am very encouraged by HR -- MS did not seem to be panning out. Cheeers, stevep -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- gr, Wim den Ouden Custom Google App Engine http://code.google.com/intl/nl/appengine/ based webapps https://neighborshare.appspot.com/. Free open source neighborshare framework http://code.google.com/p/relat/. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
I'd like to highlight that we're not stubborn enough to admit we were right all along, because we weren't. It's unrealistic to think that someone gets something right the first time, and we're no different. High replication is our admission that the problem isn't that we can't get the average error rate down, the problem is that we can bring it down but we can't eliminate datastore latency spikes. As far as eventual consistency goes, our users that are on high replication have not seen any issues. If we weren't always fussing about transparency, we would probably have been able to not even bring it up, and it's unlikely anyone would have noticed - we just figured it'd make more sense to communicate it, though now we are seeing that users are overreacting a bit. Did you know index creation is also asynchronous? That is, when you save a property, it returns before indexes have been created? Technically, this constitutes a consistency issue, but it has no real world impact because it is instantaneous. We documented eventual consistency issues because they are an order of magnitude slower than asynchronous index creation, so in some small cases of write an entity then do a cross entity group query the query results will be off. If you've run any kind of SQL service at scale that replicates to a read-slave, you should be used to this pattern of persistence. We plan on setting High Replication as the default for all new applications. It's not quite removing it, but perhaps we will explore that direction, though we are current hesitant to do so. We also plan on having some guest blog posts ... soon. If we had the manpower, we would work with each and every one of you to migrate over, then help you benchmark. Unfortunately, we can't. We can, however, try to publish as much data as we have as it comes in. I think the most useful data here will be data from third-parties. As far as our tests go, everything seems alright, but I'm pretty sure you'll all have more confidence in the results if they're being written by users whose entire businesses depend on the reliability and performance of App Engine. Spines: Please link me to where the documentation says that master/slave is the best option so we can fix it. To answer your question: yes, High Replication mitigates the impact of datastore spikes, and provides a path for us to pretty much eliminate them altogether. Lastly, I want to apologize if I came off as irate. I'm usually quick to respond emotionally, and the damn undo send button went away too quickly in my gmail. I don't mean to shut down any kind of joking around, but do remember that this group is internationally read. Sarcasm doesn't come off well in email, and it comes off 10x more confusing for non-native English speakers. If there's any chance any of you guys are coming to Pycon in Atlanta next week, Wesley and I will be there. Let's smooth things over with some drinks/food =). Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Thu, Mar 3, 2011 at 6:37 PM, Darien Caldwell darien.caldw...@gmail.comwrote: I'm kind of disappointed with Google's recent attitude change. When master/slave was all there was, it was deemed 'production worthy'. Now suddenly it's If you are running a production service, you should not use Master/Slave datastore. Ever. Given that the HR datastore has some pretty big problems from a user perspective (costs 3 times more, only supports Eventual consistency), it's not really feasable to use for production code in *every* instance. So I'm very disappointed to hear these kinds of comments coming from Google. You shouldn't make something, then complain when people use it. If you don't want people using Master/Slave, maybe you should remove it. To berate customers for preferring an option you *provide* is just mind blowing. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
+1 On Fri, Mar 4, 2011 at 1:48 PM, Jeff Schwartz jefftschwa...@gmail.comwrote: I can't remember who said it but it went something like, The greatest failure is to not see the opportunities that come from failing! Oh, wait, I said it he he he. Ikai, you and all the other Googlers have nothing to feel defensive about. HR is obviously the chosen path and to ignore that is, well, foolish. Perhaps we don't say it often enough and you guys and gals (Googlers) don't get to hear it often enough and maybe we should say it more often but we are grateful for all the great technologies that you are enabling and allowing us to participate in and personally I just want to say thank you and tell you all how grateful I am. Jeff On Fri, Mar 4, 2011 at 2:06 PM, Ikai Lan (Google) ika...@google.comwrote: I'd like to highlight that we're not stubborn enough to admit we were right all along, because we weren't. It's unrealistic to think that someone gets something right the first time, and we're no different. High replication is our admission that the problem isn't that we can't get the average error rate down, the problem is that we can bring it down but we can't eliminate datastore latency spikes. As far as eventual consistency goes, our users that are on high replication have not seen any issues. If we weren't always fussing about transparency, we would probably have been able to not even bring it up, and it's unlikely anyone would have noticed - we just figured it'd make more sense to communicate it, though now we are seeing that users are overreacting a bit. Did you know index creation is also asynchronous? That is, when you save a property, it returns before indexes have been created? Technically, this constitutes a consistency issue, but it has no real world impact because it is instantaneous. We documented eventual consistency issues because they are an order of magnitude slower than asynchronous index creation, so in some small cases of write an entity then do a cross entity group query the query results will be off. If you've run any kind of SQL service at scale that replicates to a read-slave, you should be used to this pattern of persistence. We plan on setting High Replication as the default for all new applications. It's not quite removing it, but perhaps we will explore that direction, though we are current hesitant to do so. We also plan on having some guest blog posts ... soon. If we had the manpower, we would work with each and every one of you to migrate over, then help you benchmark. Unfortunately, we can't. We can, however, try to publish as much data as we have as it comes in. I think the most useful data here will be data from third-parties. As far as our tests go, everything seems alright, but I'm pretty sure you'll all have more confidence in the results if they're being written by users whose entire businesses depend on the reliability and performance of App Engine. Spines: Please link me to where the documentation says that master/slave is the best option so we can fix it. To answer your question: yes, High Replication mitigates the impact of datastore spikes, and provides a path for us to pretty much eliminate them altogether. Lastly, I want to apologize if I came off as irate. I'm usually quick to respond emotionally, and the damn undo send button went away too quickly in my gmail. I don't mean to shut down any kind of joking around, but do remember that this group is internationally read. Sarcasm doesn't come off well in email, and it comes off 10x more confusing for non-native English speakers. If there's any chance any of you guys are coming to Pycon in Atlanta next week, Wesley and I will be there. Let's smooth things over with some drinks/food =). Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Thu, Mar 3, 2011 at 6:37 PM, Darien Caldwell darien.caldw...@gmail.com wrote: I'm kind of disappointed with Google's recent attitude change. When master/slave was all there was, it was deemed 'production worthy'. Now suddenly it's If you are running a production service, you should not use Master/Slave datastore. Ever. Given that the HR datastore has some pretty big problems from a user perspective (costs 3 times more, only supports Eventual consistency), it's not really feasable to use for production code in *every* instance. So I'm very disappointed to hear these kinds of comments coming from Google. You shouldn't make something, then complain when people use it. If you don't want people using Master/Slave, maybe you should remove it. To berate customers for preferring an option you *provide* is just mind blowing. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
On Fri, Mar 4, 2011 at 1:07 PM, Darien Caldwell darien.caldw...@gmail.com wrote: Eventual Consistency is really the #1 reason I'm reluctant to make any change to HR. I sell virtual goods, people pay real money for things that exist in the datastore. If they buy something, and it doesn't show up for minute/hours, I get complaints. Your reply seems to indicate that maybe Eventual Consistency isn't quite that inconsistent, but I haven't seen any data to express just how long things are out of sync, so I have to expect the worst. If you have that kind of reliability requirement, you should be using transactions - which provide consistency guarantees to boot. If you aren't using transactions, eventual consistency is likely far less of a problem than the outright failures that the M/S datastore provides. Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Why are you being charged for idle instances? You are only charged for CPU consumed by API calls. Don't make facetious comments about the SLA. Even if we never end up supporting one (this is unlikely), we will be focusing on reliability from the perspective of High Replication datastore, not Master/Slave. If you are running a production service, you should not use Master/Slave datastore. Ever. Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Wed, Mar 2, 2011 at 9:15 AM, Stephen Johnson onepagewo...@gmail.comwrote: Awesome input! Thanks Simon. I've updated my list. Not looking good for HR. PROs --- 1. No downtime during maintenance. 2. Latency/Spikes suppossedly decreased.?? 3. Faster gets/queries.?? 4. If Google ever does support an SLA it will only be HR (Ikai's post). But that might be 2020 or later :-) CONs -- 1. Cost: $.45/GB (seems expensive - is it worth it?) 2. Increased CPU (Testing is still early but it seems CPU cost is increased?? Is this just for puts with writing to multiple datacenters??) 3. Instances last longer in idle state causing a CPU charge for next request while idle. (This has never been addressed or acknowledged by Google but many have complained about it.) This may be what is causing #2 above. (See excellent comments by Simon Knott). 4. Increased latency time for Memcache requests perhaps 2 to 10 times longer?? 5. Increased loading time for loading requests. Almost double?? On Wed, Mar 2, 2011 at 10:04 AM, Simon Knott knott.si...@gmail.comwrote: HR instances definitely seem to last longer than M/S ones - I've had instances last for days on my HR app, which is only in development. However, that's highlighted a strange problem in that you seem to be charged for idle instances. The very rough idle costs are: 1hr idle = additional 2000 CPU_ms for next request 2hr idle = additional 4000 CPU_ms for next request 4hr idle = additional 8000 CPU_ms for next request ... linear increase to: 24hr+ idle = additional 45000 CPU_ms for next request This probably isn't a major problem for apps which are busy, but it should be taken into account for small apps I guess! Another thing I've noticed is that HR apps have additional latency for MemCache API calls in the region of 2-10 times longer. I've tested my app in both M/S and HR and the HR version consistently has MemCache API Gets taking 20ms on average, for tiny amounts of data, which only take 4ms on the M/S version. Loading requests seem quite long on HR instances as well, almost double that of the M/S. *NB. *It should be noted that all of my experience is based on a single app which is still in development, so all of my observations may not ring true for real-life, high-throughput apps. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
* and if you have a production service running Master/Slave datastore, you should seriously consider migrating to HR datastore. If you have large amounts of data and this is unfeasible, please let us know via a billing ticket so we can figure out a plan for moving you. Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Thu, Mar 3, 2011 at 11:19 AM, Ikai Lan (Google) ika...@google.comwrote: Why are you being charged for idle instances? You are only charged for CPU consumed by API calls. Don't make facetious comments about the SLA. Even if we never end up supporting one (this is unlikely), we will be focusing on reliability from the perspective of High Replication datastore, not Master/Slave. If you are running a production service, you should not use Master/Slave datastore. Ever. Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Wed, Mar 2, 2011 at 9:15 AM, Stephen Johnson onepagewo...@gmail.comwrote: Awesome input! Thanks Simon. I've updated my list. Not looking good for HR. PROs --- 1. No downtime during maintenance. 2. Latency/Spikes suppossedly decreased.?? 3. Faster gets/queries.?? 4. If Google ever does support an SLA it will only be HR (Ikai's post). But that might be 2020 or later :-) CONs -- 1. Cost: $.45/GB (seems expensive - is it worth it?) 2. Increased CPU (Testing is still early but it seems CPU cost is increased?? Is this just for puts with writing to multiple datacenters??) 3. Instances last longer in idle state causing a CPU charge for next request while idle. (This has never been addressed or acknowledged by Google but many have complained about it.) This may be what is causing #2 above. (See excellent comments by Simon Knott). 4. Increased latency time for Memcache requests perhaps 2 to 10 times longer?? 5. Increased loading time for loading requests. Almost double?? On Wed, Mar 2, 2011 at 10:04 AM, Simon Knott knott.si...@gmail.comwrote: HR instances definitely seem to last longer than M/S ones - I've had instances last for days on my HR app, which is only in development. However, that's highlighted a strange problem in that you seem to be charged for idle instances. The very rough idle costs are: 1hr idle = additional 2000 CPU_ms for next request 2hr idle = additional 4000 CPU_ms for next request 4hr idle = additional 8000 CPU_ms for next request ... linear increase to: 24hr+ idle = additional 45000 CPU_ms for next request This probably isn't a major problem for apps which are busy, but it should be taken into account for small apps I guess! Another thing I've noticed is that HR apps have additional latency for MemCache API calls in the region of 2-10 times longer. I've tested my app in both M/S and HR and the HR version consistently has MemCache API Gets taking 20ms on average, for tiny amounts of data, which only take 4ms on the M/S version. Loading requests seem quite long on HR instances as well, almost double that of the M/S. *NB. *It should be noted that all of my experience is based on a single app which is still in development, so all of my observations may not ring true for real-life, high-throughput apps. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
The comments about charges for idle instances are based around the fact that if you leave an instance unused for any amount of time, the next request to that instance will have an elevated CPU_ms cost - you can request a static resource on an instance which has been left idle for 24hrs and the cost of that resource will be +4 CPU_ms. Don Schwarz was looking into it from your side around a month ago, although as I clarified earlier the actual daily cost amounts to virtually nothing in the grand scheme of things if you have relatively few instances allocated. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Don't make facetious comments about the SLA. -- Who are you telling me what I can make light of or not. I still live in America. Do you? On Thu, Mar 3, 2011 at 12:38 PM, Simon Knott knott.si...@gmail.com wrote: The comments about charges for idle instances are based around the fact that if you leave an instance unused for any amount of time, the next request to that instance will have an elevated CPU_ms cost - you can request a static resource on an instance which has been left idle for 24hrs and the cost of that resource will be +4 CPU_ms. Don Schwarz was looking into it from your side around a month ago, although as I clarified earlier the actual daily cost amounts to virtually nothing in the grand scheme of things if you have relatively few instances allocated. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
lol On Thu, Mar 3, 2011 at 2:29 PM, Stephen Johnson onepagewo...@gmail.comwrote: Don't make facetious comments about the SLA. -- Who are you telling me what I can make light of or not. I still live in America. Do you? On Thu, Mar 3, 2011 at 12:38 PM, Simon Knott knott.si...@gmail.comwrote: The comments about charges for idle instances are based around the fact that if you leave an instance unused for any amount of time, the next request to that instance will have an elevated CPU_ms cost - you can request a static resource on an instance which has been left idle for 24hrs and the cost of that resource will be +4 CPU_ms. Don Schwarz was looking into it from your side around a month ago, although as I clarified earlier the actual daily cost amounts to virtually nothing in the grand scheme of things if you have relatively few instances allocated. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Look, I'm not telling you you're breaking the law or that I'm going to do anything about it. We will figure out how to support an SLA - that's a fact. And it'll happen before 2020. I haven't violated your constitutional rights, but I am going to prevent people from spreading FUD. Anyway, this thread is getting extremely off topic, so let's stick to the original topic of issues with HR datastore. Don't look at it purely from an SLA perspective. You should never be considering master/slave for new projects because we will be focusing our resources on making High Replication better. Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Thu, Mar 3, 2011 at 1:29 PM, Stephen Johnson onepagewo...@gmail.comwrote: Don't make facetious comments about the SLA. -- Who are you telling me what I can make light of or not. I still live in America. Do you? On Thu, Mar 3, 2011 at 12:38 PM, Simon Knott knott.si...@gmail.comwrote: The comments about charges for idle instances are based around the fact that if you leave an instance unused for any amount of time, the next request to that instance will have an elevated CPU_ms cost - you can request a static resource on an instance which has been left idle for 24hrs and the cost of that resource will be +4 CPU_ms. Don Schwarz was looking into it from your side around a month ago, although as I clarified earlier the actual daily cost amounts to virtually nothing in the grand scheme of things if you have relatively few instances allocated. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Front page of appengine.google.com This is a preview release of Google App Engine Google App Engine BETA the beta wasn't on there for no good reason. Roll with the punches understand there will be hick-ups. Weren't you around for the Microsoft days to learn Software Development 101? Don't bet the farm on anything that isn't labeled For Enterprise and has a version number less than 3.1. Let others fail miserably to get the bugs out. Well Google is the same beasty, accept the old wait for the first service pack doesn't work because when they upgrade, you go with them like it or not. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Darien Caldwell Sent: Thursday, March 03, 2011 6:38 PM To: Google App Engine Subject: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost? I'm kind of disappointed with Google's recent attitude change. When master/slave was all there was, it was deemed 'production worthy'. Now suddenly it's If you are running a production service, you should not use Master/Slave datastore. Ever. Given that the HR datastore has some pretty big problems from a user perspective (costs 3 times more, only supports Eventual consistency), it's not really feasable to use for production code in *every* instance. So I'm very disappointed to hear these kinds of comments coming from Google. You shouldn't make something, then complain when people use it. If you don't want people using Master/Slave, maybe you should remove it. To berate customers for preferring an option you *provide* is just mind blowing. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Would I be wrong then to assume that session requests are likewise impacted because like memcache sessions are also backed by the datastore? On Wed, Mar 2, 2011 at 12:04 PM, Simon Knott knott.si...@gmail.com wrote: HR instances definitely seem to last longer than M/S ones - I've had instances last for days on my HR app, which is only in development. However, that's highlighted a strange problem in that you seem to be charged for idle instances. The very rough idle costs are: 1hr idle = additional 2000 CPU_ms for next request 2hr idle = additional 4000 CPU_ms for next request 4hr idle = additional 8000 CPU_ms for next request ... linear increase to: 24hr+ idle = additional 45000 CPU_ms for next request This probably isn't a major problem for apps which are busy, but it should be taken into account for small apps I guess! Another thing I've noticed is that HR apps have additional latency for MemCache API calls in the region of 2-10 times longer. I've tested my app in both M/S and HR and the HR version consistently has MemCache API Gets taking 20ms on average, for tiny amounts of data, which only take 4ms on the M/S version. Loading requests seem quite long on HR instances as well, almost double that of the M/S. *NB. *It should be noted that all of my experience is based on a single app which is still in development, so all of my observations may not ring true for real-life, high-throughput apps. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- *Jeff Schwartz* http://jefftschwartz.appspot.com/ http://www.linkedin.com/in/jefftschwartz follow me on twitter: @jefftschwartz -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Awesome input! Thanks Simon. I've updated my list. Not looking good for HR. PROs --- 1. No downtime during maintenance. 2. Latency/Spikes suppossedly decreased.?? 3. Faster gets/queries.?? 4. If Google ever does support an SLA it will only be HR (Ikai's post). But that might be 2020 or later :-) CONs -- 1. Cost: $.45/GB (seems expensive - is it worth it?) 2. Increased CPU (Testing is still early but it seems CPU cost is increased?? Is this just for puts with writing to multiple datacenters??) 3. Instances last longer in idle state causing a CPU charge for next request while idle. (This has never been addressed or acknowledged by Google but many have complained about it.) This may be what is causing #2 above. (See excellent comments by Simon Knott). 4. Increased latency time for Memcache requests perhaps 2 to 10 times longer?? 5. Increased loading time for loading requests. Almost double?? On Wed, Mar 2, 2011 at 10:04 AM, Simon Knott knott.si...@gmail.com wrote: HR instances definitely seem to last longer than M/S ones - I've had instances last for days on my HR app, which is only in development. However, that's highlighted a strange problem in that you seem to be charged for idle instances. The very rough idle costs are: 1hr idle = additional 2000 CPU_ms for next request 2hr idle = additional 4000 CPU_ms for next request 4hr idle = additional 8000 CPU_ms for next request ... linear increase to: 24hr+ idle = additional 45000 CPU_ms for next request This probably isn't a major problem for apps which are busy, but it should be taken into account for small apps I guess! Another thing I've noticed is that HR apps have additional latency for MemCache API calls in the region of 2-10 times longer. I've tested my app in both M/S and HR and the HR version consistently has MemCache API Gets taking 20ms on average, for tiny amounts of data, which only take 4ms on the M/S version. Loading requests seem quite long on HR instances as well, almost double that of the M/S. *NB. *It should be noted that all of my experience is based on a single app which is still in development, so all of my observations may not ring true for real-life, high-throughput apps. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
I've was thinking about the charging of idle instances on my drive home and have realised that it really doesn't add up to very much - basically, you end up having ~8mins of idle CPU cost for an entire day, if you have 10 instances for your app, assuming that the idle CPU_ms cost doesn't fluctuate and stays on that linear scale. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Where are you getting the $0.45/GB? Looking in my billing settings, this is only $0.015/GB for HR storage. I'd also say that whilst the loading times appear to be increased, there are much, much less of them currently so I think that's a very minor con. My M/S version of the app appears to have a loading request every 5-10 minutes or so, whereas I've had 7 loading requests since Saturday on my HR app. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Regardless of the cost, it's not documented anywhere or explained why a charge is occurring. It seems rather unfair (and well there are other words for it) to charge users for Always On feature yet still bill them CPU. I would think that paying $0.30 per day for Always On would cover the idle CPU costs but then still try to under the radar charge people for the CPU as well and then when people bring it up in the discussion boards to just ignore it. Well, seems fishy. On Wed, Mar 2, 2011 at 1:31 PM, Simon Knott knott.si...@gmail.com wrote: I've was thinking about the charging of idle instances on my drive home and have realised that it really doesn't add up to very much - basically, you end up having ~8mins of idle CPU cost for an entire day, if you have 10 instances for your app, assuming that the idle CPU_ms cost doesn't fluctuate and stays on that linear scale. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Ah yes, forgot that was a per-day charge - thanks for the clarification. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
