Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Also to understand index creation speed in HR datastore I have tested in same request standard (non-ancestor) datastore query immediately after 4 datastore put operation and I was able to get 3 out of 4 newly stored entites, so currently index creation in HR datastore is pretty fast. These entities don't have much indexes and they aren't big, but this gives us some HR datastore architecture performance indicator. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
In fact performances are much closer than I first thought : MS : 1189ms 1421cpu_ms 651api_cpu_ms HR : 1230ms 1631cpu_ms 756api_cpu_ms I might still have a few configuration differences to check. François -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
http://code.google.com/intl/en/appengine/docs/python/datastore/hr/#Comparing_the_Data_Storage_Options You can select one of two data storage options when you create your application: The Master/Slave Datastore The Master/Slave Datastore is the default for new applications. It uses a master-slave replication system, which asynchronously replicates data as you write it to another data center. Since only one datacenter is the master for writing at any given time, this option offers strong consistency for all reads and queries, but your data may be temporarily unavailable during data center issues or planned downtime. This option also offers the lowest storage and CPU costs for storing data. The High Replication Datastore The High Replication Datastore is a new, highly available, highly reliable storage solution. It remains available for reads and writes during planned downtime and is extremely resilient in the face of catastrophic failure—but it costs more than the Master/Slave option. __ - Due to the higher cost, we recommend High Replication Datastore primarily for those developers building mission critical applications on App Engine. On 4 mar, 20:06, Ikai Lan (Google) ika...@google.com wrote: I'd like to highlight that we're not stubborn enough to admit we were right all along, because we weren't. It's unrealistic to think that someone gets something right the first time, and we're no different. High replication is our admission that the problem isn't that we can't get the average error rate down, the problem is that we can bring it down but we can't eliminate datastore latency spikes. As far as eventual consistency goes, our users that are on high replication have not seen any issues. If we weren't always fussing about transparency, we would probably have been able to not even bring it up, and it's unlikely anyone would have noticed - we just figured it'd make more sense to communicate it, though now we are seeing that users are overreacting a bit. Did you know index creation is also asynchronous? That is, when you save a property, it returns before indexes have been created? Technically, this constitutes a consistency issue, but it has no real world impact because it is instantaneous. We documented eventual consistency issues because they are an order of magnitude slower than asynchronous index creation, so in some small cases of write an entity then do a cross entity group query the query results will be off. If you've run any kind of SQL service at scale that replicates to a read-slave, you should be used to this pattern of persistence. We plan on setting High Replication as the default for all new applications. It's not quite removing it, but perhaps we will explore that direction, though we are current hesitant to do so. We also plan on having some guest blog posts ... soon. If we had the manpower, we would work with each and every one of you to migrate over, then help you benchmark. Unfortunately, we can't. We can, however, try to publish as much data as we have as it comes in. I think the most useful data here will be data from third-parties. As far as our tests go, everything seems alright, but I'm pretty sure you'll all have more confidence in the results if they're being written by users whose entire businesses depend on the reliability and performance of App Engine. Spines: Please link me to where the documentation says that master/slave is the best option so we can fix it. To answer your question: yes, High Replication mitigates the impact of datastore spikes, and provides a path for us to pretty much eliminate them altogether. Lastly, I want to apologize if I came off as irate. I'm usually quick to respond emotionally, and the damn undo send button went away too quickly in my gmail. I don't mean to shut down any kind of joking around, but do remember that this group is internationally read. Sarcasm doesn't come off well in email, and it comes off 10x more confusing for non-native English speakers. If there's any chance any of you guys are coming to Pycon in Atlanta next week, Wesley and I will be there. Let's smooth things over with some drinks/food =). Ikai Lan Developer Programs Engineer, Google App Engine Blog:http://googleappengine.blogspot.com Twitter:http://twitter.com/app_engine Reddit:http://www.reddit.com/r/appengine On Thu, Mar 3, 2011 at 6:37 PM, Darien Caldwell darien.caldw...@gmail.comwrote: I'm kind of disappointed with Google's recent attitude change. When master/slave was all there was, it was deemed 'production worthy'. Now suddenly it's If you are running a production service, you should not use Master/Slave datastore. Ever. Given that the HR datastore has some pretty big problems from a user perspective (costs 3 times more, only supports Eventual consistency), it's not really feasable to use for production code in *every*
[google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Hi everybody, It looks like HR apps takes more CPU than MS ones : HR : 2011-03-06 23:33:06.869 /?l=fr 200 1860ms 2304cpu_ms 437api_cpu_ms MS : 2011-03-06 23:33:51.170 /?l=fr 200 689ms 1165cpu_ms 302api_cpu_ms Is it possible to have such differences between apps for the same page loading the same data ? On an average basis, the HR apps feels slower than the MS one with exactly the same data. Thanx for your help. Francois -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
M/S: 2011-03-06 23:26:21.462 200 388ms 1690cpu_ms 1430api_cpu_ms 2011-03-06 23:25:06.242 200 155ms 1529cpu_ms 1409api_cpu_ms HR: 2011-03-06 23:26:24.337 200 148ms 1476cpu_ms 1396api_cpu_ms 2011-03-06 23:25:02.859 200 113ms 1456cpu_ms 1396api_cpu_ms Two parallel apps, no cold start, two entities store in same entity group, one task enqueu, few datastore get and memchache hits, a lot of index creations. ;) -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
As far as eventual consistency goes, our users that are on high replication have not seen any issues. We are migrating to HR datastore. I was skeptical about eventual consistency, but when finally we tried HR datastore, we couldn't get it. I know that it is there somewhere, but it is so unnoticeable. My only concern is will it be noticeable when more applications migrate, but it is small price to pay for great availability. Great job. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Here's a question for Google: Is there any chance that sometime in the future we can have a geographically remote datacenter, say in Europe or Asia, that operates on the same datastore? I wouldn't mind a few hundred additional ms of eventual consistency in exchange for a few hundred ms less latency to the client... I guess it would bring up memcache consistency issues. Hmmm. Jeff On Sat, Mar 5, 2011 at 3:30 AM, Matija matija.jerko...@gmail.com wrote: As far as eventual consistency goes, our users that are on high replication have not seen any issues. We are migrating to HR datastore. I was skeptical about eventual consistency, but when finally we tried HR datastore, we couldn't get it. I know that it is there somewhere, but it is so unnoticeable. My only concern is will it be noticeable when more applications migrate, but it is small price to pay for great availability. Great job. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Mission Critical is just PR packaging around a simple fact that Master/Slave has serious performance issues and team is giving up on it. That was my read on the situation and now Ikal is confirming it. Ikal, thank you - it takes balls to admit a failure. This is a dev forum so no need to sugar coat things too much. I too feel like Darien mostly because 3x pricing bump for HR. I am not a all concerned about eventual consistency issues. Just wondering if you take a regular car with 4 wheels (M/S) and add 8 more wheels does it make it better? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
If you schedule writes it is not a big deal. I have several apps (admittedly not yet ported to GAE) that I only update the data from time to time. Recipe Databases, little league stats. In those cases you just delay the write, no big deal. This doesn't work for everything, but there are definitely cases where M/S makes sense From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of vlad Sent: Saturday, March 05, 2011 2:43 PM To: google-appengine@googlegroups.com Subject: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost? Mission Critical is just PR packaging around a simple fact that Master/Slave has serious performance issues and team is giving up on it. That was my read on the situation and now Ikal is confirming it. Ikal, thank you - it takes balls to admit a failure. This is a dev forum so no need to sugar coat things too much. I too feel like Darien mostly because 3x pricing bump for HR. I am not a all concerned about eventual consistency issues. Just wondering if you take a regular car with 4 wheels (M/S) and add 8 more wheels does it make it better? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Hi Ikai, Don't usually comment on threads going in this direction. But noted that you seem on edge a bit. First wanted to say how important your contributions are. I'm sure it is hard to be in meetings and then try to provide insights to the forum in a manner that keeps Legal happy. Myself, I am very encouraged by HR -- MS did not seem to be panning out. Cheeers, stevep -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
+1 2011/3/4 stevep prosse...@gmail.com Hi Ikai, Don't usually comment on threads going in this direction. But noted that you seem on edge a bit. First wanted to say how important your contributions are. I'm sure it is hard to be in meetings and then try to provide insights to the forum in a manner that keeps Legal happy. Myself, I am very encouraged by HR -- MS did not seem to be panning out. Cheeers, stevep -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- gr, Wim den Ouden Custom Google App Engine http://code.google.com/intl/nl/appengine/ based webapps https://neighborshare.appspot.com/. Free open source neighborshare framework http://code.google.com/p/relat/. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
I'd like to highlight that we're not stubborn enough to admit we were right all along, because we weren't. It's unrealistic to think that someone gets something right the first time, and we're no different. High replication is our admission that the problem isn't that we can't get the average error rate down, the problem is that we can bring it down but we can't eliminate datastore latency spikes. As far as eventual consistency goes, our users that are on high replication have not seen any issues. If we weren't always fussing about transparency, we would probably have been able to not even bring it up, and it's unlikely anyone would have noticed - we just figured it'd make more sense to communicate it, though now we are seeing that users are overreacting a bit. Did you know index creation is also asynchronous? That is, when you save a property, it returns before indexes have been created? Technically, this constitutes a consistency issue, but it has no real world impact because it is instantaneous. We documented eventual consistency issues because they are an order of magnitude slower than asynchronous index creation, so in some small cases of write an entity then do a cross entity group query the query results will be off. If you've run any kind of SQL service at scale that replicates to a read-slave, you should be used to this pattern of persistence. We plan on setting High Replication as the default for all new applications. It's not quite removing it, but perhaps we will explore that direction, though we are current hesitant to do so. We also plan on having some guest blog posts ... soon. If we had the manpower, we would work with each and every one of you to migrate over, then help you benchmark. Unfortunately, we can't. We can, however, try to publish as much data as we have as it comes in. I think the most useful data here will be data from third-parties. As far as our tests go, everything seems alright, but I'm pretty sure you'll all have more confidence in the results if they're being written by users whose entire businesses depend on the reliability and performance of App Engine. Spines: Please link me to where the documentation says that master/slave is the best option so we can fix it. To answer your question: yes, High Replication mitigates the impact of datastore spikes, and provides a path for us to pretty much eliminate them altogether. Lastly, I want to apologize if I came off as irate. I'm usually quick to respond emotionally, and the damn undo send button went away too quickly in my gmail. I don't mean to shut down any kind of joking around, but do remember that this group is internationally read. Sarcasm doesn't come off well in email, and it comes off 10x more confusing for non-native English speakers. If there's any chance any of you guys are coming to Pycon in Atlanta next week, Wesley and I will be there. Let's smooth things over with some drinks/food =). Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Thu, Mar 3, 2011 at 6:37 PM, Darien Caldwell darien.caldw...@gmail.comwrote: I'm kind of disappointed with Google's recent attitude change. When master/slave was all there was, it was deemed 'production worthy'. Now suddenly it's If you are running a production service, you should not use Master/Slave datastore. Ever. Given that the HR datastore has some pretty big problems from a user perspective (costs 3 times more, only supports Eventual consistency), it's not really feasable to use for production code in *every* instance. So I'm very disappointed to hear these kinds of comments coming from Google. You shouldn't make something, then complain when people use it. If you don't want people using Master/Slave, maybe you should remove it. To berate customers for preferring an option you *provide* is just mind blowing. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
+1 On Fri, Mar 4, 2011 at 1:48 PM, Jeff Schwartz jefftschwa...@gmail.comwrote: I can't remember who said it but it went something like, The greatest failure is to not see the opportunities that come from failing! Oh, wait, I said it he he he. Ikai, you and all the other Googlers have nothing to feel defensive about. HR is obviously the chosen path and to ignore that is, well, foolish. Perhaps we don't say it often enough and you guys and gals (Googlers) don't get to hear it often enough and maybe we should say it more often but we are grateful for all the great technologies that you are enabling and allowing us to participate in and personally I just want to say thank you and tell you all how grateful I am. Jeff On Fri, Mar 4, 2011 at 2:06 PM, Ikai Lan (Google) ika...@google.comwrote: I'd like to highlight that we're not stubborn enough to admit we were right all along, because we weren't. It's unrealistic to think that someone gets something right the first time, and we're no different. High replication is our admission that the problem isn't that we can't get the average error rate down, the problem is that we can bring it down but we can't eliminate datastore latency spikes. As far as eventual consistency goes, our users that are on high replication have not seen any issues. If we weren't always fussing about transparency, we would probably have been able to not even bring it up, and it's unlikely anyone would have noticed - we just figured it'd make more sense to communicate it, though now we are seeing that users are overreacting a bit. Did you know index creation is also asynchronous? That is, when you save a property, it returns before indexes have been created? Technically, this constitutes a consistency issue, but it has no real world impact because it is instantaneous. We documented eventual consistency issues because they are an order of magnitude slower than asynchronous index creation, so in some small cases of write an entity then do a cross entity group query the query results will be off. If you've run any kind of SQL service at scale that replicates to a read-slave, you should be used to this pattern of persistence. We plan on setting High Replication as the default for all new applications. It's not quite removing it, but perhaps we will explore that direction, though we are current hesitant to do so. We also plan on having some guest blog posts ... soon. If we had the manpower, we would work with each and every one of you to migrate over, then help you benchmark. Unfortunately, we can't. We can, however, try to publish as much data as we have as it comes in. I think the most useful data here will be data from third-parties. As far as our tests go, everything seems alright, but I'm pretty sure you'll all have more confidence in the results if they're being written by users whose entire businesses depend on the reliability and performance of App Engine. Spines: Please link me to where the documentation says that master/slave is the best option so we can fix it. To answer your question: yes, High Replication mitigates the impact of datastore spikes, and provides a path for us to pretty much eliminate them altogether. Lastly, I want to apologize if I came off as irate. I'm usually quick to respond emotionally, and the damn undo send button went away too quickly in my gmail. I don't mean to shut down any kind of joking around, but do remember that this group is internationally read. Sarcasm doesn't come off well in email, and it comes off 10x more confusing for non-native English speakers. If there's any chance any of you guys are coming to Pycon in Atlanta next week, Wesley and I will be there. Let's smooth things over with some drinks/food =). Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Thu, Mar 3, 2011 at 6:37 PM, Darien Caldwell darien.caldw...@gmail.com wrote: I'm kind of disappointed with Google's recent attitude change. When master/slave was all there was, it was deemed 'production worthy'. Now suddenly it's If you are running a production service, you should not use Master/Slave datastore. Ever. Given that the HR datastore has some pretty big problems from a user perspective (costs 3 times more, only supports Eventual consistency), it's not really feasable to use for production code in *every* instance. So I'm very disappointed to hear these kinds of comments coming from Google. You shouldn't make something, then complain when people use it. If you don't want people using Master/Slave, maybe you should remove it. To berate customers for preferring an option you *provide* is just mind blowing. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to
[google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
On Mar 4, 3:57 pm, Philippe Beaudoin philippe.beaud...@gmail.com wrote: Forums posts could really use a thumb up feature to give a voice to the great silent minority. Switch back to the old google groups interface, and you can give them 5 stars. Which no one will see when they get rid of the old interface. :) -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
First of all, I just wanted to echo Jeff Schwartz's comment. Ikai, I think your response was very well thought out, for a thread which was close to going off-course. You nipped it right in the bud, while being clear about the reasons for careful dialogue. I appreciate that. I've been following different threads and notes and documentation on HR Datastore. From these, I had also come to the conclusion that the eventual consistency issue was minimal because - The write blocks while a replication to all the other sites is done in parallel - However, other requests that go to the sites might see the old or new data. - However, the stale data only exists until the write returns. - The edge case is if there is a network disconnect with one of the other sites during the replication. Then it may have to catch up later. If this is correct, then in the general scenario, all the sites are consistent when a write returns. Ikai, can you please confirm if this is more or less how it works. If so, then it may help alleviate people's concerns with the HR Datastore. Personally, my only concern with HR Datastore is that it is much more expensive than the MS, and also significantly more expensive than Amazon SimpleDB which promises the same features (HA, replication to multiple sites, etc) but at $0.25 per GB (while app engine charges $0.45 per GB for now). I understand that's a preliminary price till June/July when Google figures out what to charge, but I (and I reckon some other developers) would be more eager to jump on the HR Datastore option if the price difference wasn't so much. P.S. GAE was always cheaper than AWS and provided a richer development environment which made the decision easier for some of us. Just a thought. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
+ 1 to Ikai for making up and been so open. and some thoughts : 1) Before making HR de facto standard, a simple (just press a button and wait - no need foor alias) migration path should be devised for legacy MS appls. 2) Pricing - although new price can be acceptable for frequent used data, it becomes prohibitive for some use cases that need to store large volume of rarely used data for which access latency is not critical. So ... May be the solution could be MS and HR available side by side to all appls, though I am not sure if this is technically feasible. Happy coding;-) On Mar 4, 11:26 pm, Ugorji ugo...@gmail.com wrote: First of all, I just wanted to echo Jeff Schwartz's comment. Ikai, I think your response was very well thought out, for a thread which was close to going off-course. You nipped it right in the bud, while being clear about the reasons for careful dialogue. I appreciate that. I've been following different threads and notes and documentation on HR Datastore. From these, I had also come to the conclusion that the eventual consistency issue was minimal because - The write blocks while a replication to all the other sites is done in parallel - However, other requests that go to the sites might see the old or new data. - However, the stale data only exists until the write returns. - The edge case is if there is a network disconnect with one of the other sites during the replication. Then it may have to catch up later. If this is correct, then in the general scenario, all the sites are consistent when a write returns. Ikai, can you please confirm if this is more or less how it works. If so, then it may help alleviate people's concerns with the HR Datastore. Personally, my only concern with HR Datastore is that it is much more expensive than the MS, and also significantly more expensive than Amazon SimpleDB which promises the same features (HA, replication to multiple sites, etc) but at $0.25 per GB (while app engine charges $0.45 per GB for now). I understand that's a preliminary price till June/July when Google figures out what to charge, but I (and I reckon some other developers) would be more eager to jump on the HR Datastore option if the price difference wasn't so much. P.S. GAE was always cheaper than AWS and provided a richer development environment which made the decision easier for some of us. Just a thought. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
On Fri, Mar 4, 2011 at 1:07 PM, Darien Caldwell darien.caldw...@gmail.com wrote: Eventual Consistency is really the #1 reason I'm reluctant to make any change to HR. I sell virtual goods, people pay real money for things that exist in the datastore. If they buy something, and it doesn't show up for minute/hours, I get complaints. Your reply seems to indicate that maybe Eventual Consistency isn't quite that inconsistent, but I haven't seen any data to express just how long things are out of sync, so I have to expect the worst. If you have that kind of reliability requirement, you should be using transactions - which provide consistency guarantees to boot. If you aren't using transactions, eventual consistency is likely far less of a problem than the outright failures that the M/S datastore provides. Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
On Mar 4, 12:48 pm, Jeff Schwartz jefftschwa...@gmail.com wrote: HR is obviously the chosen path and to ignore that is, well, foolish. I think this is part of the problem too. Until today, I was never aware there was a path to choose. Master/Slave always there, and worked fine (at least from my perspective). HR was introduced as an alternative, for people with Mission Critical needs. That is how it was introduced, as an alternative, not as a replacement. Then suddenly it's Master slave = bad out of left field. It's understandable people may be shocked/confused. Until today I never understood HR to be the new, better way. But now that I've been awakened to the writing on the wall for Master/ Slave, I guess I'll have to start looking at HR more seriously. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Hi Darien, Darien wrote: Eventual Consistency is really the #1 reason I'm reluctant to make any change to HR. I sell virtual goods, people pay real money for things that exist in the datastore. If they buy something, and it doesn't show up for minute/hours, I get complaints. I don't want to waylay this discussion, but I'd *really* like to know how you have you sales process structured. Is a quick overview possible?? Many thanks in advance, stevep -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Why are you being charged for idle instances? You are only charged for CPU consumed by API calls. Don't make facetious comments about the SLA. Even if we never end up supporting one (this is unlikely), we will be focusing on reliability from the perspective of High Replication datastore, not Master/Slave. If you are running a production service, you should not use Master/Slave datastore. Ever. Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Wed, Mar 2, 2011 at 9:15 AM, Stephen Johnson onepagewo...@gmail.comwrote: Awesome input! Thanks Simon. I've updated my list. Not looking good for HR. PROs --- 1. No downtime during maintenance. 2. Latency/Spikes suppossedly decreased.?? 3. Faster gets/queries.?? 4. If Google ever does support an SLA it will only be HR (Ikai's post). But that might be 2020 or later :-) CONs -- 1. Cost: $.45/GB (seems expensive - is it worth it?) 2. Increased CPU (Testing is still early but it seems CPU cost is increased?? Is this just for puts with writing to multiple datacenters??) 3. Instances last longer in idle state causing a CPU charge for next request while idle. (This has never been addressed or acknowledged by Google but many have complained about it.) This may be what is causing #2 above. (See excellent comments by Simon Knott). 4. Increased latency time for Memcache requests perhaps 2 to 10 times longer?? 5. Increased loading time for loading requests. Almost double?? On Wed, Mar 2, 2011 at 10:04 AM, Simon Knott knott.si...@gmail.comwrote: HR instances definitely seem to last longer than M/S ones - I've had instances last for days on my HR app, which is only in development. However, that's highlighted a strange problem in that you seem to be charged for idle instances. The very rough idle costs are: 1hr idle = additional 2000 CPU_ms for next request 2hr idle = additional 4000 CPU_ms for next request 4hr idle = additional 8000 CPU_ms for next request ... linear increase to: 24hr+ idle = additional 45000 CPU_ms for next request This probably isn't a major problem for apps which are busy, but it should be taken into account for small apps I guess! Another thing I've noticed is that HR apps have additional latency for MemCache API calls in the region of 2-10 times longer. I've tested my app in both M/S and HR and the HR version consistently has MemCache API Gets taking 20ms on average, for tiny amounts of data, which only take 4ms on the M/S version. Loading requests seem quite long on HR instances as well, almost double that of the M/S. *NB. *It should be noted that all of my experience is based on a single app which is still in development, so all of my observations may not ring true for real-life, high-throughput apps. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
* and if you have a production service running Master/Slave datastore, you should seriously consider migrating to HR datastore. If you have large amounts of data and this is unfeasible, please let us know via a billing ticket so we can figure out a plan for moving you. Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Thu, Mar 3, 2011 at 11:19 AM, Ikai Lan (Google) ika...@google.comwrote: Why are you being charged for idle instances? You are only charged for CPU consumed by API calls. Don't make facetious comments about the SLA. Even if we never end up supporting one (this is unlikely), we will be focusing on reliability from the perspective of High Replication datastore, not Master/Slave. If you are running a production service, you should not use Master/Slave datastore. Ever. Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Wed, Mar 2, 2011 at 9:15 AM, Stephen Johnson onepagewo...@gmail.comwrote: Awesome input! Thanks Simon. I've updated my list. Not looking good for HR. PROs --- 1. No downtime during maintenance. 2. Latency/Spikes suppossedly decreased.?? 3. Faster gets/queries.?? 4. If Google ever does support an SLA it will only be HR (Ikai's post). But that might be 2020 or later :-) CONs -- 1. Cost: $.45/GB (seems expensive - is it worth it?) 2. Increased CPU (Testing is still early but it seems CPU cost is increased?? Is this just for puts with writing to multiple datacenters??) 3. Instances last longer in idle state causing a CPU charge for next request while idle. (This has never been addressed or acknowledged by Google but many have complained about it.) This may be what is causing #2 above. (See excellent comments by Simon Knott). 4. Increased latency time for Memcache requests perhaps 2 to 10 times longer?? 5. Increased loading time for loading requests. Almost double?? On Wed, Mar 2, 2011 at 10:04 AM, Simon Knott knott.si...@gmail.comwrote: HR instances definitely seem to last longer than M/S ones - I've had instances last for days on my HR app, which is only in development. However, that's highlighted a strange problem in that you seem to be charged for idle instances. The very rough idle costs are: 1hr idle = additional 2000 CPU_ms for next request 2hr idle = additional 4000 CPU_ms for next request 4hr idle = additional 8000 CPU_ms for next request ... linear increase to: 24hr+ idle = additional 45000 CPU_ms for next request This probably isn't a major problem for apps which are busy, but it should be taken into account for small apps I guess! Another thing I've noticed is that HR apps have additional latency for MemCache API calls in the region of 2-10 times longer. I've tested my app in both M/S and HR and the HR version consistently has MemCache API Gets taking 20ms on average, for tiny amounts of data, which only take 4ms on the M/S version. Loading requests seem quite long on HR instances as well, almost double that of the M/S. *NB. *It should be noted that all of my experience is based on a single app which is still in development, so all of my observations may not ring true for real-life, high-throughput apps. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
The comments about charges for idle instances are based around the fact that if you leave an instance unused for any amount of time, the next request to that instance will have an elevated CPU_ms cost - you can request a static resource on an instance which has been left idle for 24hrs and the cost of that resource will be +4 CPU_ms. Don Schwarz was looking into it from your side around a month ago, although as I clarified earlier the actual daily cost amounts to virtually nothing in the grand scheme of things if you have relatively few instances allocated. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Don't make facetious comments about the SLA. -- Who are you telling me what I can make light of or not. I still live in America. Do you? On Thu, Mar 3, 2011 at 12:38 PM, Simon Knott knott.si...@gmail.com wrote: The comments about charges for idle instances are based around the fact that if you leave an instance unused for any amount of time, the next request to that instance will have an elevated CPU_ms cost - you can request a static resource on an instance which has been left idle for 24hrs and the cost of that resource will be +4 CPU_ms. Don Schwarz was looking into it from your side around a month ago, although as I clarified earlier the actual daily cost amounts to virtually nothing in the grand scheme of things if you have relatively few instances allocated. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
lol On Thu, Mar 3, 2011 at 2:29 PM, Stephen Johnson onepagewo...@gmail.comwrote: Don't make facetious comments about the SLA. -- Who are you telling me what I can make light of or not. I still live in America. Do you? On Thu, Mar 3, 2011 at 12:38 PM, Simon Knott knott.si...@gmail.comwrote: The comments about charges for idle instances are based around the fact that if you leave an instance unused for any amount of time, the next request to that instance will have an elevated CPU_ms cost - you can request a static resource on an instance which has been left idle for 24hrs and the cost of that resource will be +4 CPU_ms. Don Schwarz was looking into it from your side around a month ago, although as I clarified earlier the actual daily cost amounts to virtually nothing in the grand scheme of things if you have relatively few instances allocated. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Look, I'm not telling you you're breaking the law or that I'm going to do anything about it. We will figure out how to support an SLA - that's a fact. And it'll happen before 2020. I haven't violated your constitutional rights, but I am going to prevent people from spreading FUD. Anyway, this thread is getting extremely off topic, so let's stick to the original topic of issues with HR datastore. Don't look at it purely from an SLA perspective. You should never be considering master/slave for new projects because we will be focusing our resources on making High Replication better. Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Thu, Mar 3, 2011 at 1:29 PM, Stephen Johnson onepagewo...@gmail.comwrote: Don't make facetious comments about the SLA. -- Who are you telling me what I can make light of or not. I still live in America. Do you? On Thu, Mar 3, 2011 at 12:38 PM, Simon Knott knott.si...@gmail.comwrote: The comments about charges for idle instances are based around the fact that if you leave an instance unused for any amount of time, the next request to that instance will have an elevated CPU_ms cost - you can request a static resource on an instance which has been left idle for 24hrs and the cost of that resource will be +4 CPU_ms. Don Schwarz was looking into it from your side around a month ago, although as I clarified earlier the actual daily cost amounts to virtually nothing in the grand scheme of things if you have relatively few instances allocated. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
I'm kind of disappointed with Google's recent attitude change. When master/slave was all there was, it was deemed 'production worthy'. Now suddenly it's If you are running a production service, you should not use Master/Slave datastore. Ever. Given that the HR datastore has some pretty big problems from a user perspective (costs 3 times more, only supports Eventual consistency), it's not really feasable to use for production code in *every* instance. So I'm very disappointed to hear these kinds of comments coming from Google. You shouldn't make something, then complain when people use it. If you don't want people using Master/Slave, maybe you should remove it. To berate customers for preferring an option you *provide* is just mind blowing. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
On Mar 4, 8:19 am, Ikai Lan (Google) ika...@google.com wrote: If you are running a production service, you should not use Master/Slave datastore. Ever. Um... until very recently, we HAD to use master/slave for our production apps, so I think the sudden pressure from Google to switch is unreasonable. HR is still very new, and I'd prefer to wait and see how it performs for a few months - PARTICULARLY for a production app where any HR teething problems could jeopardise the business. And the 100% increase in storage costs are a big incentive to stay on master/slave. Comments like Ikai's above make think that Google is keen to get everyone switched over as soon as possible. If this is the case, I'd suggest they encourage publication of real-world comparative performance and cost data, and migration case studies. Reducing the HR cost differential would make a difference too. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Front page of appengine.google.com This is a preview release of Google App Engine Google App Engine BETA the beta wasn't on there for no good reason. Roll with the punches understand there will be hick-ups. Weren't you around for the Microsoft days to learn Software Development 101? Don't bet the farm on anything that isn't labeled For Enterprise and has a version number less than 3.1. Let others fail miserably to get the bugs out. Well Google is the same beasty, accept the old wait for the first service pack doesn't work because when they upgrade, you go with them like it or not. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Darien Caldwell Sent: Thursday, March 03, 2011 6:38 PM To: Google App Engine Subject: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost? I'm kind of disappointed with Google's recent attitude change. When master/slave was all there was, it was deemed 'production worthy'. Now suddenly it's If you are running a production service, you should not use Master/Slave datastore. Ever. Given that the HR datastore has some pretty big problems from a user perspective (costs 3 times more, only supports Eventual consistency), it's not really feasable to use for production code in *every* instance. So I'm very disappointed to hear these kinds of comments coming from Google. You shouldn't make something, then complain when people use it. If you don't want people using Master/Slave, maybe you should remove it. To berate customers for preferring an option you *provide* is just mind blowing. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Ikai, The docs seem to indicate that master/slave is the way to go unless you have a mission critical application. Maybe the docs should be updated? Also when creating a new app, master/slave is the default storage option, and you don't even see the High replication option unless you click edit. Shouldn't the default be High replication? Also, does the line in docs about the high replication datastore: Offers the highest level of availability for reads and writes, imply that we can expect much fewer latency spikes for gets and queries? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
HR instances definitely seem to last longer than M/S ones - I've had instances last for days on my HR app, which is only in development. However, that's highlighted a strange problem in that you seem to be charged for idle instances. The very rough idle costs are: 1hr idle = additional 2000 CPU_ms for next request 2hr idle = additional 4000 CPU_ms for next request 4hr idle = additional 8000 CPU_ms for next request ... linear increase to: 24hr+ idle = additional 45000 CPU_ms for next request This probably isn't a major problem for apps which are busy, but it should be taken into account for small apps I guess! Another thing I've noticed is that HR apps have additional latency for MemCache API calls in the region of 2-10 times longer. I've tested my app in both M/S and HR and the HR version consistently has MemCache API Gets taking 20ms on average, for tiny amounts of data, which only take 4ms on the M/S version. Loading requests seem quite long on HR instances as well, almost double that of the M/S. *NB. *It should be noted that all of my experience is based on a single app which is still in development, so all of my observations may not ring true for real-life, high-throughput apps. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Would I be wrong then to assume that session requests are likewise impacted because like memcache sessions are also backed by the datastore? On Wed, Mar 2, 2011 at 12:04 PM, Simon Knott knott.si...@gmail.com wrote: HR instances definitely seem to last longer than M/S ones - I've had instances last for days on my HR app, which is only in development. However, that's highlighted a strange problem in that you seem to be charged for idle instances. The very rough idle costs are: 1hr idle = additional 2000 CPU_ms for next request 2hr idle = additional 4000 CPU_ms for next request 4hr idle = additional 8000 CPU_ms for next request ... linear increase to: 24hr+ idle = additional 45000 CPU_ms for next request This probably isn't a major problem for apps which are busy, but it should be taken into account for small apps I guess! Another thing I've noticed is that HR apps have additional latency for MemCache API calls in the region of 2-10 times longer. I've tested my app in both M/S and HR and the HR version consistently has MemCache API Gets taking 20ms on average, for tiny amounts of data, which only take 4ms on the M/S version. Loading requests seem quite long on HR instances as well, almost double that of the M/S. *NB. *It should be noted that all of my experience is based on a single app which is still in development, so all of my observations may not ring true for real-life, high-throughput apps. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- *Jeff Schwartz* http://jefftschwartz.appspot.com/ http://www.linkedin.com/in/jefftschwartz follow me on twitter: @jefftschwartz -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Awesome input! Thanks Simon. I've updated my list. Not looking good for HR. PROs --- 1. No downtime during maintenance. 2. Latency/Spikes suppossedly decreased.?? 3. Faster gets/queries.?? 4. If Google ever does support an SLA it will only be HR (Ikai's post). But that might be 2020 or later :-) CONs -- 1. Cost: $.45/GB (seems expensive - is it worth it?) 2. Increased CPU (Testing is still early but it seems CPU cost is increased?? Is this just for puts with writing to multiple datacenters??) 3. Instances last longer in idle state causing a CPU charge for next request while idle. (This has never been addressed or acknowledged by Google but many have complained about it.) This may be what is causing #2 above. (See excellent comments by Simon Knott). 4. Increased latency time for Memcache requests perhaps 2 to 10 times longer?? 5. Increased loading time for loading requests. Almost double?? On Wed, Mar 2, 2011 at 10:04 AM, Simon Knott knott.si...@gmail.com wrote: HR instances definitely seem to last longer than M/S ones - I've had instances last for days on my HR app, which is only in development. However, that's highlighted a strange problem in that you seem to be charged for idle instances. The very rough idle costs are: 1hr idle = additional 2000 CPU_ms for next request 2hr idle = additional 4000 CPU_ms for next request 4hr idle = additional 8000 CPU_ms for next request ... linear increase to: 24hr+ idle = additional 45000 CPU_ms for next request This probably isn't a major problem for apps which are busy, but it should be taken into account for small apps I guess! Another thing I've noticed is that HR apps have additional latency for MemCache API calls in the region of 2-10 times longer. I've tested my app in both M/S and HR and the HR version consistently has MemCache API Gets taking 20ms on average, for tiny amounts of data, which only take 4ms on the M/S version. Loading requests seem quite long on HR instances as well, almost double that of the M/S. *NB. *It should be noted that all of my experience is based on a single app which is still in development, so all of my observations may not ring true for real-life, high-throughput apps. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
I've was thinking about the charging of idle instances on my drive home and have realised that it really doesn't add up to very much - basically, you end up having ~8mins of idle CPU cost for an entire day, if you have 10 instances for your app, assuming that the idle CPU_ms cost doesn't fluctuate and stays on that linear scale. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Where are you getting the $0.45/GB? Looking in my billing settings, this is only $0.015/GB for HR storage. I'd also say that whilst the loading times appear to be increased, there are much, much less of them currently so I think that's a very minor con. My M/S version of the app appears to have a loading request every 5-10 minutes or so, whereas I've had 7 loading requests since Saturday on my HR app. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Regardless of the cost, it's not documented anywhere or explained why a charge is occurring. It seems rather unfair (and well there are other words for it) to charge users for Always On feature yet still bill them CPU. I would think that paying $0.30 per day for Always On would cover the idle CPU costs but then still try to under the radar charge people for the CPU as well and then when people bring it up in the discussion boards to just ignore it. Well, seems fishy. On Wed, Mar 2, 2011 at 1:31 PM, Simon Knott knott.si...@gmail.com wrote: I've was thinking about the charging of idle instances on my drive home and have realised that it really doesn't add up to very much - basically, you end up having ~8mins of idle CPU cost for an entire day, if you have 10 instances for your app, assuming that the idle CPU_ms cost doesn't fluctuate and stays on that linear scale. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: HR Vs. Master-Slave Comparison. Is it worth extra cost?
Ah yes, forgot that was a per-day charge - thanks for the clarification. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
