[google-appengine] Re: Average CPU Warnings
There could be all kinds of things that cause random long periods of delay, depending on how the framework actually measures the time. For example: 1) Virtualization scheduling -- if they use something like Xen, it may at times pre-empt your entire kernel image and take time out. 2) Garbage collection -- if the process keeps running, at some point it will have accumulated enough garbage that it may need to stop-and- collect. 3) Some driver or interrupt handler -- if the measurement is in wallclock time, then any mis-behavior on the kernel part (flushing buffers synchronously, a driver that poll-waits for some timeout, etc) will show up as yours. Without knowing more how the duration is calculated, and what the execution environment is in the cluster, it's really hard to focus on only one possible cause. Especially if you can't even get kernel log outputs or syslog type data from the (virtual?) machine itself. Sincerely, jw On Dec 9, 3:14 pm, Greg [EMAIL PROTECTED] wrote: Thanks to those who responded. My observation that the blips occur at apparently random times during the handler run lead me to be a little suspicious of the new interpreter theory - I only do fairly minor from xxx import yyy imports in some of the sections that have included blips. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: GAE will hurt Linux unless...
Why does this matter? Linux will be available for as long as Linux solves a real problem in a cost effective way. Windows will be available for as long as Windows solves a real problem in a cost effective way. Google App Engine will be available for as long as Google App Engine solves a real problem in a cost effective way. Repeat for all other technologies. Slide rules were among us while they solved a real problem in a cost effective way. Did slide rule affecionados complained when the pocket calculator came around? If GAE is a pocket calculator, and something else is a slide rule, then the slide rule will die, and it will be a good thing. (Note: I'm not necessarily saying Linux is a slide rule) Btw: it's interesting to see that Microsoft Azure takes almost exactly the same approach as Google App Engine: You have to write new applications using a new technology and APIs to deploy onto their cloud, just like with Google App Engine. Meanwhile, Amazon ECC makes it possible to incrementally migrate your existing applications to a cloud without any large up-front re-development investment. History shows that the Amazon approach has a 95% chance of winning, assuming it's even a competition and people actually don't need to have their infrastructure in their own server closets or co-lo cages. On Dec 3, 12:23 pm, Amir Michail [EMAIL PROTECTED] wrote: I suspect that the Google App Engine (and cloud computing more generally) will have the unintended effect of significantly reducing usage of Linux among web developers. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Best practices implementing paging
The only downside is that it's not possible to page back HTTP REFERER (not accurate but reliable in most conditions..) Couldn't you build a paging index incrementally, and put it in a cookie? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Observing GAE Scalability (or trying to ...)
On Nov 8, 11:03 am, Ross Ridge [EMAIL PROTECTED] wrote: For example, Sun claims that Project Darkstar will let you scale your massively multiplayer server infrastructure. Well, Google App Engine would be a terrible place to serve most massively multiplayer games. I hope that's not what got you interested in GAE. Clearly not. That was just an example of where claims of scalability by a vendor, and actual scalability, as observed, differ, because it appears that some messages in this thread imply that we take the scalability of GAE for granted, rather than measure it ourselves. If you're about to develop a real application to be used by real people, knowing whether a vendor's claims about scalability are actually applicable to your technology is a pretty useful metric. Which is why your test case needs to be realistic, otherwise it won't Right. There are a number of typical workloads that could be put into benchmarks to show that when the workload is X, it will scale like Y. If nothing else, then the GAE people should have some internal benchmarks (or else how do they know enough to claim it will scale?) A simple benchmark would be to emulate the wall of Facebook. Let users add short messages to other users. In this case, the two operations are: 1) user X adds message to user Y, which records the message, the time, the source, and the destination 2) user X checks messages, which selects all messages to user X, ordered by time descending, paginated to 20 messages at a time Build it on GAE with BigTable; build it on LAMP with SQL, and build it on ECC with SimpleDB, and compare the response times and other behavior across different amounts of load. This would give us, the community, something to compare and consider. The application uses some of the basic operations that most applications will need, yet is simple enough that you can write it in an afternoon and won't get bogged down in minutiae when analyzing it, and it would allow us to point at the test and say on application X, it scales like Y, whereas a single LAMP scales like Z and ECC scales like W. Given that the original posted claimed to have a cluster of 20 boxen to serve as load generators, I assume he has the necessary resources to do this, should he choose to. I know that I would be very interested in seeing the results. Sincerely, jw --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Observing GAE Scalability (or trying to ...)
I agree. The GAE team claims that GAE scales. Do we just take their word for it? Or is there some independent verification we can perform on ourselves that we can point at and say just run XYZ.py and you'll see it for yourself ? For example, Sun claims that Project Darkstar will let you scale your massively multiplayer server infrastructure. Unfortunately, the database back-and is structured such that you can't put your game simulation into the actual darkstar engine (it won't keep up) -- and the current implementation of Darkstar makes the overall throughput for the entire system go *down* when you add server nodes. So, they claim that at some point they will scale, but currently, they don't. If you're about to develop a real application to be used by real people, knowing whether a vendor's claims about scalability are actually applicable to your technology is a pretty useful metric. Sincerely, jw On Nov 8, 8:09 am, Andy Freeman [EMAIL PROTECTED] wrote: Scalability is something of a red-herring. It doesn't matter if a PC scales if GAE tops out before said PC. Why GAE appears to be topping out for this application seems like a good thing to find out. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Entites which must not exist
Ian Bambury wrote: But really the question is a more general one of how do you approach this scenario with an open-ended number of clubs? In a normal relational database, you could hold just the booked slots and with one SQL request, get the top 20 available courts by time, distance or whatever. My client is very keen on using GAE, so I need to find a scalable way of doing it, or a very solid reason to use a relational DB. Is a billion records that bad for 10,000 clubs with 10 courts each? Really, at that scale, you have a true enterprise application, and the storage for a billion records is likely to just be a few hundred gigabytes. Fits on one hard disk. If you really don't want to store too far in advance, then you can use a separate entry for the club/date combination, and only pre-allocate records for a club/date combination when someone wants to book (and then mark that club/date as not fully available). The nice side effect of that would be that certain clubs can shut down for certain dates (christmas, international hello day, or whatever). Thus, you'd select a court either if the booking time is empty, or if the club/date for the desired date says everything is free. However, that sounds like a complication you should put off until it's shown that you actually need it. Sincerely, jw --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Entites which must not exist
Ian Bambury wrote: I know I can reduce the number of records required by having a hierarchical structure of months/weeks/days/hours or whatever and then create dummy records if I need them, and write all the code to work out that if I can get the year record and then I can get the month record and then I can get the week record but I can't find a day record then the hour I want is free and then moving on the the next health club or whatever and doing it all again and again and again until I have enough results to display, but it seems such a pain in the arse compared to just having records for actual bookings and making one query to get back a list of the next n free slots. If we assume a 50% occupancy percentage, and you have all the records in one big table, then how many simultaneous users can your SQL server handle before bogging down? Chances are, the distributed database way of doing it will perform a lot better when there are many, many users using the system all the time, for a reasonable dollar cost. And chances are that Oracle might not be able to sell you a big enough database to scale to the same number of users, at any cost. So, which is more important? Formulating the query using the tools you already know, or being able to scale to millions of simultaneous users? You can't get anything for free, but you can do what you want on Google apps, and reap the alleged scalability benefits. Sincerely, jw --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Entites which must not exist
Because the Google database does support transactional updates, you could pre-create all possible time slots at all possible clubs, and have a property on it saying who has booked it. Set it to NULL initially to show that it's not booked. Then, just select the set of bookings that are at the right time, with a booker of NULL. When updating, your transaction will fail if someone else has stolen the booking in the meanwhile, and the user has to go back and try again. (Also known as level 8 retry :-) I could be wrong, but that's how I understand it should work. It's up to you to make sure that old bookings get purged, and new possible bookings get created enough ahead of time from some authoritative source, and also to make sure there are no duplicated possible bookings for a given court at a given club. On Nov 5, 4:09 pm, Ian Bambury [EMAIL PROTECTED] wrote: I think this is the last question I need answered in order to see if GAE can do what I need. Imagine a database of sports clubs all of which have customers for tennis courts My idea is that I have Club Customer Booking Court where a booking has a reference to the court and the customer and the club, and also holds the date and time of the booking (just the hour to make things simple, and we'll assume all bookings are an hour long) Two things: a) Is this evil 'old' thinking and I should be doing something else? b) If not, then how would I find out if there were a free tennis court somewhere in any of the clubs at a certain time and on a certain date? It's easy enough to find existing bookings on that day and at that time, but how do I find a court without a booking? I believe it is not possible to retrieve all the courts and get rid of the ones with bookings because there will be more than 1000 courts in the system. Can anyone help me out please? Thanks Ian --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
