Re: [google-appengine] Re: Project too successful - due to Arctic Sea Ice Drama.
The big question for me is where are you serving these images from? If you are serving them directly from NASA servers or from an alternative source then you would most likely see very little traffic as most of it will be just URI's pointing to the images. There are a lot of hosting companies out there that claim to deliver unlimited bandwidth (not true I am sure, but worth giving it a shot). GAE seems to have been purely at delivering functionality actual content should be served from other locations if you want to keep bill within reason. If you are already showing people the images from an location other then GAE you will have to choose to limit the amount of data people can consume (only a static image no zooming, panning or any other fancy stuff) informing visitors that you are unable to afford this functionality due to the high bandwidth demands. If that still won't do it you could attempt to find some company willing to sponsor your efforts, unfortunately the nature of GAE means that portability is not to great so moving to an alternative host will be difficult at best. I have seen many projects that got sponsorship from companies or government organisations to allow them to continue to provide the unique information to viewers. On Wed, Aug 22, 2012 at 11:40 AM, Richard Watson richard.wat...@gmail.comwrote: Which resources are being hit the hardest? Outgoing bandwidth? On Wednesday, August 22, 2012 6:51:45 AM UTC+2, Torsten Becker wrote: Hi, since two years I’m running a blog at GAE focusing the Arctic and as an unique feature a Google Map with daily high resolution Arctic satellite images from NASA was included. The NASA images need to be tiled, cached and served and this process runs on GAE, too. Usually interest is low during dark Arctic Winter and rises in September the time sea ice reaches its minimum extent. No problem so far with the free quota on outgoing bandwidth. This year is different: Latest Arctic storm reduced sea ice extent by a million square kilometers in a week and public interest was so high free quota was exhausted 5 hours after reset. Now - end of August - it is absolutely clear that this year will or has already broke all records in terms of sea ice minimum and makes a major step direction ice free-ness. When in a few years the Arctic lacks sea ice completely in September, it will change weather pattern all over the northern hemisphere - one explanation of accelerating public interests. I’d like to mention the project is ad free and totally beyond any economic interests. All I want is to keep it running and give everybody on the planet the chance to see with his own eyes how dramatic the situation in the Arctic is. True color satellite images are free of interpretation and do not lead to discussions whether there is sea ice or not. Here is the thing: If I enable billing to satisfy the need for pure information I’m bankrupt next month. If not 99% of the users are going to see nothing, get frustrated and possibly never come back. So my best option is to close the site now. What do you think? -- Torsten -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/OTYSfYfbDgoJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Project too successful - due to Arctic Sea Ice Drama.
That really depends on how you get your images... Assuming the images are accessible via http on the NASA servers I would begin by using that and let NASA deal with the traffic. Contact their web admin and inform them about the expected traffic and ask them if this is going to be a problem before you do so. No reason to upset a US government agency if you don't have to ;-) If NASA is not happy with the idea of so much additional traffic (I would expect them to protest) or you cannot access the images via http but only after logging in or via some other protocol etc. Then I would basically find any provider that is offering web hosting where I could run a code from cron or a similar scheduler and of course the unlimited bandwidth bit in the conditions. After that simply run a script from a scheduler to pull the images from NASA over to this host and figure out a way to inform your app about the URI of the images and any other additional data required (I would expect some geo-coding information would be needed to place the images correctly) In both situation your current application provides the Google maps API with a URI to each of the images you want to overlay, this URI happens to be pointing to your GAE instance but there is no reason for that it could just as well point to another machine (like the one that you now have a copy of the NASA images stored) As far as the Maps API is concerned there is no problem it does it's thing and points the client to the image location using the URI. The client follows the URI requests the image and you save a lot of bandwidth. That should do the trick, every image is now being served by NASA or another provider and you only point the clients to a source where they can cheaply access the images. On Wed, Aug 22, 2012 at 1:09 PM, noiv noi...@gmail.com wrote: Thanks Rob, Richard, the image tiles are directly served from GAE and only outgoing bandwidth limits capacity. I'm going to reach out in Winter for sponsoring, but the question is how to survive next 8 weeks? If I understand you correctly there might be an option to put a provider with unlimited bandwidth in front, so GAE serves each image only once. How could I start that? -- Torsten On Wednesday, August 22, 2012 11:55:13 AM UTC+2, Rob Coops wrote: The big question for me is where are you serving these images from? If you are serving them directly from NASA servers or from an alternative source then you would most likely see very little traffic as most of it will be just URI's pointing to the images. There are a lot of hosting companies out there that claim to deliver unlimited bandwidth (not true I am sure, but worth giving it a shot). GAE seems to have been purely at delivering functionality actual content should be served from other locations if you want to keep bill within reason. If you are already showing people the images from an location other then GAE you will have to choose to limit the amount of data people can consume (only a static image no zooming, panning or any other fancy stuff) informing visitors that you are unable to afford this functionality due to the high bandwidth demands. If that still won't do it you could attempt to find some company willing to sponsor your efforts, unfortunately the nature of GAE means that portability is not to great so moving to an alternative host will be difficult at best. I have seen many projects that got sponsorship from companies or government organisations to allow them to continue to provide the unique information to viewers. On Wed, Aug 22, 2012 at 11:40 AM, Richard Watson richard...@gmail.comwrote: Which resources are being hit the hardest? Outgoing bandwidth? On Wednesday, August 22, 2012 6:51:45 AM UTC+2, Torsten Becker wrote: Hi, since two years I’m running a blog at GAE focusing the Arctic and as an unique feature a Google Map with daily high resolution Arctic satellite images from NASA was included. The NASA images need to be tiled, cached and served and this process runs on GAE, too. Usually interest is low during dark Arctic Winter and rises in September the time sea ice reaches its minimum extent. No problem so far with the free quota on outgoing bandwidth. This year is different: Latest Arctic storm reduced sea ice extent by a million square kilometers in a week and public interest was so high free quota was exhausted 5 hours after reset. Now - end of August - it is absolutely clear that this year will or has already broke all records in terms of sea ice minimum and makes a major step direction ice free-ness. When in a few years the Arctic lacks sea ice completely in September, it will change weather pattern all over the northern hemisphere - one explanation of accelerating public interests. I’d like to mention the project is ad free and totally beyond any economic interests. All I want is to keep it running and give everybody on the planet the chance to see with his
Re: [google-appengine] Re: GAE starting unnecessary instances
1/ Idle Always On instance Spawning a new Dynamic instance 2/ Spawning a new Dynamic instance Busy Always On instance 3/ Idle Dynamic instance Busy Always On instance 4/ Idle Dynamic instance Idle Always On instance So App engine prefers to use bored Always On instances over spawning new dynamic once that's good. If the Always On instances are busy it spawns a new Dynamic instance good. If a Dynamic instance is bored but the Always On once are busy the Dynamic instance gets the load still good. But then you loose me if the Always one instance is idle the Dynamic instance still gets the load, why? In this last case I would expect the Always on instance to get the load otherwise the Dynamic instance will keep on being busy and will not get stopped because of it. I don't know what the cost are of spawning and later destroying a Dynamic instance but I cannot imagine that this is such a huge cost that you would have to prefer using the Dynamic instances over the Always On once. I believe that this last rule should read: 4/ Idle Always On instance Idle Dynamic instance In which case the Idle Always On instances would get the load and the bored Dynamic instances would get cleaned up much faster then they are now. I suspect this is a typo though as I cannot imagine that this is really the setup but if it is I would say that is a candidate for change. :-) The other thing I would suggest is altering the load balancing rules for Dynamic instances from the picture painted in this email it looks like the load balancing of multiple Dynamic instances is pretty much round robin (or equal load based). If this would be changed to always try and load use one Dynamic instance till the load reaches 80% or so before using the second one and so on this would allow the despawning of excess Dynamic instances much sooner then when one uses the current setup. This does mean a slightly bigger hit in case of a serious failure of the currently preferred Dynamic instance. Hence the 80% mark for the load which is arbitrarily chosen by randomly picking a number above 50 and might need some more scientific work to ensure that in an average scenario the remaining instances will usually be able to take the load caused by the sudden death of an Dynamic instance. From what I currently see it looks like the safest option has been chosen, meaning that in all cases the service will remain active no matter what happens but this means a significant cost on the customer side. I suspect that many customers are happy with that, but I think that an equal amount of them will want to see a situation where the costs are less likely to spike while providing a similar albeit slightly less high availability solution. It might be an interesting idea to offer several flavors of high availability, ranging from the current supper safe but relatively unpredictable cost to a pretty decent with very predictable costs one. I have no idea if this is technically possible but something tells me that it should not be that hard to do. And even if it is a little harder to do my guess is that Google would be up to that task. Well that's my two cents... Regards, Rob ( reads as has priority for handling the incoming request) 2/ Spawning a new Dynamic instance Busy Always On instance 4/ Idle Dynamic instance Idle Always On instance On Fri, Jul 22, 2011 at 4:57 PM, Johan Euphrosine pro...@google.com wrote: HI Galoch, Thanks for the followup, I think you are experiencing a combinaison fo the two following rules I was pointing to in my previous email: ( reads as has priority for handling the incoming request) 2/ Spawning a new Dynamic instance Busy Always On instance 4/ Idle Dynamic instance Idle Always On instance Applied to your example it could means that: Resident Instance 1: Requests: 49 Age: 1Hr Resident Instance 2: Requests: 6 Age: 1Hr Resident Instance 3: Requests: 2 Age: 1Hr Dynamic Instance 1: Requests: 7 Age: 2min Dynamic Instance 2: Requests: 291 Age: 1Hr Dynamic Instance 3: Requests: 322 Age: 1Hr - 1 Hours ago while all your Always On instance were busy and you had a burst of incoming requests and the scheduler spawned new Dynamic instances as per rule 2/ highlighted above. - After the burst and back to normal traffic the new Dynamic Instances were handing incoming requests in priority as per rule 4/ highlighted above. - 2 Minutes ago all your instances Always On + Dynamic were busy again and the scheduler spawned a new Dynamic instance that handle 7 incoming requests. Hope that make more sense for you and Francois, but as I said earlier we are open to suggestion and I will make sure someone working on the scheduler team monitor this thread for your input. On Fri, Jul 22, 2011 at 9:09 AM, Galoch galoch...@gmail.com wrote: @Johan, The issue is not about Always On instance being busy. Its actually the other way ... the Always On instance is never busy ... at least that
Re: [google-appengine] Re: how to let the user sign-in as a different user
The trick is the same as logging into Facebook using your Google account or to Google+ using your Facebook account. The first time a user visits your site even if they are logged in to another Google product they are asked to allow this site to allow them to login using their Google account. If they accept then in their Google account a note is made that this site can now be logged into using their Google account. The next time the user visits the site they will be asked to login to their Google account, unless they are already logged in for instance via Google+ or Gmail. In that last case the Google account server will check the account see that the site has already been approved by the user and thus log them in without showing the screen asking them to allow the site. So for a user to use a different login simply have them logout. Once done send them to the login page again and Google will popup with the same question with of course a login box. After all they just logged out of their Google account... So for you to be able to offer a switch user option you will have to ask the users to logout and log back in with a different account or make it so that your application sends the to the logout page and as soon as they return back to the login page again. At the moment there is no switch user option offered by Google so the ugly load logout page, load login page is the only option if you want to use the Google authentication. If you need to offer your users this switch user option you will have to simply create your own login service at which point you should be able to switch the user without all the page reloading in the middle. On Thu, Jul 14, 2011 at 10:47 AM, Ernesto Oltra ernestoka...@gmail.comwrote: You don't have to delete any cookies from google; the GMail users DON'T log automatically, they have to give their permission; Google Accounts doesn't means GMail accounts, so @hotmail, @yahoo, etc. users can log to your application too. You should see: http://code.google.com/intl/en/appengine/docs/python/gettingstarted/usingusers.html http://code.google.com/intl/es-ES/appengine/docs/python/users/loginurls.html -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/4b0TeRPDdKMJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: SMSLib With Google App Engine
You do realize that this means your site will only work as long as your laptop is switched on and in reach of a cell tower so it can sent the SMS? Anyway, I remember several years ago working with a SMS module connected to a linux machine, what we did (as the http server ran on a different machine) was simply send an email to a specific mail address, on the machine with the SMS module a little script would poll the mailbox every few minutes and once an email was found the body was read. The body contained something like this: Sender user ID ### Telephone number Telephone number ... ### Message ... ### Each telephone number on the list would all telephone numbers that would get the message the Sender user ID was simply for internal tracking just in case someone would send something that was less appreciated by the receiver(s) Using the chanel or XMPP communication is an option but as you are going to be using a laptop to do the sending you have to make the assumption that the laptop will not be always on. By using the mailbox and the polling script your mail server will cache the messages till your laptop is available again and the polling script can be bothered to pick up the emails. (of course the mail solution will introduce a bit more delay between the user hitting the button and the SMS being send but with the sending system being a laptop I have the feeling that this might not be such a big drawback) Personally I don't see why this stuff should be done by a laptop but its your system and I am sure you have a good reason to use the setup as you are planning to do. Regards, Rob On Tue, Jul 12, 2011 at 4:37 PM, Rohit Bhat smashingro...@gmail.com wrote: Thanks for the replies. I would want a free application preferably. That's the reason I chose SMSLib. Now although I can send and receive SMSes and perform other functions from my own laptop, the only hindrance is how to integrate it with GAE. So from what I gather, the suggestion by Gubbi seems to be the best. I'll look at the channel API and XMPP. Is there any other way? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/FnamLngwOL4J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] WEB-INF\classes\appengine-api-1.0-sdk-1.5.1.jar. Consider using --enable_jar_splitting.
As the subject says this is the advice appcfg provides me: Found a jar file too large to upload: C:\Users\Rob\AppData\Local\Temp \appcfg6444875077946004627.tmp\WEB-INF\classes\appengine-api-1.0- sdk-1.5.1.jar. Consider using --enable_jar_splitting. So here is what I do to try and fix this: C:\appcfg -- enable_jar_splitting update C:\Users\Rob\workspace\Dashbord\war The result is the exact same error :-( The weird thing is that I have been uploading this app several times today but since this evening app engine doesn't like the appengine SDK anymore :-/ Am I doing something silly or incorrect? Is there away to get my project uploaded via a tool that is a little friendlier about the Google supplied jar's :-) Regards, Rob -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] WEB-INF\classes\appengine-api-1.0-sdk-1.5.1.jar. Consider using --enable_jar_splitting.
This problem has been resolved. The solution was to trash the project and start it over from an empty project. This uploaded without any complaints and so far still does. The reasons why appcfg decided to complain about the appengine api is beyond my knowledge of the system. For the moment all is fine and I'll just pretend that this was just bad luck or something and hope that it will not happen again. On Sun, Jul 10, 2011 at 2:51 AM, Rob Coops rco...@gmail.com wrote: As the subject says this is the advice appcfg provides me: Found a jar file too large to upload: C:\Users\Rob\AppData\Local\Temp \appcfg6444875077946004627.tmp\WEB-INF\classes\appengine-api-1.0- sdk-1.5.1.jar. Consider using --enable_jar_splitting. So here is what I do to try and fix this: C:\appcfg -- enable_jar_splitting update C:\Users\Rob\workspace\Dashbord\war The result is the exact same error :-( The weird thing is that I have been uploading this app several times today but since this evening app engine doesn't like the appengine SDK anymore :-/ Am I doing something silly or incorrect? Is there away to get my project uploaded via a tool that is a little friendlier about the Google supplied jar's :-) Regards, Rob -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
