[google-appengine] Re: is app engine down? I'm seeing a huge number of deadline errors and our app failing to server, hr datastore
Its is happening again right now. All Task are stuck, and app serving is also impacted. On Jan 31, 11:49 am, Mark markrobertdav...@gmail.com wrote: This seems to be happening again now. My dashboard has been reset (only see activity from past 10 minutes and only have 1 instance running) and you can see from the Java status charts (http:// code.google.com/status/appengine/detail/serving-java/2012/01/31#ae- trust-detail-helloworld-get-java-latency) that something bad is happening. On Jan 28, 10:58 am, Sean Kovacs seankova...@gmail.com wrote: My task queues are not getting worked. Just building upusing the default queue. GAE is very inconsistent. On Jan 27, 6:09 pm, dimi dimitri.beck...@gmail.com wrote: I've had a wake up call at 2AM (Australia) while my clients in Europe were affected by this problem. If I recall it correctly, the community has had the same 'complaint' about very late information about a previous short down time. I guess the 'very late' is rather relative. I'd rather have correct information to provide to my customers, than incomplete info. My main complaint is why you don't let this down time happen during the day (in Australia) so they don't have to wake me up.. haha :) -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: is app engine down? I'm seeing a huge number of deadline errors and our app failing to server, hr datastore
+1 . Complete app is down. Somone from AppEngine team, please look into this asap. On Jan 27, 9:34 am, Tom Willis tom.wil...@gmail.com wrote: I'm seeing it too actually. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: is app engine down? I'm seeing a huge number of deadline errors and our app failing to server, hr datastore
Come on guys, look into it and fix it. Also, please acknowledge that someone from AppEngine team is looking into it. On Jan 27, 9:46 am, Kenneth kennet...@aladdinschools.com wrote: Yeah, mine's gone again. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Maximum length of query string in GET request
Can someone from AppEngine team please let me know what is the maximum acceptable query_string length for GET requests using webapp framework. Thank you -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Cost of mapreduce was $6,500 to update a ListProperty on 14.1 million entities
Even i have a table with 1.5TB of data. I need to truncate it but dont want to give thousands to delete data(i had paid thousands in old pricing model for another table. Not sure how much more it will cost now), while i pay hundreds for the data to be there. AppEngine team really needs to have a cheaper way to delete data. On Jan 5, 6:57 pm, Yohan yohan.lau...@gmail.com wrote: Hi, I feel your pain. it cost me a few thousand dollars to delete my millions enities from the datastore after a migration job (ikai never replied my post though...) and im still paying since the deletion is not completed yet (spending 100-300$ a day for the past 2 weeks now!!). Not doing much just running the delete all mapreduce job from the admin panel. There is totally somethig wrong with the way datastore writes are priced and google should seriously do something about it before they lose their big customers (i.e. the ones affected by this problem). It is simply too costly to go through your data to change an index or update stuff or delete your data. And in your case (like mine) even if you want to take your data out to externalize your custom search an storage it will cost you X000$+ to take it out and another XX,000$ to cleanup behind you (you seem to have a lot of indexed properties in your dataset). Please keep me posted on how things go with you as I'm still hoping i can get some credit/refund/assisance from google at this stage although i havent heard from them. On Jan 6, 7:24 am, Corey [Firespotter] co...@firespotter.com wrote: I work with Petey on this and can help clarify some of the details. The Entities; We have a lot of entities (~14mi) each of which have a StringListProperty called geoboxes. Like so: class Place(search.SearchableModel): name = db.StringProperty() ... # Location specific fields. coordinates = db.GeoPtProperty(default=None) geohash = db.StringProperty() geoboxes = db.StringListProperty() Background (details on geoboxing at bottom): We're running a mapreduce to change the geobox sizes/precision for a large number of entities. These entities currently have a 'geoboxes' StringListProperty with ~20 strings. For example: geoboxes = [u'37.341|-121.894|37.339|-121.892', u'37.341|-121.892| 37.339|-121.891', ...] We are changing those 20 strings to 20 new strings. Example: geoboxes = [u'37.3411|-121.8940|37.3395|-121.8926', u'37.3411|-121.8929|37.3395|-121.8916', ...] The Cost: We did almost this same mapreduce when we first added the geoboxes back in July. In that case we were populating the list for the first time so we can assume half as many operations were required (no removing of old values). Total cost i July was ~$160 for the CPU time. When we ran the mapreduce again this week to change the box sizes the cost was $18 for Frontend Instance Hours, $15 for Datastore Reads (21mil) and $2,500 for Datastore Writes (2500mil). This was not a complete run of the mapreduce. We aborted it after 5.4mil (38%) of the entities were updated. Hence Petey's estimate that the full update would cost $6,500. The Operations: Each entity update is removing ~20 existing strings from the geoboxes StringList and adding 20 more. The geobox property is indexed (and has to be) and is involved in 3 composite indexes so as best I understand it this means each string change results in 10 writes (4 + 2 * 3). So on every entity we update the geoboxes we perform 401 write operations (1 + 10 * 40). This agrees pretty well with the charges (2,500,000,000 ops / 5,424,000 entities) = 460 ops per entity. That's a lot of writes and likely the core of the surprising cost. However, I'm not sure how we could avoid that with App Engine (open to ideas!), and since we could pay for dedicated servers for that amount, I think the pricing is probably off as well. Even if we treat the geobox update as a one-time cost, we have other properties like scores, labels, etc that require occasional tweaking. Updating even a single indexed property across all these entities costs us $60-$100 and typically many times that in practice because these interesting fields tend to be used in composite indexes. -Corey Geoboxing Details Geoboxing is a technique used to search for entities near a point on the earth in a database that can only perform equality queries (like App Engine). In short, you break up the world into boxes and record which box each entity belongs to as well as any nearby boxes. Then you break up the world into larger boxes and repeat until you have a good range of sizes covered. There's a good article on the logic of algorithm here:http://code.google.com/appengine/articles/geosearch.html On Jan 5, 11:58 am, Ikai Lan (Google) ika...@google.com wrote: Brian (apologies if that is not your name), How much of the costs are
Re: [google-appengine] Re: Timeouts to Paypal
Here is the link - https://www.x.com/developers/paypal/forums/general-developer-support/requests-google-appengine-timing-out -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/wR0HPmyuJrsJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Timeouts to Paypal
I was still getting timeouts and finally put a proxy in place to avoid further issues. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/oJXkjJAhJA0J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Is it possible to have the GAE launcher keep the dev server running after hitting deploy? (Mac)
You can start it from Terminal and deploy from another Terminal tab or from Launcher. On Dec 16, 3:39 pm, Tyler tylerwa...@gmail.com wrote: Hello. I'm using the GAE launcher on the Mac. I find myself constantly having to start the development server up again because it gets turned off after I deploy. I was wondering if this is for a special reason or not? I often deploy, demo the site and fix a number of changes in my code and would like to seem them right away on my dev server, but always have to load it up again first. Any help is greatly appreciated! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Isn't .08/hr 1.92/day $59.52/month for a 600 MHZ CPU instance with 128 MB memory a LITTLE EXPENSIVE
Great points Jeff. Double pricing for double CPU and memory is very similar to Mac pricing on memory. So, what do new Mac buyers do? They just get the lowest memory possible and get the cheapest memory deal from ebay. Unfortunately, we cannot do that here. On Dec 14, 10:36 am, Jeff Schnitzer j...@infohazard.org wrote: I think there is a legitimate gripe here which is that large-memory instances are unreasonably expensive. There's some significant value-add for GAE's whole package - automatic scaling, memcache, edge caching, deployment system, API access (although these APIs are generally charged separately). This makes the $60/mo for a basic (multithreaded) instance worthwhile. It's expensive but it's convenient, and most frontend work fits fine in the F1. Also it's a little bit of apples/oranges because the GAE # is heap whereas an Amazon # is VM size, but this is probably less than a factor of 2 difference. On the other hand, there are many application components whose primary requirement is a significant chunk of RAM. All that Google infrastructure is nice but it isn't nice enough to warrant a 10X premium just for a measly 1G of RAM. And you can't even get more. Seriously, a cheap amazon standard instance has significantly more RAM than the most expensive GAE instance... lame. Consequently, backends are useful as a long-running frontend, but absolutely useless as an in-memory index. We're priced into going the inconvenient route of placing memory indexes in other cloud services. I've been generally accepting of GAE's recent pricing changes, but the price of large-memory instances basically means I have to treat that option as if it doesn't exist. Which means when Google adds all these fancy features to support different kinds of instances, from my perspective, they're wasting their time. I can't use them until they make them cheaper. So here's my plea: a 256MB instance shouldn't cost twice as much as a 128MB instance, and a 512MB instance shouldn't cost twice as much as a 256MB instance. The price curve should drop off. There's a reasonable premium to pay for running on GAE, but a factor of 10 isn't it. Just for comparison... the largest GAE backend, at 1G, costs $460/mo. A 1.5G linode instance costs $60/mo. And I can get a 4G linode instance for $160/mo. And while it's not exactly an apples/apples comparison, when I need RAM, the priority of all those other Google niceties goes down considerably. And if I needed (say) four 1G backends, you can absolutely bet that I will go with Linode and pocket the extra $20k per year. Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Too many memory errors with 1.6.1?
Looks like async urlfetch is no longer a good idea. On Dec 14, 4:51 am, Waleed Abdulla wal...@ninua.com wrote: Today I started noticing a lot of instances being killed for exceeding the memory limit (see error log attached). I didn't change any app settings! This seems to coincide with the latest SDK release. Was the memory size of front-end instance reduced with this release? Waleed GAE_memory_errors.png 229KViewDownload -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Isn't .08/hr 1.92/day $59.52/month for a 600 MHZ CPU instance with 128 MB memory a LITTLE EXPENSIVE
Saving your hard earned $$s doesnt equate to cheapskates. I just want worth of my $$s that i earned after hours of toiling in front of a screen. Sometimes 16 hours per day. On Dec 14, 12:38 pm, Joshua Smith joshuaesm...@charter.net wrote: No, new Mac buyers get what the kid at the store tells them to get, and never open their mac, or buy memory from ebay. If they were cheapskates, they'd be buying a PC that looks like a Mac on the outside, and costs a ton less. On Dec 14, 2011, at 12:26 PM, Vivek Puri wrote: Great points Jeff. Double pricing for double CPU and memory is very similar to Mac pricing on memory. So, what do new Mac buyers do? They just get the lowest memory possible and get the cheapest memory deal from ebay. Unfortunately, we cannot do that here. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Isn't .08/hr 1.92/day $59.52/month for a 600 MHZ CPU instance with 128 MB memory a LITTLE EXPENSIVE
I had hoped that AppEngine team will say - all of you guys are on F2 instances that cost $.08. I guess i was naive. So, all of us now by default end up on these crappy instances that cannot run your code, and pretty soon will be forced to upgrade to $.16 instances. Its very similar to a worthless currency. Want to get 1 cup of coffee for 1 million YadaYadaDollars? Oh yeah, bring it on baby! Heck, even iPhone 1 came with same memory. On Dec 14, 12:32 pm, Jeff Schnitzer j...@infohazard.org wrote: On Wed, Dec 14, 2011 at 1:26 PM, Vivek Puri v...@vivekpuri.com wrote: Great points Jeff. Double pricing for double CPU and memory is very similar to Mac pricing on memory. So, what do new Mac buyers do? They just get the lowest memory possible and get the cheapest memory deal from ebay. Unfortunately, we cannot do that here. With other cloud providers and the Remote API you *totally* can do that here. It's actually quite easy. It's just lame. Jeff -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Isn't .08/hr 1.92/day $59.52/month for a 600 MHZ CPU instance with 128 MB memory a LITTLE EXPENSIVE
Looking at Heroku, they offer instance for $.05 with 512mb RAM. Dont see any wording asking for minimum hours of commitment. Besides that, this minimum hours of commitment is such a pain. As i discovered, you cannot increase/decrease hours at will. Any changes you make today, it does not come into effect till next week. All these changes are turning out to be very un-Google, and makes me feel i am dealing with Verizon. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: App Engine 1.6.1 Released!
Ikai, Looking at my Application setting, i see that the Frontend Instance Class is set to F2 by default. I didnt change it to F2. However the docs(http://code.google.com/appengine/docs/adminconsole/ performancesettings.html#Setting_the_Frontend_Instance_Class) say that F1 is default . Let me know which statement is correct. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: App Engine 1.6.1 Released!
+1 for point 1. But i think to reach there, there was to be some kind of profiling information that appengine needs to be provide, like /app/ url1 url needs to be run on F4 instances, while /app/url2 needs to run on F1. On Dec 13, 3:04 pm, Waleed Abdulla wal...@ninua.com wrote: A couple of questions: 1. Since not all my requests are the same, I wouldn't want to use bigger front-end instances across the board. Are there plans to allow setting instance type based on app.yaml routes? 2. Can you clarify this point? Will mapreduce not be supported on Python 2.7? WARNING: Starting with the 1.6.2 release of the experimental Python 2.7 runtime, the mapreduce and datastore_admin builtins will not be supported. Regards, Waleed On Tue, Dec 13, 2011 at 11:46 AM, Vivek Puri v...@vivekpuri.com wrote: Ikai, Looking at my Application setting, i see that the Frontend Instance Class is set to F2 by default. I didnt change it to F2. However the docs(http://code.google.com/appengine/docs/adminconsole/ performancesettings.html#Setting_the_Frontend_Instance_Class) say that F1 is default . Let me know which statement is correct. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: App Engine 1.6.1 Released!
Can someone from AppEngine team acknowledge these questions and answer them. On Dec 13, 2:46 pm, Vivek Puri v...@vivekpuri.com wrote: Ikai, Looking at my Application setting, i see that the Frontend Instance Class is set to F2 by default. I didnt change it to F2. However the docs(http://code.google.com/appengine/docs/adminconsole/ performancesettings.html#Setting_the_Frontend_Instance_Class) say that F1 is default . Let me know which statement is correct. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Intermittent - InvalidSenderError: Unauthorized sender
Browsing through the logs, i noticed the below given error. This happened when the user sending the email is in fact a valid user listed on the app permissions page. Besides that, not all emails hitting that function call fail. So, there has to be a bug on the AppEngine side of things. ** Unauthorized sender Traceback (most recent call last): File /base/data/home/apps/app_name/app_version/folder/file.py, line 119, in send_page_email message.send() File /base/python_runtime/python_lib/versions/1/google/appengine/ api/mail.py, line 894, in send raise ERROR_MAP[e.application_error](e.error_detail) InvalidSenderError: Unauthorized sender ** -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Warmup Requests
For an application with 40-50 idle instances, is having warmup requests feature useful? If answer is yes, and we need to switch on the option, what be the impact on the cost? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: What are folks using for Backup?
I need a solution too. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Move data from one local app to another
James
Thank you for the ideas. I forgot to mention that i am using sqllite
as the datastore. Also, the second idea might work but hard to
implement.
On Dec 3, 2:49 pm, James X Nelson jamie.nel...@promevo.com wrote:
Ok, so I hex dumped my local file, and it has the app id all over the
place.
You may be able to try out the remote api between local installations,
or you're going to need to grep/cat find/replace the app ids in a copy
of your local ds
If you don't like either of these options, write a servlet that takes
a namespace, parent key, kind, id/name value, and a fields param.
You can loop through all namespaces, then all kinds/tables with
metadata
queries.http://code.google.com/appengine/docs/java/datastore/metadataqueries
Post each entity from one server of old app id, to a second server on
new appid.
Send all of the key data, and all of the fields. You can say any
request parameter that isn't a key value is a field, or use a
fields=name_of_other_fields, param to identify other request
params that are fields.
old server loops through all data, posts to new server, where a
listening servlet reconstructs and saves the entity.
Your only big worry would be field serialization / class cast problems
{putting a double where a long goes}.
If your data is simple, no worries.
Otherwise, I would prefix the field name with data type.
! NS : Parent/23 : Kind/Name = {
one: data
two: 1.0
three: [45,71,3]
}
example url:
/localhost_save?ns=NS kind=Kind name=Name parent=Parent%2023
str_fields=one one=data dbl_fields=two two=1 ...
I will leave it to you to decide if this is worth your time and how to
properly serialize your data to urls.
A very lazy way to serialize would be to make a DeferredTask class
that takes your entities as a parameter, save them to typed,
serializable java fields,
then just use an object output stream to convert to byte[], encode
bytes in UTF-8, post to the other server that has same DeferredTask
class,
where it uses ObjectInputStream to deserialize the job, call .run(),
which just saves the entity/ies, and you don't have to deal with url
serialization.
public class DataPort implements DeferredTask{
YourPojo data;
public DataPort(YourPojo data){
this.data = data;
}
public void run(){
DatastoreServiceFactory.getDatastoreService().put(convertToEntity(data));
}
}
Note, you can probably send pojos, or you could save raw entities to a
bunch of HashMaps,
HashMapString, Double doubles, HashMapString, Long longs,
HashMapString,
This would also mean you could use a single class to do all your data.
Here is the code from TaskOptions.Builder.payload(DeferredTask task)
public TaskOptions payload(DeferredTask deferredTask) {
ByteArrayOutputStream stream = new ByteArrayOutputStream(1024);
ObjectOutputStream objectStream = null;
try {
objectStream = new ObjectOutputStream(stream);
objectStream.writeObject(deferredTask);
} catch (IOException e) {
throw new DeferredTaskCreationException(e);
}
payload = stream.toByteArray();
if (getMethod() != Method.PULL) {
header(content-type,
DeferredTaskContext.RUNNABLE_TASK_CONTENT_TYPE);
method(Method.POST);
if (getUrl() == null) {
url(DeferredTaskContext.DEFAULT_DEFERRED_URL);
}
}
return this;
}
You quite likely cannot post save tasks directly from one app to
another with task queue api,
but you can serialize that byte array, mimic the headers and use url
fetch to post to the /_ah/queue/deferred of your other app.
...Whatever you choose to use, you are to, never, ever upload a script
this insecure to production!
...just saying.
On Dec 3, 11:03 am, Vivek Puri v...@vivekpuri.com wrote:
I have moved my app from MS to HRD and as a result the app name has
changed. As a result all the local data is associated with the old app
name and i have to keep switching back and forth between app names
while deploying and developing. Would anyone know how to migrate data
from one local app to another local app.
Thank you
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Timeouts to Paypal
Ikai This is the production Paypal. Timeouts are happening in 1 out of 10 urlfetch calls. Thank you On Dec 2, 1:03 pm, Ikai Lan (Google) ika...@google.com wrote: Vivek, is this to the Paypal sandbox or the real Paypal API? The sandbox has been observed as being pretty slow, but the real API is supposed to be much better. -- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com | twitter.com/ikai On Wed, Nov 30, 2011 at 1:49 PM, Brandon Wirtz drak...@digerat.com wrote: Pay pal throttles by IP, you should use their Call back method where you submit the request, and they hit a url when the transaction is processed. Also consider using a proxy to do the submission. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Vivek Puri Sent: Wednesday, November 30, 2011 1:28 PM To: Google App Engine Subject: [google-appengine] Timeouts to Paypal For the past couple weeks, we have been getting timeouts on urlfetch requests made to Paypal. We have set deadline of 55 seconds, and even then the request times out. If anyone else is also facing similar issue, please respond. Also, if anyone from GAE team has any ideas on this, please let me know. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Move data from one local app to another
I have moved my app from MS to HRD and as a result the app name has changed. As a result all the local data is associated with the old app name and i have to keep switching back and forth between app names while deploying and developing. Would anyone know how to migrate data from one local app to another local app. Thank you -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: The Performance of the SDK development server has become unbearable
I got the proxy working on my end and it has definitely helped. Local
pages are loading in around 20 seconds, which were earlier taking 60+
seconds.
On Nov 30, 2:20 pm, Ikai Lan (Google) ika...@google.com wrote:
Vivek,
I get you. There are definitely parts of the SDK and dev appserver that we
can improve, and this is one of them.
--
Ikai Lan
Developer Programs Engineer, Google App Engine
plus.ikailan.com | twitter.com/ikai
On Wed, Nov 30, 2011 at 10:49 AM, Vivek Puri v...@vivekpuri.com wrote:
Ikai,
Is it possible to bundle this into the sdk itself. All this is way too
much custom config to deal with. I already loose my email sending
capabilities from localhost(needed for testing purposes) every time i
upgrade to new version of sdk.
On Nov 29, 1:56 pm, Ikai Lan (Google) ika...@google.com wrote:
Desmond, thanks for that! I think nginx is a bit lighter weight and
easier
to configure:
https://help.ubuntu.com/community/Nginx/ReverseProxy
nginx is a bit easier to configure than Apache, especially if you're just
using it for development. Here's a sample reverse proxy config:
server { listen 80; server_name foo.bar.no
foo.bar.yes foo.bar.ok; access_log /var/log/nginx/access.log;
location / { proxy_pass http://172.27.0.2/;
include /etc/nginx/proxy.conf; }}
--
Ikai Lan
Developer Programs Engineer, Google App Engine
plus.ikailan.com | twitter.com/ikai
On Fri, Nov 25, 2011 at 5:36 PM, Desmond Brand d...@desmondbrand.com
wrote:
I wrote more detailed instructions for how to set this up:
http://desmondbrand.com/blog/2011/11/15/speed-up-your-app-engine-dev-.
..
On 23 November 2011 18:43, Calvin calvin.r...@gmail.com wrote:
On Wednesday, November 23, 2011 4:38:48 PM UTC-8, PK wrote:
3. Finally, this is not new and was already worked around in my
environment, but if you have a realistic application you need to
proxy the
serving of static content (images/javascript/css) if you want to get
any
work done. I use apache for that and a mini framework I have built. I
describe the issue more here
http://www.gae123.com/articles/gaet/fir-tim-exp.html.
It might also help in the long term, if you star issues 343
http://code.google.com/p/googleappengine/issues/detail?id=343,
6005 and 4720.
Do you have more details on how you do this? Do you just do a
redirect
to the Apache server when a static asset is requested from
dev_appserver?
--
You received this message because you are subscribed to the Google
Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/M5yP6ScsCC4J.
To post to this group, send email to
google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google
Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: What happened to the promised reliability improvement for email delivery?
'Cause we like startups! On Dec 1, 8:19 pm, voscausa robert@gmail.com wrote: Why Sendgrid and not AWS SES. I moved to AWS SES and it is looking veyy good. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] App is Down
My app is down. Can someone from AppEngine team email me. This is urgent. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: App is Down
App is back up after quota issues. On Dec 1, 9:01 pm, Vivek Puri v...@vivekpuri.com wrote: My app is down. Can someone from AppEngine team email me. This is urgent. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: The Performance of the SDK development server has become unbearable
Ikai,
Is it possible to bundle this into the sdk itself. All this is way too
much custom config to deal with. I already loose my email sending
capabilities from localhost(needed for testing purposes) every time i
upgrade to new version of sdk.
On Nov 29, 1:56 pm, Ikai Lan (Google) ika...@google.com wrote:
Desmond, thanks for that! I think nginx is a bit lighter weight and easier
to configure:
https://help.ubuntu.com/community/Nginx/ReverseProxy
nginx is a bit easier to configure than Apache, especially if you're just
using it for development. Here's a sample reverse proxy config:
server { listen 80; server_name foo.bar.no
foo.bar.yes foo.bar.ok; access_log /var/log/nginx/access.log;
location / { proxy_pass http://172.27.0.2/;
include /etc/nginx/proxy.conf; }}
--
Ikai Lan
Developer Programs Engineer, Google App Engine
plus.ikailan.com | twitter.com/ikai
On Fri, Nov 25, 2011 at 5:36 PM, Desmond Brand d...@desmondbrand.comwrote:
I wrote more detailed instructions for how to set this up:
http://desmondbrand.com/blog/2011/11/15/speed-up-your-app-engine-dev-...
On 23 November 2011 18:43, Calvin calvin.r...@gmail.com wrote:
On Wednesday, November 23, 2011 4:38:48 PM UTC-8, PK wrote:
3. Finally, this is not new and was already worked around in my
environment, but if you have a realistic application you need to proxy the
serving of static content (images/javascript/css) if you want to get any
work done. I use apache for that and a mini framework I have built. I
describe the issue more
herehttp://www.gae123.com/articles/gaet/fir-tim-exp.html.
It might also help in the long term, if you star issues
343http://code.google.com/p/googleappengine/issues/detail?id=343,
6005 and 4720.
Do you have more details on how you do this? Do you just do a redirect
to the Apache server when a static asset is requested from dev_appserver?
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/M5yP6ScsCC4J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Timeouts to Paypal
For the past couple weeks, we have been getting timeouts on urlfetch requests made to Paypal. We have set deadline of 55 seconds, and even then the request times out. If anyone else is also facing similar issue, please respond. Also, if anyone from GAE team has any ideas on this, please let me know. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Video encoding
Has anyone tried video uploading, encoding and serving on appengine. With backends being available now, you can potentially transcode videos on appengine itself. If anyone is aware of libraries enabling, please share. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: New Pricing is now in place!
Can we please get the old estimated bills back in the Billing history. I wanted to compute the total hours my app consumed for the past few weeks but the history has suddenly vanished. Please help! On Nov 7, 4:26 am, Gregory D'alesandre gr...@google.com wrote: Hello All, Today is Nov 7th and App Engine is out of Preview and the new pricing is now active! After the changeover we have yet to see any issues except that, for paid apps, we're still updating datastore quotas, so your datastore usage might not be included in your dashboard estimate, your bill will be accurate though. If you are running into any unexpected issues due to this change and would like to get a direct response from the App Engine team feel free to email us at appengine_updated_pric...@google.com. Of course we'll be monitoring this list and stackoverflow as well. Thank you! Greg D'Alesandre Senior Product Manager, Google App Engine -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Dashboard - Current Load: please return back Avg Latency CPU per URL
+1 On Nov 7, 5:23 am, Sharp-Developer.Net alexander.trakhime...@gmail.com wrote: Greg, thanks for prompt reply. I've raised an issue: *http://code.google.com/p/googleappengine/issues/detail?id=6259* Anyone feeling it reasonable please start it! To answer your question: *Why we need both Latency CPU with the new pricing model?* High latency identify there could be sequence of sync calls to API or improper limit. High CPU with low latency can identify here can be a case when we have many async request to API (while we should/can use in-instance cashed data). Does my logic sounds reasonable? At least it works for us. We use this approach to incrementally optimize top handlers one by one. Usually we get 10x improvements - completed 3 iterations at the moment. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: URLFetch is timing out excessively
+1 . Requests to Facebook are also timing out. On Oct 26, 3:10 pm, nischalshetty nischalshett...@gmail.com wrote: I agree twitter times out but this is not because of them. I'm talking about hundreds of timeout every minute. We have things in place to retry timeouts, but I'm talkin about timeouts even after 3 retries with a timeout of 20 second each. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: URLFetch is timing out excessively
Nischal, please star - http://code.google.com/p/googleappengine/issues/detail?id=6189 On Oct 26, 3:20 pm, Vivek Puri v...@vivekpuri.com wrote: +1 . Requests to Facebook are also timing out. On Oct 26, 3:10 pm, nischalshetty nischalshett...@gmail.com wrote: I agree twitter times out but this is not because of them. I'm talking about hundreds of timeout every minute. We have things in place to retry timeouts, but I'm talkin about timeouts even after 3 retries with a timeout of 20 second each. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: URLFetch is timing out excessively
Thank you. On Oct 26, 3:30 pm, Nischal Shetty nischalshett...@gmail.com wrote: Done. I too have raised a production issue -http://code.google.com/p/googleappengine/issues/detail?id=6188 On 26 October 2011 16:25, Vivek Puri v...@vivekpuri.com wrote: Nischal, please star - http://code.google.com/p/googleappengine/issues/detail?id=6189 On Oct 26, 3:20 pm, Vivek Puri v...@vivekpuri.com wrote: +1 . Requests to Facebook are also timing out. On Oct 26, 3:10 pm, nischalshetty nischalshett...@gmail.com wrote: I agree twitter times out but this is not because of them. I'm talking about hundreds of timeout every minute. We have things in place to retry timeouts, but I'm talkin about timeouts even after 3 retries with a timeout of 20 second each. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- -Nischal twitter: NischalShetty http://twitter.com/nischalshetty facebook: Nischal http://facebook.com/nischal http://www.justunfollow.com http://www.buffr.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: URLFetch is timing out excessively
Nischal, Send me link to the issue you report on Twitter. Thank you On Oct 26, 5:46 pm, Nischal Shetty nischalshett...@gmail.com wrote: It works from a non google environment. We have a service running on AWS which is working smooth. It's not a whitelist problem because these are signed calls which do not account for rate limiting based on IP. Someone from google has replied saying the problem is not on their end. I have raised an issue on the twitter discussion forum (stupid of me to not have done that simultaneously). On 26 October 2011 18:33, Brandon Wirtz drak...@digerat.com wrote: So you think that Fetches to only Twitter and FB are broken? Did you try from a non-Goog Environment? There are Fail Whales all over today. ** ** And if you are running “Hundreds of API calls a minute” you are likely hitting the API IP Limits since Twitter won’t grant a white list for Google IP’s. ** ** You should be using a proxy to access twitter through a single IP what is on the Twitter API white List. ** ** ** ** *From:* google-appengine@googlegroups.com [mailto: google-appengine@googlegroups.com] *On Behalf Of *nischalshetty *Sent:* Wednesday, October 26, 2011 12:11 PM *To:* google-appengine@googlegroups.com *Subject:* Re: [google-appengine] URLFetch is timing out excessively ** ** I agree twitter times out but this is not because of them. I'm talking about hundreds of timeout every minute. ** ** We have things in place to retry timeouts, but I'm talkin about timeouts even after 3 retries with a timeout of 20 second each. ** ** ** ** -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/YJ2bm0bM_l8J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- -Nischal twitter: NischalShetty http://twitter.com/nischalshetty facebook: Nischal http://facebook.com/nischal http://www.justunfollow.com http://www.buffr.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Datastore Read and Writes Statstics
For Estimated Charges Under New Pricing, my app is showing $40/per day for Writes and $13/day for Reads. $40 worth of writes is 40 million write operations. Right now i have not idea which table is being written to most and contributing to that number. Can AppEngine team provide statistics on these write and read operations. I would hope this is something similar to existing Datastore Statistics tool. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Datastore Read and Writes Statstics
Rishi Thank you for your suggestion. I have been using it. But for a large app, it is a manual way of finding out which table is getting hit most. Also, if tables have few fields with indexed=false, it becomes really hard to figure out the true cost of writes across tables. On Sep 22, 8:57 am, Rishi Arora rishi.ar...@ship-rack.com wrote: There was a similar question just a few hours ago on this forum: Re: Datastore Reads and Writes Crazy High Here's my answer: I used the appstats tool to figure out where all my datastore writes were coming from. Super easy to configure, and barely any processing overhead. http://code.google.com/appengine/docs/python/tools/appstats.htmlhttp://googleappengine.blogspot.com/2010/03/easy-performance-profilin... The only problem I had with this tool was it made my memory usage grow exponentially for a cron job that my backend was executing. The backend does a lot of work, and this job usually lasts 3 minutes, even though it happens only once every hour. But since this memory explosion was limited to a single type of request, I was able to exclude it from appstats by using appstats_FILTER_LIST in appengine_config.py. Hope this helps. On Thu, Sep 22, 2011 at 7:29 AM, Vivek Puri v...@vivekpuri.com wrote: For Estimated Charges Under New Pricing, my app is showing $40/per day for Writes and $13/day for Reads. $40 worth of writes is 40 million write operations. Right now i have not idea which table is being written to most and contributing to that number. Can AppEngine team provide statistics on these write and read operations. I would hope this is something similar to existing Datastore Statistics tool. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Datastore Read and Writes Statstics
As per this doc: http://code.google.com/appengine/kb/postpreviewpricing.html#operations_charged_for There are 3 categories of Datastore operations: Write operations (Entity Put, Entity Delete, Index Write) Each of these operations will cost $0.10 per 100k operations. Read operations (Query, Entity Fetch) Each of these operations will cost $0.07 per 100k operations. Small operations (Key Fetch, Id Allocation) Each of these operations will cost $0.01 per 100k operations From App Stats, i see the following datastore related operations: datastore_v3.Get datastore_v3.RunQuery datastore_v3.Put datastore_v3.AllocateIds datastore_v3.BeginTransaction datastore_v3.Commit Can someone from AppEngine team let me know under which category do the above operations fall under. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Frequent Warning in Logs tells me to contact App Engine Team
Please see this - http://code.google.com/p/googleappengine/issues/detail?id=5944 On Sep 22, 9:24 am, Tom Newton tom.new...@gmail.com wrote: Has anyone seen this before? We're seeing it a lot in the past few days. Request was aborted after waiting too long to attempt to service your request. This may happen sporadically when the App Engine serving cluster is under unexpectedly high or uneven load. If you see this message frequently, please contact the App Engine team. How do I contact the App Engine team? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Changing max idle instances affects task queues?
I havent tried yet, but personally, i would change 1 of those settings per week to be able to tell the difference in the bill. On Sep 7, 4:52 pm, Waleed Abdulla wal...@ninua.com wrote: I noticed today that the performance of my task queue has dropped considerably (see image). The highlighted queue is set to run 8 tasks/second (i.e. ~ 480 per minute) but it's running at 25% of that rate, and a backlog is building up. The only thing I changed recently is reducing my max idle instances to 20 and min pending latency to 100ms. Anyone else noticing something similar? any tips to get the task queue to run faster? [image: Screen shot 2011-09-07 at 1.26.22 PM.png] Screen shot 2011-09-07 at 1.26.22 PM.png 47KViewDownload -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Want to talk to the App Engine team in person?
Can we have a similar meetup in NYC? On Sep 2, 8:42 pm, Gregory D'alesandre gr...@google.com wrote: Hi Everyone, I know that the pricing changes are challenging and in some ways hard to understand. So, we thought it might help to talk to the App Engine team about it in person, ask us questions you might have, and maybe have a beer with us. If you are interested let us know on the form so we can put you on the list: SIGNUP:https://docs.google.com/spreadsheet/viewform?formkey=dEpzUWdiMU9ZLVlS... WHEN: September 8, 2011 6-9PM PDT WHERE: Thirsty Bear (http://g.co/maps/rmvt) 661 Howard Street San Francisco, CA 94103 Hope to see you there! Greg D'Alesandre Senior Product Manager, Google App Engine -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: the min instace time is 24 hours?
Folks, reading all the posts(to be true, i read only the first 10 of this thread), it seems everyone where is expected to be a GAE billing expert to survive. Although all the discussions sound interesting, i dont really have the time to understand all the intricacies, and i believe same is the case for other developers. Personally, sole reason for choosing GAE was that i can focus on my business model and not about balancing set of servers and the billing. It would have been way better and easier if you had increased the CPU, bandwidth, storage price to cover up for revenues. If you look at AWS, they have never done this drastic change in the their service and pricing model. Every new product comes with its pricing and stackable in some fashion onto existing products. GAE team still has the chance to cancel this change and let everyone get back to their core job, their application. Please dont make it an ego issue, if there is still a chance to cancel this change. Just for comparison, Facebook had cancelled its Facebook Beacon product in 2007 after putting in tons and tons of effort and partnerships into it. And as far as i can recall, the amount of resistance to Beacon product was much less than what i have seen with GAE pricing change. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Even the HR Datastore is (slightly) having performance issue now.
I will disagree with this post. We were on MS earlier and now on HRD. While on MS, we used to have datastore timeouts all the time and it was driving me insane looking at 20-30k error emails come in. After the move to HRD, everything is stable and there are no datastore timeouts anymore. I am happy with the move. On Sep 2, 11:15 am, Pol i...@pol-online.net wrote: This is really scary. I'm not so obsessed about the price increase as this is something you can control, but if the HRD starts becoming slower and slower, there's absolutely nothing you can do. According to these official graphs, it's literally 2X slower than 1 year ago. I'm OK with training some speed against increase reliability, the promise of HRD, but at some point, you need to draw a line: what tells us it's not going to take 500ms to do basic queries a year from now? App Engine team: what's your explanation for this? Are you going to put clauses in the SLA to guarantee max average get / put / query latencies? Seems to me that if you want to go enterprise-focus that's needed. On Sep 2, 6:23 am, Mike Wesner mbwes...@gmail.com wrote: Yes, I agree that it won't (can't) get as bad as m/s did get because HRD is awesome. My concern is mostly over how they provision and that the latency is that much higher than it was. Why does the load have such a big impact on latency of datastore query? We keep very detailed statistics about the latency of our services and its clear by our average latencies that it has creeped up over time. It is roughly 200ms higher than it was when we first moved over to HRD. Mike On Sep 2, 8:16 am, Tim Hoffman zutes...@gmail.com wrote: Hi Actually I would disagree on the M/S front, all of my apps are on M/S and even despite the recent issues M/S is way more stable than it was in 2008 and 2009, I mean WAY MORE STABLE ;-) Thats not to say more growing pains won't be felt. My guess is with a supposed imminent exodus of apps, we may say some performance improvements across the board ;-) Rgds Tim -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: A comparison of your current bill against the new pricing model is now available in the Billing History page.
Alon Please explain as to what would be a good number to tune down to when active instances is 150 and total is 600. Are Max idle instances in addition to the active instances, eg 150 active and 100 idle = 250 total instances? Also, are you charged for 250 now, or for 150, which is really the active number of instances. Besides this, what is the go-live date for this change. All the email mentions is sometime in September, which IMHO, is far-far too vague for such a critical change. And please, answer the questions here or on your email. It should not really take 12-18 hours for Google team to respond to such a critical issue. On Sep 1, 12:43 pm, Alon (Google) al...@google.com wrote: Hi Jason, Yes, Instance Hour billing is based on Total Instances -- turning down 'Max Idle Instances' should be an easy way to reduce your bill. Also, be aware that the way we do real-time monitoring isn't perfect, so the Instances graph or Instances page may show more than the specified number of Idle Instances. However, you will never be charged for more than the number of Max Idle Instances you specify. Give that slider a shot. It will take a few days for the usage report to appear, but if that doesn't have the desired effect on your moola, please contact us at appengine_updated_pric...@google.com and someone will try to help you out personally. -Alon On Aug 31, 3:39 pm, Jason Collins jason.a.coll...@gmail.com wrote: Alon, Today, if I look at the Instances graph, the number of Total Instances is much higher than the Active Instances (and always has been) - in our case at the moment: 600 Total to about 150 Active. Question: is the sample billing based on the Total Instances? I.e., is tuning down the Max Idle Instances a really easy cost savings? P.S., here are the shocking numbers we're facing: app1: 3.8x increase (5.3x increase when Python instances go up in price, unless we can get to 2.7 and thread-safety in time) app2: 17.8x increase (34x increase when Python instances go up in price, unless we can get to 2.7 and thread-safety in time) In terms of real $$, for app1, we are looking at going from around $60,000 / year to around $318,000 / year - that is a LOT of moola for an application. If we cannot optimize down, it will be a prohibitive amount of moola. j On Aug 31, 3:47 pm, Alon (Google) al...@google.com wrote: Hi All, If your large bill increase is attributable primarily to Frontend Instance Hours, one of your best bets is to go to your 'Application Settings' page and reduce the value for Max Idle Instances (don't forget to click 'Save Settings'). You can use the 'Instances' graph on your dashboard to help you to choose a good value -- and you should also be able watch the effect on that same graph. -Alon On Aug 31, 2:38 pm, Arnaldo M Pereira arna...@getwi.com wrote: 160% increase in one app for me, 65% in another. In my case, I don't think it'll be enough to have a comparison table and an optimization doc. I'll make an effort to optimize more my app, but I have the feeling that this has became too expensive for most of my needs now. On Wed, Aug 31, 2011 at 6:08 PM, Felippe Bueno felippe.bu...@gmail.comwrote: Greg, How could us start using it (reserved instance) before new pricing ? I'm asking because I think that what increases the instance hours is the 15-min startup penalization, am I wrong ? On Wed, Aug 31, 2011 at 5:56 PM, Gregory D'alesandre gr...@google.comwrote: Hey Robert, the simplest thing that will reduce your bill will be to use reserved instance hours. The bill estimates are for full priced instance-hours. Are most of the apps java or python? If java, are you using concurrent requests? If python, do you want to participate in the python2.7 trusted tester to try out concurrent requests and give us feedback? https://docs.google.com/spreadsheet/viewform?formkey=dDVaelk0NkhHMVA2... Thanks! Greg On Wed, Aug 31, 2011 at 1:51 PM, Robert Kluin robert.kl...@gmail.comwrote: My apps *are* quite optimized. They are extremely fast, consume, minimal RAM and very little CPU -- but there isn't much I can do about the number of instances used. I consistently see well under 2 QPS / instance -- for requests that always return in under 200ms. Seems like the scheduler is performing quite poorly. On Wed, Aug 31, 2011 at 15:36, Alon (Google) al...@google.com wrote: Please make sure to look at the strategies outlined in our new article to try to optimize your app's resource usage: http://code.google.com/appengine/articles/managing-resources.html On Aug 31, 1:33 pm, Robert Kluin robert.kl...@gmail.com wrote: Instance hours are the killer here too, seeing between 2x and 4x
[google-appengine] Re: delaying pricing change
Or better, dont change pricing for existing customers. Change it only for new customers and apps. On Sep 1, 1:31 pm, Angke manc...@gmail.com wrote: +1984 On Sep 1, 9:28 am, bFlood bfl...@spatialdatalogic.com wrote: +100 On Sep 1, 12:23 pm, Mark Bucciarelli mkb...@gmail.com wrote: i assume you guys must be getting some serious pressure from upper management to start making some money here. but asking us to: (1) convert to HR to get the SLA and (2) convert to multithreading is asking a lot. asking it in a short time frame is not realistic, especially for someone running a large app in production with many users. threads suck. they introduce a ton of complexity---race conditions that are hard to imagine, and mostly impossible to reproduce in a reliable way during unit testing. is there any possibility you can delay the price increase until after 2.7 is out and you have published some good docs on how to safely move to multi-threading and HR? thanks, m -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: A comparison of your current bill against the new pricing model is now available in the Billing History page.
@Jason, you have to very careful about this change. Couple of years back, i had made similar change without doing complete homework as to where the columns are being used and ended up breaking the code in number of places. Since single column queries are only inside your code and if code base is large, there is potentially no way you can be sure about where all a particular column is being used. So, make is perfectly sure that you dont need the index before removing it. Also, once the index is set to false, it is not clear, if you decided you need the index back, what is going to happen in that case. Will GAE rebuild the index? If yes, i dont think it shows up on the indexes page if it is being rebuilt. So you can be pretty much in the dark as to what is going on. On Sep 1, 1:04 pm, Jason Collins jason.a.coll...@gmail.com wrote: Just Datastore Writes alone is costing me more than before. We are seeing a massive, almost unexplainable, datastore write operations number. Index writes count as a datastore write operation. I've recently discovered (thanks Robert Kluin) that it's not just the indexes mentioned in your index.yaml, but also the per-attribute indexes (2/ attribute: asc, desc). Currently wading through many, many models to set indexed=False j On Aug 31, 6:45 pm, Raymond C. windz...@gmail.com wrote: Just Datastore Writes alone is costing me more than before. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Getting TypeError on datastore put()
I am getting the following error when trying to make a put call(to update an existing row). Would someone from Google team have an idea as to why this might be happening. 'NoneType' object is not iterable Traceback (most recent call last): File /base/data/home/apps/app_name/app_version/path/file_name.py, line 46, in post linkObj.put() File /base/python_runtime/python_lib/versions/1/google/appengine/ ext/db/__init__.py, line 1014, in put self._populate_internal_entity() File /base/python_runtime/python_lib/versions/1/google/appengine/ ext/db/__init__.py, line 989, in _populate_internal_entity self._entity = self._populate_entity(_entity_class=_entity_class) File /base/python_runtime/python_lib/versions/1/google/appengine/ ext/db/__init__.py, line 1054, in _populate_entity self._to_entity(entity) File /base/python_runtime/python_lib/versions/1/google/appengine/ ext/db/__init__.py, line 963, in _to_entity datastore_value = prop.get_value_for_datastore(self) File /base/python_runtime/python_lib/versions/1/google/appengine/ ext/db/__init__.py, line 3360, in get_value_for_datastore super(ListProperty, self).get_value_for_datastore(model_instance)) File /base/python_runtime/python_lib/versions/1/google/appengine/ ext/db/__init__.py, line 3318, in validate_list_contents for item in value: TypeError: 'NoneType' object is not iterable -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: A comparison of your current bill against the new pricing model is now available in the Billing History page.
My bill is up by almost 3 times. I dont think i have the money to pay for that. And the mid-september date is too near to be able to do anything about it. While Greg's reserved instance idea is great, but i dont think it can push down the bill as much for a app owner since you might be either under utilizing or over buying, either way loosing money. Besides that, in the past 2 weeks i have seen random upward and downward trends in instance count for my app, which make no sense to me at all. I have observed my app using 100 instances and then climbing to 200+ and then coming back to 130-140 range for pretty much similar number of requests/sec. I dont think the scheduler is ready for prime time pricing change. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Datastore API calls Summary
Currently my app quota page says that it makes around 20million+ Datastore API calls per day. I will really like to have a summary that lists which kind got how many calls at any moment of the 24 hour period. Maybe they can be included as part of Datastore Statistics. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: deploy not working
+1 On Aug 4, 9:21 am, Sarang sar...@mycontactid.net wrote: Hello, For the past couple of hours, deployment is not working (see the log output below, it always stops after cloning 700 files). I am using ./manage.py deploy --nosyncdb command like always. Could someone please let me know when this issue will be resolved? Thanks Sarang Cloned 700 files. Rolling back the update. Error 500: --- begin server output --- htmlhead meta http-equiv=content-type content=text/html;charset=utf-8 title500 Server Error/title /head body text=#00 bgcolor=#ff h1Error: Server Error/h1 h2The server encountered an error and could not complete your request.pIf the problem persists, please A HREF=http://code.google.com/appengine/community.html;report/A your problem and mention this error message and the query that caused it./h2 h2/h2 /body/html --- end server output --- -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Stackoverflow
Most of the ideas and answers(objective and discussion categorization) here are great. To give an example, yesterday i needed to know if regex is supported in objective c. First search on google lead to this - http://stackoverflow.com/questions/422138/regular-expressions-in-an-objective-c-cocoa-application And in 2 mins, i knew what was needed to be known. Any similar answers given here by Ikai, Nick or Robert are just lost. This is a big waste on part of everyone who is contributing. Just to add. Few days back i wanted to get to a particular comment on the post where Pricing changes were announced, and it took me quite a bit of searches and keyword combination that resulted in nothing. Eventually i had to email AppEngine team for information. On Aug 2, 6:04 pm, Stephen Johnson onepagewo...@gmail.com wrote: It is done. On Tue, Aug 2, 2011 at 2:29 PM, Ikai Lan (Google) ika...@google.com wrote: So I saw this in my Google Alerts earlier, and I thought, what the heck, people ask development questions on Yahoo Answers? http://answers.yahoo.com/question/index?qid=20110726102419AAFUzgy Can someone with a Yahoo account point him here? I realized that I haven't logged into Yahoo in years and can't seem to remember my password. FWIW, I've always though Yahoo Answers was for questions like this: http://www.funnyyahooquestions.com/sons-birthday-fail/ (I'm joking, of course. I've been in some pretty idiotic Google Groups. It's the people; not the technology.) -- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com | twitter.com/ikai On Tue, Aug 2, 2011 at 5:42 AM, Gary Frederick g...@jsoft.com wrote: I was forced to vote for Groups over Stackoverflow. My real vote is both. Stackoverflow is ok for a focused question that does not get booted Groups are better if you have a question that does not fit into the criteria for Stackoverflow or you are more having a conversation. I do a lot with Google's App Inventor groups. We looked a bit at Stackoverflow a while ago and it did not fit. we get a lot more less technical types there... and I have failed to check my Stackoverflow account in a while. It was not encouraging to me to see how much karma I would have to build in a place with limited use to me to add the pages or whatever they call them to get the App Inventor info there to where it should be to be useful. I wanted to be able to point back to the Google forums and those messages got crushed by the SO cops... and your milage may differ :-) Gary -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/r33pUmrjRIkJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Stackoverflow
Is it possible to migrate AppEngine group to Stackoverflow or a white label solution - http://stackexchange.com/ . It is becoming impossible to search anything meaningful and identify top threads in the current form of the group. Any good solutions provided are buried quickly below getting stated kind of questions or daily issue related questions. Also, any important announcements made by AppEngine team are hard to locate and best answers to questions are hard to validate. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
