[google-appengine] App updates are taking forever...

2010-06-25 Thread peterk
..Well, forever meaning several minutes. I'm trying to update my app at the moment and it's currently taking 7 minutes and counting. It's been like this for the last day. I can only test some things on a live environment. Several minutes of update time makes for terrible turnaround time on tweak t

[google-appengine] Odd behaviour with referenceproperty and collections :\

2010-09-03 Thread peterk
Hello, I hadn't used referenceproperty much before in my applications, but have cause to use it now, and the behaviour of it is really confusing me. I have an entity with a reference to a second entity. >From the documentation I believe I should be able to say something like: x.entityb_set to

[google-appengine] Re: Odd behaviour with referenceproperty and collections :\

2010-09-03 Thread peterk
SearchableModel (again, pure speculation here) has some sort of > caching property that is getting cleared on update? > > Good luck, > > Joe > > On Sep 3, 6:49 am, peterk wrote: > > > > > > > > > Hello, > > > I hadn't used referenceprope

[google-appengine] Re: Is App Engine suddenly becoming more expensive???

2011-05-11 Thread peterk
If I'm understanding this all correctly...can anyone say 'Breach of Trust'? I feel like I've just had the rug pulled out from under me. I never thought we'd see changes like this. Granular billing for cpu-time down to the millisecond was a keystone attraction for AppEngine. Getting people to buy

[google-appengine] Re: Is App Engine suddenly becoming more expensive???

2011-05-11 Thread peterk
> Instance hours are billed for the instances being up for an app.  This is > one of the reasons that we are changing our scheduler, to ensure we aren't > creating instances that aren't needed and that we are taking down instances > once they are no longer needed.  Does that help clarify? > At th

Re: [google-appengine] What do you want to see answered in Greg's pricing FAQ?

2011-05-12 Thread peterk
>> The smallest granularity will be 15 minutes, but part of the scheduler change is to ensure we don't start instances to serve 1 request. This is useful to know so thank you for letting us know this. But it's disappointing to say the least. We're going from millisecond granularity with CPU-hou

[google-appengine] Re: What do you want to see answered in Greg's pricing FAQ?

2011-05-12 Thread peterk
Ha. Well, if that's the motivation, if it's coming from 'enterprise', then to rephrase, I'd like to see that explained and outlined fully in this new FAQ, and what from Google's point of view this orientation will mean for 'grass roots' developers. I mean, they talked so much about how they wa

[google-appengine] Re: Looking for ideas on eliminating instance use

2011-09-03 Thread peterk
I think this doesn't strictly relate to the OP's specific questions, but I just wanted to brain dump some strategies I'm thinking of for my next app to minimise front end instances. Requests to my app are going to be doing one of two things, more or less. Modify/Create data or read data. The firs

[google-appengine] Re: The Amazing Story Of Appengine And The Two Orders Of Magnitude

2011-09-03 Thread peterk
Very good read, thanks for posting. Will definitely be curious to see how your changes improve things or otherwise. On Sep 3, 11:16 am, Emlyn wrote: > Hi all, > > I don't think I've posted here before, but I've been an appengine user > for a while now (closing on 2 years? Is that even possible?

[google-appengine] Re: The Amazing Story Of Appengine And The Two Orders Of Magnitude

2011-09-03 Thread peterk
You can pull from task queues instead of having them push, right? That could help a lot where task queues are the source of instance spin up. I'd even be happy to dedicated a back-end to task queue pull-work if it was necessary - at least that is totally under your control. On Sep 3, 5:05 pm, Jo

[google-appengine] Re: is app engine down? I'm seeing a huge number of deadline errors and our app failing to server, hr datastore

2012-01-27 Thread peterk
Having problems here for the last half hour. Can't update my apps, get 'Exception: Version not ready'. Apps are 500-ing after a long request time with the warning that ' A serious problem was encountered with the process that handled this request, causing it to exit.' So, I don't think you're alon

[google-appengine] A question about eventual consistency/transactions

2012-02-15 Thread peterk
I've been looking around on this and it all seems a wee bit confusing, so I'm hoping to get a little clarification :) Basically, if I have a request handler that initiates a transaction, and in that transaction some models are fetched and modified, will a subsequent request be guaranteed to see th

[google-appengine] Re: A question about eventual consistency/transactions

2012-02-15 Thread peterk
ted transaction on EntityA will be guaranteed visible to a > subsequent transaction on EntityA.  If there is overlap, the second > transaction will see ConcurrentModificationException. > > Jeff > > > > > > > > On Wed, Feb 15, 2012 at 12:21 PM, peterk wrote: >

[google-appengine] Strange-ish problem with errors (datastore timeouts, errors not showing in error log etc.)

2009-12-22 Thread peterk
Hey all, Not sure if this is 'normal' but over the last couple of days or so I've seen increasing reports of 500 server errors on requests that normally are served fine. Some users have noted them too. But somewhat stranger still is that I can't debug these errors because they don't show up under

[google-appengine] Re: Strange-ish problem with errors (datastore timeouts, errors not showing in error log etc.)

2010-01-07 Thread peterk
y your application but were instead returned by the Google > front-end (GFE) that received your request. Hopefully these are very rare -- > do you have a feeling for how often the 500s occur? If you reply with your > application ID, I may be able to look into this for you after the upcom

[google-appengine] Re: Is appspot banned by twitter?

2010-01-09 Thread peterk
These services, I presume, are throttling usage from each API user. Probably based on IP address as well as just your authentication. Appspot apps, I believe, share one IP or a limited range of IPs. So you're basically sharing a resource allocation meant for one API user with a bunch of other apps

[google-appengine] Some questions about scalability and feasibility (i.e. trying to do due diligence :))

2010-02-15 Thread peterk
I've sort of been 'visioneering' some web apps that I might implement on GAE, and as I am inclined to do, I've been thinking about the best- case scenarios, and scalability to that point. I note that for billed apps, there's a request limit of 43m requests per day - calculated from a limit of 500

[google-appengine] Re: Programmatic access the logs

2010-02-15 Thread peterk
Came to ask about this and found your post. This would be exceptionally useful. A common use case at the moment seems to be this - a request comes in with some data, dump the data to memcache, then later on a task might come along, aggregate data from memcache, and persist it to datastore in batc

[google-appengine] Re: Some questions about scalability and feasibility (i.e. trying to do due diligence :))

2010-02-16 Thread peterk
ould this be OK too? Are there apps doing this already? I'm presuming yes...but it would be nice to maybe hear it more explicitly :) Thanks for your reply, On Feb 16, 1:13 am, ryan wrote: > hi peter! thanks for the post. > > On Feb 15, 10:49 am, peterk wrote: > > > I

[google-appengine] Re: Some questions about scalability and feasibility (i.e. trying to do due diligence :))

2010-02-16 Thread peterk
That's good enough for me :) Now off to build my 4000 qps monster ;) Cheers, On Feb 16, 7:18 pm, ryan wrote: > On Feb 16, 4:58 am, peterk wrote: > > > But what if your app needed this capacity on a more persistent/ > > permanent basis - would this be OK too? Ar

[google-appengine] Re: Programmatic access the logs

2010-02-16 Thread peterk
gt; http://code.google.com/p/googleappengine/issues/list?can=2&q=programm... > > > > > > On Mon, Feb 15, 2010 at 3:44 PM, peterk wrote: > > Came to ask about this and found your post. > > > This would be exceptionally useful. > > > A common use case at t

[google-appengine] Re: Google App Engine : web-app being "hibernated" ?

2010-02-17 Thread peterk
My background is python, but yes, instances of your application are spooled down and perhaps taken out of memory altogether if the app goes idle. So the next request to your app after this happens takes a bit longer to be served, because an instance of your application has to be spooled back up bef

[google-appengine] Re: GAE seems caching urlfetch results and causing phantom contents showing up

2010-02-17 Thread peterk
I believe GAE caches URL requests for a period, so that would jive with what you're seeing. However apparently you can set the cache-control header in your request to force no caching on your request. It's discussed here: http://groups.google.com/group/google-appengine/browse_thread/thread/ae52ac

[google-appengine] Re: Programmatic access the logs

2010-02-18 Thread peterk
Thanks for your replies Ikai. Mine are below: > Can you point me to where you see this? I'll follow up with the team. I actually initially read it in Dan Sanderson's 'Programming Google App Engine' book. But as it turns out it's also mentioned here in the billing FAQ: http://code.google.com/appe

[google-appengine] Re: Question on key design: Datastore errors and tablets

2010-02-18 Thread peterk
What about keynames like: counter_standard_dbf counter_standard_clo or would something like dbfo01la_counter_standard clo091b_counter_standard work better? I'm thinking of cases where you may use keynames that can in some way be constructed/predicted for fast access later. like.._counter_stand

[google-appengine] Re: Question on key design: Datastore errors and tablets

2010-02-19 Thread peterk
eb 19, 11:58 am, "Nick Johnson (Google)" wrote: > On Fri, Feb 19, 2010 at 12:28 AM, peterk wrote: > > What about keynames like: > > > counter_standard_dbf > > counter_standard_clo > > > or would something like > > > dbfo01la_counter_standard &g

[google-appengine] Re: Question on key design: Datastore errors and tablets

2010-02-19 Thread peterk
ding on how busy my app became, I'd possibly need to start doing that (i.e. lots of such large batch updates happening in short timeframes). On Feb 19, 1:12 pm, "Nick Johnson (Google)" wrote: > Hi Peter, > > On Fri, Feb 19, 2010 at 1:04 PM, peterk wrote: > > Thanks Nic

[google-appengine] Re: what's the cost of clearing the datastore?

2010-02-20 Thread peterk
I've never tested this, but is it possible you'd see better 'per- delete' cost with a larger batch? Maybe try the maximum number of entities in a batch (500, I think), calculate the average cost per entity, and see if that's any better? On Feb 20, 11:41 am, kang wrote: > Thanks, Andrew. But afte

[google-appengine] Re: app engine down?

2010-02-24 Thread peterk
Yeah. Status page is also not really loading properly. Even groups seems..slow..for me.. When the status page does load it says they're investigating a few anomalies though - in pyhton, datastore and memcache. On Feb 24, 4:20 pm, michael wrote: > anyone else getting 500 errors for App Engine d

[google-appengine] Re: Appengine down! how to notify my app's users?

2010-02-24 Thread peterk
I think we can customise our 500 error pages...which would be probably better than the big generic 500 error my site users are getting at the mo' Oh wells, live and learn. I'll probably look to getting a custom page up, at least, once things are fixed. On Feb 24, 4:52 pm, Flips wrote: > Capabili

[google-appengine] Re: Appengine down! how to notify my app's users?

2010-02-24 Thread peterk
d by all app engine applications. > > On Feb 24, 4:59 pm, peterk wrote: > > > I think we can customise our 500 error pages...which would be probably > > better than the big generic 500 error my site users are getting at the > > mo' > > > Oh wells, live and

[google-appengine] Re: app engine is down?

2010-02-24 Thread peterk
'Why' is definitely a reasonable question to ask. If the system's fully redundant or whatever, it shouldn't take this long to switch to backups etc...should it? It kind of also reveals the truthiness of the statement that you're running on the same infrastructure as Google...I do wonder if it's t

[google-appengine] Re: app engine is down?

2010-02-24 Thread peterk
GAE team and > anyone on AppEngine.. see the Downtime > message:http://groups.google.com/group/google-appengine-downtime-notify/brows... > > On Wed, Feb 24, 2010 at 12:30 PM, peterk wrote: > > 'Why' is definitely a reasonable question to ask. > > > If the syste

[google-appengine] Re: back to life

2010-02-24 Thread peterk
It says now there should be full read-write access. Moreover they say they'll have a post-mortem up. So we may indeed get a 'why' this time :) On Feb 24, 5:56 pm, kang wrote: > right. but Datastore writes are temporarily unavailable > > On Thu, Feb 25, 2010 at 1:49 AM, bFlood wrote: > > GAE see

[google-appengine] Re: How to set indexed=False for a dynamic property of Expando?

2010-02-27 Thread peterk
I don't believe you can. However you could cast your dynamic property to one of the datastore properties that are implicitly not indexed (e.g. db.Text) e.g. for a integer 123456789 expando.dynamic_prop = db.Text(str(123456789)) Then when you want to get it back use int(expando.dynamic_prop) to c

[google-appengine] Re: Task Queue implementation details

2010-04-09 Thread peterk
A question re. 'stable storage'... ...is it possible that might be opened up in a more general way in the future? Even with limitations on how much an app can store at a time or whatnot? Memcache is nice, but the unpredictability of it makes it a no-no for me in some cases. But if there was some

[google-appengine] 'Simultaneous Active Requests' limit really needs clarifying!

2010-04-10 Thread peterk
I've been using and watching appengine since, well, the beginning, and only by a random Google about an unrelated topic did I find out about this notion of a cap on 'simultaneous active requests'. I came across discussions where people were saying that there is a cap of 30 simultaneous active requ

[google-appengine] Re: 'Simultaneous Active Requests' limit really needs clarifying!

2010-04-10 Thread peterk
ed in the FAQ > entry you quote. > > It's correct that as long as you keep your latencies low, you should never > run into any sort of request limit. > > -Nick Johnson > > > > > > On Sat, Apr 10, 2010 at 2:12 PM, peterk wrote: > > I've been usin

[google-appengine] Re: Status of Native Full-Text Search

2010-04-10 Thread peterk
I think it's worth noting here that the Google IO session this year that had previously made mention of the presentation of a full text search solution has now apparently changed...there's no longer any mention of it as part of the session: http://code.google.com/events/io/2010/sessions/next-gen-q

[google-appengine] Re: App quota

2010-04-12 Thread peterk
Not sure about increasing the number of apps you're allowed...but you can delete apps now to free up places in your quota. You put in a request to delete via the dashboard (under Application Settings) and within a day or two it'll be gone. On Apr 12, 2:52 am, jzhan wrote: > I was wondering if it'

[google-appengine] Re: GAE critical message: Exceeded soft memory limit

2010-04-12 Thread peterk
Can Google disclose the RAM limit for an instance? By the sounds of it it's ~187MB but can that be confirmed? Is it a fixed limit? It won't be revised downward, will it? Also, for those seeing these notices...just to clarify, they cause the request to not be finished, right? It's not just a warnin

[google-appengine] Questions about request logs (their circular buffer, its size etc.)

2010-04-14 Thread peterk
Reading some previous discussions and things alluded to therein, some questions cropped up. I'm considering an application that quite fundamentally requires access to all its request logs. So I guess I'd be recommended to use appcfg.py to periodically pull down the logs and append them to a txt f

[google-appengine] Re: Help! About to hit the limit on appengine taskqueue 1M tasks limit

2010-04-15 Thread peterk
Are there benchmarks available for memcache reliability on GAE? It'd be great if memcache could throw a warning or error upon a put to memcache, if that put will cause an eviction of another value, and ask you to retry your put with a token to confirm you're OK with the eviction. The equivalent of

[google-appengine] Re: Help! About to hit the limit on appengine taskqueue 1M tasks limit

2010-04-15 Thread peterk
on't know if it's practical on the backend, feasible to implement. But I believe other database systems offer that kind of write-behind cache. On Apr 15, 3:45 pm, Wooble wrote: > On Apr 15, 9:48 am, peterk wrote: > > > It'd be great if memcache could throw a warning or

[google-appengine] Re: Help! About to hit the limit on appengine taskqueue 1M tasks limit

2010-04-15 Thread peterk
ed, even if your > application runs in a degraded mode. > > We exposed a "grabTail" method for Memcache but removed it due to namespace > bugs. If we release this in the future, grabTail will return the next item > to be evicted by LRU, so you can write an asynchronous task t

[google-appengine] Re: Questions about request logs (their circular buffer, its size etc.)

2010-04-15 Thread peterk
other products, it'll likely start at a number we think is > reasonable based on overall usage and trend upward. > > Anything is possible with regards to billed features - we just can't give an > ETA unless it's explicitly stated on the roadmap. > > > > > > On We

[google-appengine] Re: Questions about request logs (their circular buffer, its size etc.)

2010-04-19 Thread peterk
fixed amount of > application logging. > > -Nick Johnson > > > > > > On Thu, Apr 15, 2010 at 7:11 PM, peterk wrote: > > Hey Ikai, thanks for the reply. > > > Are you sure about the 90 day thing? I was reading a thread here where > > Nick Johnson sug

[google-appengine] Re: App Engine For Business

2010-05-19 Thread peterk
There's a distinct lack of details that makes me not sure if I should be excited or not :| I had figured things like custom-domain SSL, perhaps even a SLA or premium support would get added to the existing offering, even if at cost. The key question will be what increase in cost there'll be to mi

[google-appengine] error logs not showing full detail?

2009-08-29 Thread peterk
Hey everyone, Not sure if this is a general fault or just for me, but right now my error logs seem to be alternating between showing me the full detail of the error when I expand them, and just showing a small portion of data about the request that caused the error (i.e. the user agent string i t

[google-appengine] Re: huge latency on all requests

2009-11-03 Thread peterk
Yep, seeing this too. Didn't notice yesterday, but noticing now :) Seeing very high latency and cpu usage on requests that yesterday where maybe 20ms..now 2 seconds to ...phhh..15 seconds in some cases! Also seeing timeouts and warnings about requests not being handled because of high latency,

[google-appengine] Help with an accurate counter with no contention...?

2009-11-28 Thread peterk
Hey all, I've been looking at the Task Queue API and counter example. In my app, each user will have a couple of counters maintained for them, counting various things. Thing is, these counters need to be accurate. So I'm not sure if the example given for the Task Queue API using memcache would be

[google-appengine] Re: Help with an accurate counter with no contention...?

2009-11-28 Thread peterk
= Select * from UserTotalCount Where BobTotalCount >= 0 > >   if len(countResult) > 0: > >     countResult.BobTotalCount += len(results) > >     db.put(countResults) > >   else: > >     newCount = UserTotalCount(BobTotalCount = len(results)) > >     newCou

[google-appengine] Re: Help with an accurate counter with no contention...?

2009-11-29 Thread peterk
ion for all users), once it is done, it adds a new Task to the > >> Queue > >> with a Task Name like I mentioned above (this would cover the extremely > >> random chance that the Task ended up getting created twice somehow), and it > >> sets the delay to

[google-appengine] Re: Can a bank system be transactional and low contentious?

2009-11-30 Thread peterk
I'm not sure if this would meet your needs or not, but it might be something to look into. A book I was reading suggested transactional enqueuing of tasks to get around having to keep all entities in a single entity group, for certain kinds of transaction. So say in a transaction you need to read

[google-appengine] Re: DB Model design, fan-out, lots of writes?

2009-12-01 Thread peterk
An extra write when someone clicks to read a message doesn't seem massively expensive...at least not from a latency point of view. You probably would be OK. However you could optimise these approach further with a write-behind cache for example. That could be particularly effective if you expect

[google-appengine] Re: DB Model design, fan-out, lots of writes?

2009-12-02 Thread peterk
nt().id() + > recipient_id) ) > messages = db.get(keys) > > You can put MessageRead in the same entity group than Message to do > transactional "mark as read", but then as Peterk said there might be > contention if the message has thousands of recipients. The other > op

[google-appengine] app-engine-patch : 'User' has no attribute 'objects' ?

2009-01-20 Thread peterk
Hey there, I've been playing around with app-engine-patch for Django on AE, and I've been pretty happy with it so far. But I've run into a bit of bump on the road. As per the Django documentation, I'm trying to create a User like so: from django.contrib.auth.models import User ... user = User

[google-appengine] Re: app-engine-patch : 'User' has no attribute 'objects' ?

2009-01-20 Thread peterk
27; is not defined". I tried qualifying the name as you suggest in your post, by using from django.contrib.auth import models and then: models.User ... but this seems to cause errors elsewhere... Thanks, On Jan 20, 10:05 pm, ryan wrote: > On Jan 20, 10:28 am, peterk wrote: > ...

[google-appengine] Re: app-engine-patch : 'User' has no attribute 'objects' ?

2009-01-21 Thread peterk
Hey Waldemar, Thanks very much for that info..I might check out the latest repository version and give that a go. Great work, btw, very much appreciate what you guys are doing. On Jan 21, 9:36 am, Waldemar Kornewald wrote: > Hi Jan, > > On 20 Jan., 19:28, peterk wrote: > >

[google-appengine] Re: app-engine-patch : 'User' has no attribute 'objects' ?

2009-01-21 Thread peterk
gt; Hey Peter, > > On Jan 21, 10:45 am, peterk wrote: > > > Hey Waldemar, > > > Thanks very much for that info..I might check out the latest > > repository version and give that a go. > > > Great work, btw, very much appreciate what you guys are doing.

[google-appengine] Re: app-engine-patch : 'User' has no attribute 'objects' ?

2009-01-21 Thread peterk
:16 pm, peterk wrote: > Sorry to come back on this..I guess I should have tried this before > replying previously..but, how can I get my sample app totally up to > date? > > I checked out the patched version of django, but when I zip it up (to > replace the zipped django), it c

[google-appengine] Re: Significant quota changes. . .

2009-02-25 Thread peterk
Maybe I'm misunderstanding you, but since when was it free to run a EC2 instance? EC2 Small Instance per CPU Hour is $0.10 (same as GAE's cost 'per CPU hour') EC2 outgoing bandwidth is $0.17 per GB for the first Terabyte out. S3 outgoing bandwidth is the same cost for the first 10TB out. GAE is $

[google-appengine] Re: Annoucement: you may now purchase additional computing resources

2009-02-25 Thread peterk
The quota is actually 500 requests per second..or 30,000 per minute, or 43m per day. Google says this should be enough to withstand 'the heaviest of slashdottings', but that if you need more, you can request a raising of caps here: http://code.google.com/support/bin/request.py?contact_type=AppEn

[google-appengine] Re: Time to Ditch App Engine

2009-02-25 Thread peterk
What are your requirements? If you want media or large-file hosting..perhaps you could use amazon's s3 to host your static files to overcome any bandwidth limitations on GAE's end (if you need more than 740MB per minute)? I'm assuming Amazon doesn't have rate limits or absolute bandwidth usage li

[google-appengine] Re: Disappointed with free quota changes (probably not surprising)

2009-02-25 Thread peterk
The last official word on DOS attacks, from Marzia Niccolai, back in September last year: "On the broader issue of denial-of-service attacks, these are an unfortunate reality in the web world. While we don't currently offer applications any specific protections against attacks of this nature, thi

[google-appengine] Challenging data requirements..any advice?

2009-02-28 Thread peterk
It might be because it's a Saturday morning and I'm not thinking very clearly, but I can't really seem to find a good solution to this problem, or specifically one that might work well on GAE. The scenario is this (bear with me, it might sound a bit silly, but within the context of my application

[google-appengine] Re: URGENT!!! Let delete my Apps, I can not pay your bill

2009-02-28 Thread peterk
Please stop clogging up the group with this stuff. If you didn't sign up for billing, you won't be charged anything. If you did sign up, go into your billing settings and disable billing. http://code.google.com/appengine/kb/billing.html#disable On Feb 28, 12:52 pm, KillIEbrowser wrote: > Sorry,

[google-appengine] Re: URGENT!!! Let delete my Apps, I can not pay your bill

2009-02-28 Thread peterk
p Engine" > > 2. Appears: "Accept New Terms of > Service"http://appengine.google.com/start/newtos > > 3. Press: [Cancel] > > 4. Redirect to:http://code.google.com/appengine/ > > 4. Now I can not delete my apps... > > On Feb 28, 1:54 pm, peterk wrote: > &

[google-appengine] Re: URGENT!!! Let delete my Apps, I can not pay your bill

2009-02-28 Thread peterk
e the new app.yaml file is uploaded, your application will be disabled." Taken from: http://code.google.com/appengine/kb/general.html On Feb 28, 1:20 pm, KillIEbrowser wrote: > Sorry, I can not accept the New Terms. > > Can you delete my apps, please? > > On Feb 28, 2:17 pm

[google-appengine] Re: Challenging data requirements..any advice?

2009-02-28 Thread peterk
es? If I were to do a m:m relationship between me and other users to track my friends, and then a 1:m relationship between users and scores, would I be able to query me.friends.scores? Long-winded..sorry..thank you again for your thoughts :) On Feb 28, 5:30 pm, Qian Qiao wrote: > On Sa

[google-appengine] Re: Challenging data requirements..any advice?

2009-02-28 Thread peterk
orking, finding good strategies for addressing that on GAE might help a lot of people. The above may not be the best approach, maybe I need to flip my thinking on the whole problem.. Thanks again :) On Feb 28, 7:23 pm, peterk wrote: > Hey Joe, > > Thanks for your thoughts! > > The re

[google-appengine] Re: a fundamental increase in App Engine transparency and locality is needed

2009-03-06 Thread peterk
It's an interesting issue...I think we're all happy for things to behave pretty much like a black box when stuff is working as it's meant to, but are we so happy with that when things aren't working? Personally I don't need to know who specifically is responsible for the machines my apps run on,

[google-appengine] Re: Go2 is going to pay the bill. Go2 accepts donation now. Any cooperation is welcome.

2009-03-08 Thread peterk
I'm not familiar with typical proxy business models or the like but, currently it seems like the lifecycle of a user on your site is: 1) Visits go2.appspot.com 2) Puts desired URL in form and submits 3) Is redirected to the site they want So really the goal of the user is to get in and out of yo

[google-appengine] Re: Question: GAE and externally hosted databases

2009-03-08 Thread peterk
I don't think you can currently connect directly to, say, a mysql server from GAE. Access to remote servers on GAE is done via urlfetch: http://code.google.com/appengine/docs/python/urlfetch/overview.html Which limits you to http requests over port 80 or 443 only. You could, however, create a w

[google-appengine] Re: Expected Database Performance with Millions of Rows?

2009-03-10 Thread peterk
As far as I know: query = A.all().filter('intProperty', val).filter('date >=', date) This is one read. Hence the relatively fast performance for getting itemAs. But this: itemBs = [itemA.typeB for itemA in itemAs] ..is n reads, where n is the number of itemAs you have. In your case, an extra

[google-appengine] Re: Expected Database Performance with Millions of Rows?

2009-03-10 Thread peterk
performs one single batch get..so it should be faster then 36 seperate reads. Try that out and let me know if it improves performance.. :) On Mar 10, 12:03 pm, peterk wrote: > As far as I know: > > query = A.all().filter('intProperty', val).filter('date >=', date)

[google-appengine] Re: Expected Database Performance with Millions of Rows?

2009-03-10 Thread peterk
ool.  The total itemB size > is not more than 512bytes. > > I would be very interested if anyone could offer some clarification on > how the datastore scales with number of entities.  Half of the reason > I have this database scheme is that queries directly on itemB were > even sl

[google-appengine] Re: Application instances seem to be too aggressively recycled

2009-03-10 Thread peterk
How frequently are you hitting appengine 'cold', requiring a start-up? I'm running an app on django using app-engine-patch. Just testing and so forth, it averages around 1 request every 5 to 10 seconds. Just looking over my last 60 requests or so, I don't see any evidence of 'cold starts', all re

[google-appengine] Re: Application instances seem to be too aggressively recycled

2009-03-10 Thread peterk
/code.google.com/p/app-engine-patch/ On Mar 10, 6:34 pm, Jason C wrote: > As of right now, we are seeing instance start-ups around every 2-3 > seconds - every 2-4 requests. > > We are usinghttp://code.google.com/p/google-app-engine-django/as the > shim. > > j > > On

[google-appengine] Re: Expected Database Performance with Millions of Rows?

2009-03-10 Thread peterk
r is in there, don't mind me! :) Cheers again, it is nice to have 'official' input. On Mar 10, 9:46 pm, ryan wrote: > On Mar 10, 11:10 am, peterk wrote: > > > A survey of read cost as entity numbers increase would indeed be > > interesting however..I've bee

[google-appengine] Re: Application instances seem to be too aggressively recycled

2009-03-11 Thread peterk
I went back to my logs..I was ignoring a certain request previously, so that's why I was missing my 'cold starts'! For some reason, one of my particular requests seems to get all my cold starts. However, the request and cpu times I'm seeing for these cold starts are quite different from what Jason

[google-appengine] A question for Jaiku's developers, if they're watching..

2009-03-11 Thread peterk
I just read on your blog (from January) the intention to release the appengine port of Jaiku as open source when the port is finished..but I was wondering if I could be so cheeky as to jump ahead with a couple of questions about it. The requirements of Jaiku seem to line up roughly similarly with

[google-appengine] Re: A question for Jaiku's developers, if they're watching..

2009-03-11 Thread peterk
mewhere ... or if > protocol buffers are involved .. > > Thanks- > - Andy Badera > - and...@badera.us > - (518) 641-1280 > - Tech Valley Code Camp 2009.1:http://www.techvalleycodecamp.com/ > - Google me:http://www.google.com/search?q=andrew+badera > > On Wed, Mar 11, 2009 a

[google-appengine] Re: Expected Database Performance with Millions of Rows?

2009-03-11 Thread peterk
of itemA's construction. On Mar 11, 12:02 am, ryan wrote: > On Mar 10, 4:38 pm, peterk wrote: > > > Sorry if some redundant points are being raised here.. my lingering > > question in light of your post is regarding the variation in > > performance between itemAs&#x

[google-appengine] Re: A question for Jaiku's developers, if they're watching..

2009-03-11 Thread peterk
financially feasible.. Anyway, I thought this project looked very interesting...it's giving me a lot to chew on now.. On Mar 11, 3:50 pm, peterk wrote: > Hmm. Good thinking Andrew. I've done a bit more digging along those > lines..I don't think Jaiku has publically said if

[google-appengine] Re: A question for Jaiku's developers, if they're watching..

2009-03-11 Thread peterk
hey're actually running somehow constantly on their own.. I can't actually get the live app at the urls above to work, but not sure if it's because background workers aren't really working, or because i'm feeding it incorrect urls/configuration etc. On Mar 11, 8:01 pm, bFlood

[google-appengine] Re: A question for Jaiku's developers, if they're watching..

2009-03-12 Thread peterk
yncRPC class) > from google3.apphosting.runtime import > _apphosting_runtime___python__apiproxy > > cheers > brian > > On Mar 11, 6:17 pm, peterk wrote: > > > The app is actually live here: > > >http://pubsubhubbub.appspot.com/http://pubsubhubbub-subscriber.appspo... &g

[google-appengine] Re: Application instances seem to be too aggressively recycled

2009-03-12 Thread peterk
As cz suggested, you could set up a request handler that does a minimum of processing (i.e. one that just returns an empty response or something), and then ping it from a third party location frequently enough to keep your app hot. That would use up requests, but shouldn't burn too much cpu or ba

[google-appengine] Puzzling datastore performance discrepancy..I think..any ideas?

2009-03-12 Thread peterk
I've been experimenting with some approaches to various things, and ran into this variation in performance that's kind of puzzling me. Here I'm creating and saving an item with 3 properties - two reference properties, and a list property of keys. I was just using dummy data initially, and set th

[google-appengine] Re: Puzzling datastore performance discrepancy..I think..any ideas?

2009-03-12 Thread peterk
t values > requires the list's "simple index" to be updated in many different > places, as opposed to a single place with the duplicate values. > > -Ryan > > On Mar 12, 2:27 pm, peterk wrote: > > > I've been experimenting with some approaches to various

[google-appengine] Re: A question for Jaiku's developers, if they're watching..

2009-03-13 Thread peterk
Does that make sense? > > So if you have over the 5000 limit in the subscribers would you write the > entity twice? Each with differnt subscriber id's? > > Paul > > 2009/3/13 Brett Slatkin > > > > > Heyo, > > > Good finds, peterk! > > >

[google-appengine] Re: A question for Jaiku's developers, if they're watching..

2009-03-13 Thread peterk
ber. Even with the costly index creation for each of these 'group' entities though, it should still work out a fair bit cheaper than writing a seperate entity for each subscriber. On Mar 13, 11:47 am, bFlood wrote: > @peterk - if you don't need to query by the subscriber, you cou

[google-appengine] Re: A question for Jaiku's developers, if they're watching..

2009-03-14 Thread peterk
28 pm, peterk wrote: > Unfortunately I do need to query them based on subscriber_id..so I > can't pack them into a non-indexed property. > > Retrieving updates particular user has subscribed to is blazingly fast > though...that's the gain in the end, I can query and f

[google-appengine] Re: Release Status of Google App Engine

2009-03-14 Thread peterk
Google services and products are often held in seemingly perpetual beta :p I wouldn't worry about the designation too much. Billing was a major step in making it a service usable for large, heavy-use, popular apps, and perhaps makes it less easy for Google to hide behind the 'preview' moniker whe

[google-appengine] Re: Parallel urlfetch utility class / function.

2009-03-16 Thread peterk
Very neat.. Thank you. Just to clarify, can we use this for all API calls? Datastore too? I didn't look very closely at the async proxy in pubsubhubub.. Asynchronous calls available on all apis might give a lot to chew on.. :) It's been a while since I've worked with async function calls or thre

[google-appengine] Re: Parallel urlfetch utility class / function.

2009-03-16 Thread peterk
roxy code. > > Per the last point, you might be better waiting for an officially > sanctioned API for doing this, albeit I doubt the protocolbuffer > definitions change all that often. > > Thanks for Brett Slatkin & co. for doing the digging required to get > the async stuff

[google-appengine] Re: when will the data be deleted?

2009-03-18 Thread peterk
Google does not delete your data. Re. exporting data from your datastore, this may be of interest (first result on google for "google app engine data export"): http://aralbalkan.com/1448 On Mar 18, 10:21 am, Coonay wrote: > Hi, GAE ENGINEER: > > Yu do a great job,but may i ask a few question

[google-appengine] Re: Faster server-side json?

2009-03-23 Thread peterk
Starred, and thanks for finding that. One of the requests I expect to be the most popular in my app spits out json, so the faster the better! On Mar 23, 3:29 pm, bFlood wrote: > agreed. starred the issue. > > cpickle would be nice too > > On Mar 23, 10:55 am, Andy Freeman wrote: > > > According

[google-appengine] Re: President Obama open for questions on Google Moderator and App Engine

2009-03-25 Thread peterk
That's pretty cool, and a nice 'poster-child' for appengine. Is this a somewhat permanent arrangement? Will the Whitehouse be doing this on a recurring basis? On Mar 25, 10:01 pm, Brett Slatkin wrote: > Hey Everyone, > > Just wanted to point out something cool that's going on with App > Engine r

[google-appengine] Re: Java is next App Engine language according to Om Malik

2009-03-26 Thread peterk
Interesting. I wonder if it would be reasonable to expect python to have a performance advantage over Java on GAE? On Mar 26, 7:52 pm, Bill wrote: > http://gigaom.com/2009/03/26/java-coming-soon-to-googles-app-engine/ > > Not surprising it's Java, since it's the most starred language.  I > hope

  1   2   >