This past Wednesday, the App Engine team hosted the latest session of its bimonthly IRC office hours. A transcript of the session and a summary of the topics covered is provided below. The next session will take place on Wednesday, November 18th from 9:00-10:00 a.m. PST in the #appengine channel on irc.freenode.net.
Note that this will be the first morning Chat Time session to occur after daylight time in the U.S. ends, which means that it will be taking place one hour earlier than usual in countries or states that don't observe daylight savings time. Please be aware of this time difference so you don't inadvertently miss the session. - Jason --SUMMARY----------------------------------------------------------- - Q: Does Google have any intention of contributing changes to Django that make it easier to use with App Engine, particularly the datastore bits? A: Our intent is to make it as easy as possible to get up and running with any framework (assuming that the framework is compatible with App Engine's sandbox restrictions). Some frameworks will work better out-of-the-box than others, particularly those with more general database abstractions that don't assume the use of a relational database. For Django in particular, while we don't plan to augment the framework directly, we don't discourage the various third- party libraries which aim to make App Engine integration easier, including Django AE Utils and app-engine-patch. [7:05, 7:07, 7:10-7:17, 7:19] - Q: Is there any status update on issue 1695 (high rate of DeadlineExceededErrors on instance startup)? A: We're constantly working on improving instance startup time for both runtimes. If you see these errors consistently in Python, try removing any unnecessary imports or move the import statements closer to the lines of code where they're needed -- this prevents the imports from loading all at once. Additional optimization techniques are described in http://code.google.com/appengine/docs/python/runtime.html#App_Caching. [7:19, 7:22-7:25] - Q: Has the cause of the daily early-morning latency spikes for Python applications been determined? A: More than likely, this was a temporary situation (the latest thread in the corresponding discussion group thread (http://groups.google.com/group/google-appengine-python/ browse_thread/thread/f30e43516030bb09) is from early last week), but if you continue to see these recurring spike, please post specific details including application ID, request information, and snippets from your request logs. [7:31-7:33, 7:36-7:39, 7:44-7:45] - Q: When can we expect the next SDK and what features will it have? A: We're shooting for a November release and it will tentatively have datastore cursor support (both runtimes), better support for JDO/JPA inheritance (Java runtime), etc. although the features are still subject to change. [7:32, 7:37, 7:46, 7:49-7:50, 7:52] - Q: Why does the Dashboard show a higher amount of storage space used than the datastore statistics page? A: Indexes are not accounted for (yet) on the datastore statistics page, so if you have a large number of unindexed properties and entities, the bulk of your storage can be consumed by indexes. [7:39-7:46, 7:48-7:50, 7:52-7:55] - Q: When building applications for multiple clients, is it reasonable to partition kinds by client and use datastore statistics to determine how much space each client is using individually? A: There are problems with this approach. For example, if any of your kinds require custom indexes (indexes defined in index.yaml or datastore- indexes.xml), you will have to re-deploy your application for every client since indexes can't be built programatically. A better solution may be to deploy a new application per client, but you have to complete a form in order to do so without violating App Engine's terms of service: http://code.google.com/support/bin/request.py?contact_type=AppEngineMultiInstanceExceptionRequest. [7:56, 7:58-8:03] --FULL TRANSCRIPT--------------------------------------------------- [7:00pm] Jason_Google: Hi Everyone. I'm kicking off today's Chat Time session. I'll be in the channel for the next hour to answer any App Engine questions, so shoot! [7:02pm] Jason_Google: Any questions? [7:05pm] salsakran: Jason_Google: in the recent django 1.2 voting for app engine support it was brought up that there was approval for one googler to help with app-engine-patch part time. what's google's take on the various python frameworks running on GAE? Will you guys be extending webapp at all, or help integrate say django more closely? [7:07pm] Jason_Google: webapp will be augmented from time to time as new App Engine features are introduced, e.g. the webapp.xmpp_handlers that was added with XMPP, but I think the general approach is to make it as easy as possible to get up and running with any preferred framework, which on the Python side, is usually Django. [7:08pm] AreEmmKay: are there going to be more libraries supported with the use_library() function? [7:09pm] AreEmmKay: (would vote for pytz) [7:09pm] Jason_Google: I don't know of any specific plans, but we did add Django 1.0 support in a recent release, so we're open to these kind of updates. [7:10pm] Jason_Google: Thanks for the request. Have you filed it in the issue tracker? [7:10pm] moraes_: app engine can't upgrade diango on each new release. that is just insane. [7:10pm] moraes_: *django [7:10pm] salsakran: Jason_Google: gotcha ... is there any hope of making the process of porting django apps a little less painful? [7:11pm] Jason_Google: moraes_: No, not on every release, but we can add support for newer versions of Django from time to time. Before 1.0 support was added, it was one of the most requested features. [7:11pm] moraes_: django must support datastore to be less painful. it's not app engine. ;) [7:11pm] salsakran: there's an ecosystem of apps/features in django that would make a big difference in how much stuff has to be custom coded in a typical application [7:12pm] moraes_: the problem is how django (admin, apps) are tied to relational db's. [7:12pm] Jason_Google: salsakran: I work mostly on the Java side of things, but what do you find especially painful when moving existing Django-based apps to App Engine, aside from datastore-oriented things? [7:12pm] salsakran: datastore-orientated things :) [7:12pm] moraes_: unsolvable. need a new django. :p [7:12pm] Jason_Google: salsakran: :) [7:13pm] salsakran: for a lot of apps, the underlying DB access patterns aren't particularly relational [7:13pm] salsakran: but even just going through and having to change all the accesses from <blah>.all to a query or GQL call is tedious and error prone [7:14pm] Jason_Google: salsakran: I wouldn't be surprised if there were third-party libraries to make this easier with App Engine, since many GAE developers do use Django. [7:15pm] moraes_: it depends on django / django community interest in making apps using non-relational db's - droppding django orm, that is. [7:15pm] Jason_Google: salsakran: Django AE Utils sounds promising: http://groups.google.com/group/google-appengine/web/google-app-engine-open-source-projects [7:16pm] salsakran: Jason: I'm using that, and app-engine-patch is another project that does the same [7:16pm] Jason_Google: salsakran: Ah, I see. [7:17pm] salsakran: I guess my question was whether google was going to take an active part in helping frameworks transition to a bigtable backed ORM model, or leave it to open source projects [7:17pm] salsakran: i.e. will we see google commits to django [7:19pm] nwinter: Jason_Google: It's not Java, but do you know of any progress on Issue 1695? Clumps of DeadlineExceededErrors on instance startup frequently happen and make all sorts of errors and slowness: http://code.google.com/p/googleappengine/issues/detail?id=1695 [7:19pm] Jason_Google: salsakran: Some Googlers may commit changes to Django in their spare time, I'm not sure. But in general, this isn't something that the core team is working on. Until Django has better support for other storage systems, you'll have to rely on other open source projects. [7:19pm] salsakran: gotcha [7:20pm] lurkdev: are logs of these chats posted on the web? [7:22pm] Jason_Google: nwinter: I'm aware of some changes being made on the Java side to speed up access time, but I'm not sure of the specifics on the Python side. Guido (Google engineer) commented in that thread so it sounds like he's aware of the issue. I do concur with his point that it sounds like something strange is happening if it takes 20 seconds to start -- that's not typical, especially for Python-based apps. [7:22pm] Jason_Google: lurkdev: I'll post a summary and transcript later this week or early next week to the three App Engine discussion groups. [7:23pm] nwinter: Yes, it seems like instance startups usually work (and quickly), but sometimes just die randomly somewhere while importing modules and hit the 30 second timeout, then leaving a zombie instance around. Frustrating. [7:24pm] lurkdev: cool, didn't know if these were routinely logged. I'll have to look at the groups [7:24pm] Jason_Google: nwinter: Understood. I'll follow up on this and hopefully update the issue soon. [7:25pm] nwinter: Thanks! I'm sure it's being worked on, because it gets better or worse every couple weeks; just haven't heard anything in a long time. [7:27pm] nwinter: Unrelated question: we're starting to hold chats like this for our users and are wondering about scheduling. Does the every-other-Wednesday-alternating-morning-and-night schedule work well, and do morning and night get similar attendance? [7:28pm] Jason_Google: Hmm. [7:29pm] Wooble: oh hey I only missed half of the chat this time ;) [7:29pm] Jason_Google: nwinter: The morning (PST) session gets more traffic than the evening, but I think it's helpful to have both so some users don't have to stay up especially late to ask their questions, even though I know some do. [7:29pm] moraes_: \o/ [7:31pm] Wooble: Question: have you guys idenitified what's causing the 8AM daily spikes? is it really just lots of people waking up at once and using apps? [7:31pm] Jason_Google: There's about half an hour left in today's Chat Time session, so keep the questions coming. :) [7:31pm] nwinter: Ah, okay. We have a lot of users in China, but they were all asleep the first time we did it. [7:32pm] Jason_Google: Wooble: Interesting, I hadn't noticed these spikes. You're seeing this consistently across all of your apps? [7:32pm] moraes_: new sdk release will happen in november, maybe? (i only ask about new features) [7:33pm] Wooble: there have been a bunch of messages about them here each morning. My apps get like no traffic and no one would notice if they vanished completely :) [7:34pm] Wooble: moraes_ can't wait to get those cursors :) [7:34pm] moraes_: oh yeah :D [7:35pm] maxoizo: Jason, hi! What about AsyncAPI in java? When you do it? (and it was promised in early summer, on Google IO) [7:36pm] Jason_Google: Wooble: Hmm, interesting. I haven't seen anything in the Java groups about this. It's possible that it's a bug in the Admin Console's reporting code, as are the dips in most of the traffic graphs, but I suppose it's possible that everyone wakes up at the same time. :) [7:37pm] Jason_Google: moraes_: Yes, we are currently targeting a November release, although circumstances can still change. [7:37pm] Wooble: the latency graphs for python seem to be spiking around the same time; the java ones aren't. [7:38pm] Wooble: I still like the "that's when the server room gets vacuumed" theory myself [7:39pm] Jason_Google: Wooble: :) [7:39pm] nwinter: Our app is being billed for 57 GBytes of stored data, but the Statistics only show 7 GBytes. From reading the docs, I was under the impression that if we hit the 1 MB metadata limit it just wouldn't show any statistics. Is there some reason the two numbers should be different? [7:39pm] Jason_Google: I think I should hang out in the Python discussion groups more often. It sounds like that's where all the excitement happens. [7:40pm] Jason_Google: nwinter: Interesting. Can you send me your app ID, either privately or not? [7:40pm] nwinter: skrit [7:40pm] Jason_Google: Thanks. [7:41pm] Jason_Google: In general, the resource usage reported in the daily billing summary should be regarded as canonical, but I agree that the statistics API should not be that far off. I'll look into it and let you know. [7:42pm] nwinter: Okay, thanks. [7:44pm] Jason_Google: nwinter: I assume you're using GlobalStat, and not KindStat, etc.? [7:44pm] bthomson: the latency spikes around 9am aren't a reporting bug, they are real serving problems [7:45pm] bthomson: not that 3 days is a trend but it is a bit strange [7:45pm] nwinter: Uhh... I'm just looking at the Statistics page on the App Engine Console, not accessing them programmatically or anything. [7:46pm] _tmatsuo: What features will likely to be introduced in the next version of SDK? [7:46pm] Jason_Google: nwinter: Ah, OK. I'd be interested to see what gets returned programmatically, but it's probably the same as the Statistics page. [7:48pm] nwinter: Just checked using GlobalStat and it agrees with that page, yeah. [7:49pm] Jason_Google: _tmatsuo: The feature set is still being nailed down, so I don't want to commit to anything. I'm pretty sure cursors are going be in though. And, if you're using Java, there's a bunch of improvements around inheritance in the JDO/JPA layer, etc. [7:49pm] Jason_Google: nwinter: Thanks. I'm curious about this too. I'll ask around and see what I can find out for you. [7:50pm] _tmatsuo: Jason: does inheritance mean something similar to PolyModel in Python sdk? [7:50pm] Jason_Google: nwinter: Actually, it's possible that a good chunk of the remaining space could be indexes, especially given the number of kinds you have. [7:52pm] Jason_Google: _tmatsuo: No, not exactly. If you want to use something like Python's PolyModel, you'll have to use the low-level API. Basically, with inheritance support, you can have PersistenceCapable classes extending other PersistenceCapable classes, with some fields defined in the superclass and some fields defined in the subclass and everything gets resolved correctly. It's currently the most popular feature request (or bug, depending on how you look at it) filed for the JDO/JPA layer. [7:52pm] nwinter: We do have a bunch of indexes. Is it possible to analyze index space usage? [7:53pm] Jason_Google: nwinter: I don't believe there's a public interface for this yet, although I have already requested it some time ago. If I were a betting man, this would be my guess, but I'll still follow up. [7:54pm] nwinter: Okay. I'm not too concerned about it. [7:55pm] bthomson: yes, would be nice, the indexes are often 10x the size of the data [7:56pm] dw: idle question.. is there any way to get composite indexes to apply to a model that overrides kind()? i'm wondering if its possible to reuse the db stats api to get per-customer usage stats, by suffixing my models with the customer id. but some of those kinds need composite indexes [7:56pm] dw: *models = models' kind [7:57pm] _tmatsuo: Jason: thank you for clarification ! [7:57pm] nwinter: Jason_Google: Thanks for all your help tonight! [7:58pm] Jason_Google: dw: Not sure I'm following. The Stats API doesn't currently break down space needed for composite indexes. I think I may be misunderstanding your intent. [7:59pm] dw: Jason_Google: currently stats are broken down by kind.. rather than manually counting a particular customer's DS usage, i thought it might be possible to instead prefix, e.g. the "Posting" model's kind() with the customer id that is responsible for that kind. i can then summarize customer's storage usage by summing the stats for all kinds ending with that customer id [8:00pm] dw: dont think its possible.. unless i had some crazy script to update index.yaml and reupload every so often [8:02pm] Jason_Google: dw: Ah yes, I see what you're saying. Unfortunately, there's no way to programatically define indexes, so if you need to handle more advanced queries, this won't be possible. For apps like this, we will consider granting a TOS exemption to deploy your app multiple times, one per client. Would this be something you'd be interested in? [8:02pm] Jason_Google: Let me dig up the form URL just in case... [8:03pm] Jason_Google: dw: Here's the link: http://code.google.com/support/bin/request.py?contact_type=AppEngineMultiInstanceExceptionRequest [8:04pm] Jason_Google: Thanks for the great chat, Everyone. I'll be back in two weeks, November 18th, for the next office hour session, from 9:00-10:00 a.m. PST. I'll be posting a summary and transcript of this session to the discussion groups by the end of the week or early next week. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~----------~----~----~----~------~----~------~--~---
