Two Wednesday back, the App Engine team hosted the latest session of its bimonthly IRC office hours. A transcript of the session and a summary of the topics covered is provided below. The next session will take place this coming Wednesday, February 3rd from 7:00-8:00 p.m. PST in the #appengine channel on irc.freenode.net.
- Jason --SUMMARY----------------------------------------------------------- - Q: Should you split a single entity into two or more entities if one or more properties are updated frequently, e.g. per request? A: No, but if you expect to update a single entity or entity group more than once per second, you should consider sharding to avoid datastore contention (http://code.google.com/appengine/articles/scaling/ contention.html). You generally don't need to split entities, although you can do so to save bandwidth if your entities contain several large properties that only need to be accessed occasionally. [9:01-9:06] - Q: Does the 30-second request deadline applied during Blobstore uploads? A: No. In general, time spent by the user doesn't count towards your execution time limit. [9:07-9:08] - Discussion on the maximum length of task queue parameters. [9:10-9:12] - Discussion on bulkloader dump and restore, whether there are any plans to allow for clearing a single datastore kind only, and whether tasks or remote_api is the best mechanism for clearing the datastore [9:16-9:20, 9:24-9:25] - Q: How can one determine the string-encoded key of a given entity for another application, e.g. when updating foreign keys ahead of a bulk upload to another datastore? A: This could change going forward, but for now you can supply the '_app_id_namespace' argument to db.Key.from_path. Or instead of using string-encoded keys, you can use Key.to_xml which provides a transparent encoding. Bear in mind that App Engine currently doesn't allow for referencing entities in other applications, though this could change in the next release. [9:24-9:25, 9:27-9:28, 9:32, 9:35-9:36] - Discussion on whether, when building a master-detail relationship (e.g. orders and items), it's better to store all order information in a single entity or to use an individual entity for each item in the order. [9:27-9:35] - Q: If you're new to App Engine, which runtime should you start with? A: Pick the runtime language most familiar to you to minimize the amount of new learning that you have to do all at once -- even though many use standard APIs, App Engine's various services will take time to master, especially the datastore. [9:35-9:38] - Q: For list properties, is the name of the property stored for each item (e.g. N times for N items) even when its unindexed? A: Yes -- the datastore is schemaless, so a list foo=[bar,baz] is actually stored as two properties with the same name: foo=bar and foo=baz. Hence, each item in the list requires the name to be stored with it. See the article at http://code.google.com/appengine/articles/storage_breakdown.html which explains exactly what gets stored for all entity and index rows. [9:40-9:43, 9:52-9:56, 9:59-10:01] - Discussion of deferred.defer, its ability to schedule tasks to be executed during off-peak hours, and its relation to the core product. [9:41-9:46, 9:50] --FULL TRANSCRIPT--------------------------------------------------- [09:00am] nickjohnson: Welcome to the fortnightly App Engine developer chat! With me today are Jason and Ikai. [09:00am] nickjohnson: And Ryan [09:00am] ikai_google: Hi everybody! [09:00am] ryan_google: and me! (...something) [09:01am] dennis_tw: I have a question about entities that are changed often (eg: changed on every user action. Like modification time or view count). I'm thinking of separating them out into a separate entity so that that 'high frequency' entity is small and can be moved between the datastore and my app with little overhead. Is that a reasonable entity design? [09:01am] ikai_google: *crickets* [09:02am] nickjohnson: dennis_tw: Optimising to minimize the amount of unwanted data you're moving is a good idea. It doesn't have to relate specifically to how frequently it's updated, though. [09:02am] mcrady: any plans to support stretching of images? [09:02am] ikai_google: dennis_tw: In the past, even outside of App Engine, I've used a design where I do frequent writes to MemCache and flush. Of course, you run the risk of some data loss [09:03am] dennis_tw: cool [09:03am] dennis_tw: I'm also wondering if 'small entities' and 'medium sized entities' have pretty much the same overhead. At what point does it become irrelevant how small the entity is? is there a minimum transfer size? [09:03am] ikai_google: mcrady: Do you mean resizing to a larger image? [09:03am] ryan_google: +1 to ikai and nick's points. i'd also note that entity size, in bytes or number of properties or whatever, generally doesn't matter in these cases, but you definitely want to take chances to coalesce, shard, etc to avoid writing to a single entity (or group) too often [09:03am] nickjohnson: There's no 'minimum transfer size'. Cost is roughly linear with entity size. [09:04am] nickjohnson: Well, it's polynomial, but that's one of the factors. [09:04am] mcrady: no, resizing in one dimension [09:04am] ryan_google: dennis: you mean in bytes, right? [09:05am] dennis_tw: well, i can move all my dymanic stuff into a smalll dynamic entity. or i can just put everything in 1 entity and it's not that much bigger. so wondering if it will really make a different (and at what point it will make a difference) [09:05am] ikai_google: mcrady: I'm not aware of any changes coming to the Images API, but please add it to the issues list [09:05am] ikai_google: mcrady: http://code.google.com/p/googleappengine/issues/list [09:06am] mcrady: ok, the image api could really use some attention in general [09:06am] ryan_google: dennis: again, what do you mean by "small" and "bigger." total size in bytes? number of properties? number of property values? ... [09:06am] nickjohnson: dennis_tw: Generally I would only worry if there's a large number of properties, or a very large quantity of data that you're fetching and don't need. [09:07am] mcrady: the 30 second timeout doesn't apply to blobstore timeouts, right? Is there any timeout at all on the blobstore upload? [09:07am] mcrady: blobstore timeouts -> blobstore uploads [09:08am] ryan_google: mcrady: afaik no there's no timeout [09:08am] nickjohnson: mcrady: For uploads and downloads in general, time spent by the user doesn't count towards your execution time limit [09:10am] lent: a question in regards to task queues, what is the maximum length that I can pass through as a parameter (for a POST) after which I would have to switch over to using payload? [09:10am] nickjohnson: lent: Roughly 10k [09:10am] nickjohnson: The exact limit in bytes varies a little bit with other parameters, so if you anticipate hitting the limit, a try/ catch may be your best bet. That's the approach deferred takes. [09:11am] gQuash: I arrived late again :( [09:11am] lent: nickjohnson: isn't 10k the total limit? I can pass a 10k string as single parameter? [09:12am] moraes: gQuash, no you're in time! [09:12am] nickjohnson: gQuash: You haven't missed much yet, and the transcript will be up later. [09:12am] gQuash: [09:12am] gQuash: awesome !!!! [09:12am] • moraes waves at gQuash [09:12am] nickjohnson: lent: Yes, 10k is the total limit. If you're letting the task queue lib do your encoding for you, then 10k is the limit on the url encoded string. [09:13am] nickjohnson: gQuash: Any questions, or are you here to spectate? [09:14am] moraes: i'm excited about http://code.google.com/events/io/2010/sessions/next-gen-queries-appengine.html [09:14am] moraes: this is not a question. [09:14am] ryan_google: [09:14am] ryan_google: thanks moraes! we are too [09:14am] ikai_google: moraes: so am I [09:15am] gQuash: nickjohnson: I was really ansious to spectate awesome conversations (I'm not that smart to ask question, preffer to read) [09:15am] ikai_google: Hey Ryan, care to give us a preview? [09:15am] ryan_google: actually alfred's giving it, not me [09:16am] ryan_google: (alfred fuller; he's new to the datastore team since last i/o) [09:16am] moraes: "various features", aka cursors! [09:16am] lent: a question about backing and restore. The bulkloader dump and restore provides the basic dump and restore but in order to restore to the dumped state, the database has to be cleared out. Is there any plan to offer an operation which will clear out a database or a particular kind? [09:16am] ryan_google: heh. cursors are actually much sooner. we don't consider them next gen any more. the talk will be much more exotic. [09:16am] nickjohnson: lent: You don't have to clear the datastore - entities from the dump will simply overwrite existing ones. [09:16am] moraes: great, heh [09:17am] nickjohnson: As far as bulk delete goes, the task queue is your friend here. I've been planning to write a more robust mapping framework for this sort of task, but it's something anyone with sufficient determination could write. [09:17am] lent: nickjohnson: yes, but there are entities which may have been created after the dump which if left will lead to an inconsistent state [09:18am] nickjohnson: lent: Quite true [09:19am] ikai_google: lent: Yep, but that's a problem with any date migration [09:19am] ikai_google: lent: *data [09:19am] ryan_google: lent: we've had some thoughts about this for a while, including doing it at the bigtable level, we just haven't been able to prioritize it yet [09:19am] ryan_google: (fair warning all, "priorities" is kind of my answer to everything) [09:20am] gQuash: hahaha [09:20am] lent: nickjohnson: is bulk delete possible to do through remote_api or is the simpler way to just do it programatically through tasks [09:20am] ikai_google: lent: The solutions aren't great, either, MySQL's built in dumping mechanism locks the entire table ... it's a bit more difficult with our distributed store [09:20am] nickjohnson: lent: You can do it via remote_api, but doing it via tasks will use less quota and be faster. [09:22am] mcrady: I'm unable to create an embedded class that contains a collection. Is that a known issue? [09:22am] mcrady: embedded jdo class [09:22am] ryan_google: mcrady: is that an app engine specific q? do you mean, in JDO or JPA? [09:23am] ryan_google: ah [09:23am] ikai_google: mcrady: What is it a collection of? [09:23am] mcrady: even an arraylist doesn't seem to work [09:24am] ikai_google: mcrady: Hrm ... can you post a test case to the groups? [09:24am] lent: in some cases, we want to restore a dump to a different application and we have been using String encoded Key value as foreign keys in some of our kinds and we're running into a problem of the String encoded Key value being different in a different application. Is there any way to compute what the String encoded Key value would be in a different application id so that we can patch the dump file before restoring? [09:24am] mcrady: yes, sounds good [09:24am] ikai_google: mcrady: As well as the exact error? I know there are limitations to storing embedded classes. Whether we've done a good job documenting them is a different question [09:25am] nickjohnson: lent: Yes, just supply the '_app_id_namespace' argument to db.Key.from_path. Bear in mind that this is an implementation detail, though, so could change. [09:26am] lent: nickjohnson: what's the equivalent of that in Java? [09:27am] dennis_tw: got another data modeling question: i have a 'master-detail' type of relationship (like orders and order details). in a relational model, i would separate the details into a different table. but for gae, i'm thinking of putting everything in 1 entity (this will reduce the chance of a timeout since there is 1 entity instead of N). does this seem reasonable? have others done this? [09:27am] ryan_google: lent: an alternative would be to use Key.ToXml instead of converting it to a string, since that encoding is transparent, not opaque [09:27am] ryan_google: (maybe to_xml, not sure if we got that alias right) [09:27am] nickjohnson: ryan_google: He wants to use them as key names, though, and xml is probably a bit verbose for that. [09:27am] ikai_google: dennis_tw: Should be fine either way [09:28am] ikai_google: dennis_tw: If you split into 2 entities, you'd do 2 retrieve by keys (you would encode OrderDetails key to be generatable from Order's key). With one entity you do a single retrieve [09:28am] ikai_google: dennis_tw: Between 1 and 2 ... there isn't a whole lot of difference. [09:28am] ryan_google: nickjohnson: eh if it's just the key, they probably end up pretty similar lengths (sigh) [09:28am] ryan_google: lent: do reference properties not work? do we complain if you try to store a reference property to another app id? [09:29am] dennis_tw: actually, it's 1 vs (1+N) for N details [09:29am] ryan_google: i remember writing code for that recently, we may or may not allow it. if we don't, we actually might in 1.3.1. [09:29am] ryan_google: (we don't talk about it much though since it's such an unusual use case) [09:29am] ikai_google: dennis_tw: What's the order of N? 10? 100? 1000? [09:29am] dennis_tw: between 10 and 100 [09:29am] gQuash: did you code appengine on python ? [09:30am] Wooble: ryan_google: you might allow referencing entities owned by another application?? [09:30am] ryan_google: qQuash: there are lots of different parts in different languages: python, java, and c++ [09:30am] nickjohnson: Some of us wrote some of it. None of us wrote all of it. [09:31am] dennis_tw: this is one dimension of what i was eluding to before: are lots of small entities OK or better to pack them myself? [09:31am] ryan_google: nickjohnson: actually i wrote all of it. little known piece of trivia. [09:31am] nickjohnson: ryan_google: Even the small bits I wrote? [09:31am] ryan_google: Wooble: not sure if we do or not now. i know how to find out though. just a sec... [09:31am] ryan_google: nickjohnson: yup. like i said, little known. fortunately i'm modest! [09:31am] nickjohnson: ryan_google: indeed! [09:31am] ikai_google: dennis_tw: My suspicion is that the best design require some amount of moving away from normalization way of thinking [09:31am] gQuash: you love python ... [09:31am] ikai_google: dennis_tw: That is, you store most of the data you need in Order. Calculate sums, averages at write time [09:32am] lent: ryan_google: by reference properties, do you mean those determined through ownership (I'm using java)? I'm using java and those properties which are annoted as being keys or parent values do work, but where we just have arbitrary references in String fields to other keys, don't work because restore doesn't know that it is an encoded Key. [09:32am] ikai_google: dennis_tw: OrderDetails can be an additional entity type which you only retrieve when you really, really need [09:32am] ryan_google: dennis: ikai's rule of thumb is right. as for the q of one vs many, it depends on more info than we have. there's no single right answer. [09:33am] laheadle: what's the recommended way to enable eclipse to show me javadoc info about javax.jdo classes like PersistenceManager? [09:33am] ristoh1: when I do datastore get with get_by_key_name (multiple_keys), does that show up a single get or multiple gets? [09:33am] dennis_tw: yup, i'm trying to get a feel for gae data modeling. my current impression is that i should pack everything into 1 entity unless i want to index that data separately [09:33am] ryan_google: lent: probably ownership, yes. sorry, i'm not an expert on the java api. but yes, if you store encoded keys as straight strings, then we don't know to do anything special with them. [09:33am] nickjohnson: ristoh1: It does a single get on the backend [09:34am] nickjohnson: ristoh1: It's syntactic sugar, basically [09:34am] ristoh1: so, no need to run it inside transaction? [09:34am] ryan_google: dennis_tw: again, there's no single answer for all cases [09:35am] nickjohnson: ristoh1: Gets for multiple entity groups aren't transactional regardless of how you do them. [09:35am] ikai_google: dennis_tw: there's an advantage to using a single entity: you can do transactional things without mucking with keys and such. I think the jury is still out on "best practices with NoSQL" [09:35am] dennis_tw: good point [09:35am] gQuash: any recommended software technique to develop an app on GAE ? [09:35am] ikai_google: dennis_tw: er, mucking with keys, entity groups, etc [09:35am] ikai_google: gQuash: Java [09:35am] ryan_google: ok, so, we don't currently let you use reference properties that point to other app ids. from http://shell.appspot.com/ : [09:35am] ryan_google: >>> class Foo(db.Expando): [09:35am] ryan_google: x=db.ReferenceProperty() [09:35am] ryan_google: >>> Foo(x=db.Key.from_path('Foo', 1, _app_id_namespace='asdf')).save() [09:35am] ryan_google: Traceback (most recent call last): [09:35am] ryan_google: File "/base/data/home/apps/shell/ 1.335852500710379686/shell.py", line 267, in get [09:35am] ryan_google: exec compiled in statement_module.__dict__ [09:35am] ikai_google: gQuash: kidding! that's a really general question [09:35am] ryan_google: File "<string>", line 1, in <module> [09:35am] ryan_google: File "/base/python_lib/versions/1/google/ appengine/ext/db/__init__.py", line 797, in put [09:35am] ryan_google: return datastore.Put(self._entity) [09:35am] ryan_google: File "/base/python_lib/versions/1/google/ appengine/api/datastore.py", line 203, in Put [09:35am] ristoh1: nickjohnson: thanks! that explains my headache over the last few hours [09:35am] ryan_google: raise _ToDatastoreError(err) [09:36am] ryan_google: BadRequestError: app shell cannot access app asdf's data [09:36am] ryan_google: that *might* change in 1.3.1 but no promises [09:36am] gQuash: ikai_google: I love java ... likes python but don't know [09:36am] gQuash: I know its general... but which techinique you recommend ? [09:36am] nickjohnson: ristoh1: What are you doing that makes it important it be transactional? [09:36am] ryan_google: qQuash: definitely start with java then. learning one new thing at a time is generally a good idea, and app engine will be a new thing [09:36am] ryan_google: qQuash: we don't understand the question. what do you mean by "technique"? [09:37am] ikai_google: gQuash: When you're learning App Engine I have come to the conclusion that Python is easier [09:37am] Wooble: "write lots of bad code, then debug it until it works." [09:37am] gQuash: ryan_google: software engineering [09:37am] laheadle: When is 1.3.1 due out and is there a list available of features expected in it? [09:37am] ikai_google: gQuash: If you are new to Java, you are learning App Engine, JDO, JPA, Servlets, JUnit, crazy Java stuff [09:37am] gQuash: well ... I use GAE + GWT and I love it ... [09:38am] laheadle: qQuash me too. rock! [09:38am] gQuash: Yeap ... all that stuff ... so I get quite lost sometimes ! [09:38am] laheadle: qQuash what are you doing for serializing appengine objects over the wire to gwt? gilead? [09:39am] ristoh1: nickjohnson: I'm trying to map some many:many and 1:many relationships over some datasets with taskqueue, and trying to find ways optimize my task [09:39am] gQuash: I'm not advanced programmer ... just try GAE examples to few stuffs. That's why I'm here ... seeing what does people does.. [09:39am] nickjohnson: ristoh1: ah. Well, batching is always a good start [09:39am] johnvdenley: Wooble: thats what ive had to do! not clever, but it works! [09:40am] dennis_tw: another data modeling Q: list properties that are not indexed: is the property name stored for each list instance? would be kind of a waste but there might be some reason for it. if so, i should pack the data myself. [09:40am] ristoh1: nickjohnson: any pointers to perhaps a good blog article that covers something like that? [09:40am] ikai_google: dennis_tw: Yes, but it's Protobuf so it's ... semi-efficient [09:40am] dennis_tw: nice [09:40am] nickjohnson: ristoh1: I'm not quite sure what you're doing, so not really, no. [09:41am] gQuash: do you have any other language in mind for GAE ? [09:41am] ikai_google: dennis_tw: At any rate, I don't feel like the savings you'd get from doing so are worth the tradeoffs in time taken away from writing your application [09:41am] ikai_google: gQuash: I am partial to Scala. [09:41am] johnvdenley: do we have any news on the defer.deferrable thing? i need to programatically schedule tasks to be executed overnight and ive not found any obvious way to achieve this yet (though its possible im just being a newbie again!) [09:41am] nickjohnson: dennis_tw: Yes, they are stored for each instance - a list property is just a'multiply valued' property - one that appears multiple times in the entity [09:41am] ikai_google: gQuash: Groovy is also popular [09:41am] ryan_google: qQuash: new languages aren't a priority for us right now. we're excited about JVM-based languages though! [09:41am] dennis_tw: oops, did you mean that the prop name is stored N times?? [09:41am] ryan_google: scala, groovy, and ruby in particular, at least internally [09:41am] nickjohnson: johnvdenley: What specifically about deferred.defer? [09:41am] ryan_google: dennis_tw: http://code.google.com/appengine/articles/storage_breakdown.html [09:42am] nickjohnson: johnvdenley: You can use it for exactly that [09:42am] nickjohnson: dennis_tw: yes [09:42am] ryan_google: ristoh1: yes, lots. try http://code.google.com/appengine/articles/modeling.html among others [09:42am] ryan_google: also a few i/o talks [09:42am] gQuash: ryan_google: You have sorte really well your priorities that's great hahaha [09:42am] dennis_tw: yup, i looked at that article. and, wow, stored N times... hmmm [09:42am] gQuash: sorted* [09:43am] ryan_google: dennis_tw: it was a space/performance/cost/etc tradeoff [09:43am] ryan_google: gotta love tradeoffs! [09:43am] ryan_google: imho we're actually incredibly cost competitive on storage space [09:43am] dennis_tw: it's fine. just good for us developers to know. [09:43am] ikai_google: dennis_tw: Yeah, I wouldn't optimize for storage [09:43am] ryan_google: the datastore is comparable to (e.g.) amazon simpledb, but at 1/10th of the price [09:43am] johnvdenley: i meant about bringing deferred.defer into the core product [09:43am] ikai_google: dennis_tw: Your indexes are going to take up so much more space [09:44am] dennis_tw: i'm going to use store IDs only, not paths so i'm hoping indexes will be smaller [09:44am] gQuash: what about more examples for java in GAE examples gallery ? [09:44am] nickjohnson: ryan_google: And with a much better type model [09:45am] nickjohnson: johnvdenley: I'm not sure how much more part of the core product it could be? [09:45am] nickjohnson: It's as much so as, say, remote_api, or ext.db. [09:45am] ikai_google: gQuash: We could probably do a better job of surfacing and organizing this page http://groups.google.com/group/google-appengine/web/google-app-engine-open-source-projects [09:45am] ikai_google: gQuash: I've found that one of the best ways to learn is to study and contribute to open source [09:45am] johnvdenley: nickjohnson: perhaps im being a newbie again then, or perhaps its in python, but not java? [09:46am] lent: any longer term plans to offer an option to use a more conventional (relational) database in addition to one that's based on BigTable? [09:46am] nickjohnson: johnvdenley: That's true. There's a patch from an external contributor pending for a Java version, though naturally it can't operate in exactly the same manner. [09:46am] schuey: nickjohnson: i also want a better deferred.defer, or at least a longer timeout on background processes that are processing large amounts of data. [09:46am] gQuash: ikai_google: I didn't know about that page. Thanks !!! ... and I agree with you. Open Source it's really great way [09:46am] nickjohnson: schuey: The 30 second request timeout is independent of deferred, or any other library. [09:47am] ikai_google: gQuash: I'm also going to take this time to talk about how AWESOME Github is [09:47am] • nickjohnson gasps in mock horror at ikai_google [09:47am] schuey: nickjohnson: so there are no plans to extend that time? [09:47am] gQuash: ikai_google: I'll be there reading ... haha. Need too many things to learn [09:47am] ikai_google: gQuash: We get so many requests from people whose computers crash and they ask us if we can help them download their code from appspot ... and the answer is no. So ... use source control [09:47am] nickjohnson: schuey: I couldn't say. I'm just pointing out that it's not related specifically to the deferred library. [09:48am] gQuash: ikai_google: I have my full eclipse GAE +GWT + VISUALIZATION API on an USB stick... I'll die if get lost [09:48am] ryan_google: [ikai_google has been removed by the google brand police, who would like to reaffirm that google code project hosting is a very very good product.] [09:48am] ryan_google: [...very.] [09:49am] ryan_google: [(very)] [09:49am] gQuash: *veeeeeery* [09:49am] gQuash: haha [09:50am] johnvdenley: nickjohnson, ah, yes, so thats my issue, i dont know python at all, so i dont know how it works there or why it "naturally cant operate in exactly the same manner" but yes i can see your reply on the 26th November in the groups conversation, hence my question for an update [09:50am] ikai_google: HAHAHA [09:50am] ikai_google: learning Mercurial has been on my "to do" list [09:50am] ryan_google: wait i thought that guy was gone [09:50am] ikai_google: after using Git, you go back to SVN and go man, this sucks [09:50am] ryan_google: they're letting anyone in here these days [09:50am] • ikai_google is back with a VENGEANCE [09:50am] gQuash: hahaha ... [09:50am] • ryan_google cowers [09:51am] gQuash: preffer code hosting... it's easier for newbies [09:51am] gQuash: like me [09:51am] johnvdenley: @ryan_google... ooo, i think ill look into google code project hosting [09:52am] ikai_google: quick poll here (to fill the dead air): what kind of source control is everyone using here? what's your favorite? [09:52am] ikai_google: <--- Perforce. Git [09:52am] moraes: <- hg [09:52am] nickjohnson: Perforce because I need to, Git because I want to. [09:52am] Wooble: svn, occasionally cvs on local machine [09:52am] dennis_tw: i'm still a bit taken back about the list properties names being stored N times: did you guys mean that for INDEXED list properties that's true. If my list props are not indexed then the names are stored N times?? [09:52am] ryan_google: ikai_google: <-- svn. or rsync. [09:53am] ikai_google: hahaha rsync [09:53am] moraes: please people, you can use git, but stop using that damn slow and unreliable github. [09:53am] moraes: [09:53am] ikai_google: dennis_tw: Nope, each property is stored, but it's stored as a protobuff [09:53am] ikai_google: dennis_tw: The reason is that the datastore is typeless [09:53am] ikai_google: dennis_tw: Err not typeless, schemaless [09:53am] gQuash: <--- google code hosting ? hahaha [09:54am] dennis_tw: so it does not know about lists either ... it's just a list of properties [09:54am] nickjohnson: dennis_tw: Check out the article linked earlier on how the datastore storage works. A list foo=[bar,baz] is exactly identical to two properties foo=bar,foo=baz. [09:54am] ikai_google: dennis_tw: So one entity could have a list property named "skills" and another named "experience" [09:54am] dennis_tw: ok, just wanted to make sure i heard right, even for non-indexed properties [09:55am] lent: any longer term plans to offer a more conventional (relational) database in addition to one based on BigTable? That would open up a large market for you. [09:55am] ikai_google: dennis_tw: It's one of the tradeoffs of storing data in a schemaless key value store [09:55am] dennis_tw: got it [09:56am] dennis_tw: actually, packing and unpacking myself is not a big deal. [09:56am] ryan_google: lent: there's definitely been lots of discussion about that externally at least, and we're paying attention. we're very aware that lots of people don't really need to scale, e.g. internal apps for SMBs, but they do want more of the power of an RDBMS. the core app engine team is focused on app engine itself, but it's definitely on our radar. [09:56am] ikai_google: dennis_tw: This is semi related. But it shows the challenges of using a K/V store http://bret.appspot.com/entry/how-friendfeed-uses-mysql [09:57am] gQuash: not a priority [09:57am] gQuash: hehe [09:57am] dennis_tw: cool, i like bret's stuff [09:59am] ikai_google: dennis_tw: Anyway, if I were you I wouldn't bother saving that space [09:59am] ryan_google: +1! [09:59am] dennis_tw: yeah, indexes will use most space, right? [10:00am] ikai_google: dennis_tw: Even without indexes I'm willing to bet your time implementing and debugging > storage space $ [10:01am] ikai_google: dennis_tw: Your time is valuable to us [10:01am] ryan_google: +2 [10:01am] ikai_google: And hey! That's the end of our office hours this time [10:01am] dennis_tw: makes sense. but it's good to know by having these chats! thx! [10:01am] ikai_google: We'll have office hours again in 2 weeks. Join our Google Groups! Follow us on Twitter! http://www.twitter.com/app_engine -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
