[google-appengine] _ah/admin - stability, seems to crash a lot
I noticed the '_ah/admin' admin data-store viewer tends to crash a lot and often requires an appEngine restart, does anyone else have this problem? I have this problem on both Win32 and Mac64. If I perform a fresh start of appEngine and then run '_ah/admin' it always runs fine; the problem I'm having occurs 'after' I run my user/ application code followed by running '_ah/admin'. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: App updates are taking forever...
Luís, I am not sure this would be a good idea, since indexes are global to the app. How would you handle the case when you are just uploading an index and not a version? Perhaps your rule would only apply when uploading indexes with a new version? Robert 2010/6/27 Luís Marques : > On Jun 26, 11:19 am, Jody Belka wrote: >> What you can do of course, is deploy the new version but don't make it live >> until the indexes are ready > > That is true, but perhaps a feature proposal should be made for the > following. Add an option for clients not to be served the updated > version until all the new indexes are built. Make that option on by > default. > > Luís > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Datastore design question
The answer to this question depends on several other questions too. For example: 1) how many items are you expecting? 2) how often do a users items change, and how do they get changed (the user or something else)? 3) what type of querying / reporting do you need to be able to do? 4) what type of information would you need to store about each users item? Robert On Sun, Jun 27, 2010 at 9:59 PM, Tom Wu wrote: > Hi Luís, > > If you have a "User" model, with a "items" heavy Property like TextProperty > which is not necessary for every query. > Which db structure is better ? > > Best Regards > Tom Wu > > > 2010/6/28 Luís Marques >> >> Hello Phil, >> >> Anyone correct me if I'm wrong but you use one of the following >> options, among others. >> >> You can have a "User" model, with a "items" ListProperty. The pros: if >> you have the key for the user (e.g. you can use the email), then with >> a simple get (faster than a query) you can retrieve the entity >> including the items. Writing a new object is transactional. The cons: >> you must always get a set the entire user entity. The entity might get >> large if there are a lot of items. If you use custom indexes with the >> "items" property the index might get large quickly, especially if you >> use more than one ListProperty (exploding indexes). >> >> You can have a "User" model, and a "UserItem" whose parent is the >> User, making it part of a transaction group. Cons: you have limited >> QPS writes to the transaction group (about 1-10 QPS), but that's the >> same limitation you have to a standalone User with items. You have to >> query (seek) the items, although I wouldn't call that a problem >> without profile data. Pros: you can write individual UserItems. You >> can have transactions for the user and the items s/he has. >> >> You can have a "User" model and a "UserItem" with a User key >> reference. Cons: you can no longer have a transaction depending on the >> user and the items s/he has. Pros: you can write individual items >> without write collisions to the User. >> >> You can have a "User" and a flat list (e.g. text, non-indexed) of the >> corresponding items. Cons: limits the queries you can make, have to >> update all items at once. Pros: better index performance. >> >> Etc. I hope it was helpful and correct? >> >> Best regards, >> Luís >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appeng...@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Datastore design question
Hi Luís, If you have a "User" model, with a "items" heavy Property like TextProperty which is not necessary for every query. Which db structure is better ? Best Regards Tom Wu 2010/6/28 Luís Marques > Hello Phil, > > Anyone correct me if I'm wrong but you use one of the following > options, among others. > > You can have a "User" model, with a "items" ListProperty. The pros: if > you have the key for the user (e.g. you can use the email), then with > a simple get (faster than a query) you can retrieve the entity > including the items. Writing a new object is transactional. The cons: > you must always get a set the entire user entity. The entity might get > large if there are a lot of items. If you use custom indexes with the > "items" property the index might get large quickly, especially if you > use more than one ListProperty (exploding indexes). > > You can have a "User" model, and a "UserItem" whose parent is the > User, making it part of a transaction group. Cons: you have limited > QPS writes to the transaction group (about 1-10 QPS), but that's the > same limitation you have to a standalone User with items. You have to > query (seek) the items, although I wouldn't call that a problem > without profile data. Pros: you can write individual UserItems. You > can have transactions for the user and the items s/he has. > > You can have a "User" model and a "UserItem" with a User key > reference. Cons: you can no longer have a transaction depending on the > user and the items s/he has. Pros: you can write individual items > without write collisions to the User. > > You can have a "User" and a flat list (e.g. text, non-indexed) of the > corresponding items. Cons: limits the queries you can make, have to > update all items at once. Pros: better index performance. > > Etc. I hope it was helpful and correct? > > Best regards, > Luís > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: What's the best way to enable blobstore without getting charged?
On Jun 25, 4:46 pm, Mark Ivey wrote: > (I want to use blobstore for its API. The files I want to serve are > actually small so I can't imagine going over the 1GB free quota. Given > this it seem strange that I have to enable billing to use blobstore.) Is there any disadvantage to using a db BlobProperty? They are not indexed, and if the files are small they should be within the datastore RPC limits. Luís -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: App updates are taking forever...
On Jun 26, 11:19 am, Jody Belka wrote: > What you can do of course, is deploy the new version but don't make it live > until the indexes are ready That is true, but perhaps a feature proposal should be made for the following. Add an option for clients not to be served the updated version until all the new indexes are built. Make that option on by default. Luís -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Investigating usage costs, profiling
Interesting analysis, really appreciate it. Some responses to your points: - I'm looking at some other examples of the stats and I don't have that big break between get/set, so I'm thinking that's a one-time thing. Going to continue looking at the code to see if I can find that what could have been. - I unfortunately can't combine the gets, they are all doing different things in different places, but I'll dig deeper and maybe find some opportunities. - I'll definitely post any answer I get to this here. Thanks again for the advice...hopefully someone from Google gives us some insight On Jun 27, 6:13 pm, Luís Marques wrote: > Hi Lenny, > > If you find the answer to this one please post it here, since it seems > quite a few interesting questions go unanswered in this group. > > I'm no expert on this matter, but there's something hard to understand > indeed. The CPU cost (without API) makes sense if you think about the > following. In the middle of the graph, between 20 ms / 21 ms get / set > memcache calls there is a gap, about 600 ms in length. That is your > could executing, without RPCs, so that explains about 0.6s of the 2s > of CPU usage. Presumably, the remaining 1.4s would be code executing > after the last memcache get RPC, and so wouldn't be visible on the > timeline (perhaps you've got something slow there?). > > Now, what doesn't make sense to me is the CPU + API grand total. There > are 66 ms of RPC API summed up at the RPC total. That makes sense, > it's the two datastore calls, a get and a query. But where does the > remaining API time come from? I thought API times only came from RPC > calls. It seems something else must be responsible for that. I'm > curious what. > > BTW, can't you merge the memcache gets, sets and deletes in only three > RPC calls? E.g. memcache.get_multi()? > > On Jun 27, 11:54 pm, Lenny Rachitsky wrote: > > > > > I'm having trouble understanding where the usage costs are coming > > from, specifically arround the CPU/API usage. I'm looking at the > > following app stats example: > > >http://img.skitch.com/20100627-x6s7s988y2kg2at3kg3t6984ig.png > > > To me this looks rather well optimized, except when you get to the > > "Grand Total" which has a huge cpu+api cost. Can anyone help me > > understand what could be causing this? My costs are growing quickly, > > and I'm considering some major revamping to deal with this situtation > > I can't explain. > > > Any help would be much appreaciated, > > Lenny -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Datastore design question
Hello Phil, Anyone correct me if I'm wrong but you use one of the following options, among others. You can have a "User" model, with a "items" ListProperty. The pros: if you have the key for the user (e.g. you can use the email), then with a simple get (faster than a query) you can retrieve the entity including the items. Writing a new object is transactional. The cons: you must always get a set the entire user entity. The entity might get large if there are a lot of items. If you use custom indexes with the "items" property the index might get large quickly, especially if you use more than one ListProperty (exploding indexes). You can have a "User" model, and a "UserItem" whose parent is the User, making it part of a transaction group. Cons: you have limited QPS writes to the transaction group (about 1-10 QPS), but that's the same limitation you have to a standalone User with items. You have to query (seek) the items, although I wouldn't call that a problem without profile data. Pros: you can write individual UserItems. You can have transactions for the user and the items s/he has. You can have a "User" model and a "UserItem" with a User key reference. Cons: you can no longer have a transaction depending on the user and the items s/he has. Pros: you can write individual items without write collisions to the User. You can have a "User" and a flat list (e.g. text, non-indexed) of the corresponding items. Cons: limits the queries you can make, have to update all items at once. Pros: better index performance. Etc. I hope it was helpful and correct? Best regards, Luís -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: AppEngineJS base on python ?
Hello Tom, As good as the language is, Python isn't the best one to implement most other languages, due to its speed, so you don't see many languages or DSLs implemented on top of Python (PyPy being a notable exception). So I wouldn't much expect to find a Python JS implementation, upon which an App Engine JS framework could be built. Best regards, Luís -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Investigating usage costs, profiling
Hi Lenny, If you find the answer to this one please post it here, since it seems quite a few interesting questions go unanswered in this group. I'm no expert on this matter, but there's something hard to understand indeed. The CPU cost (without API) makes sense if you think about the following. In the middle of the graph, between 20 ms / 21 ms get / set memcache calls there is a gap, about 600 ms in length. That is your could executing, without RPCs, so that explains about 0.6s of the 2s of CPU usage. Presumably, the remaining 1.4s would be code executing after the last memcache get RPC, and so wouldn't be visible on the timeline (perhaps you've got something slow there?). Now, what doesn't make sense to me is the CPU + API grand total. There are 66 ms of RPC API summed up at the RPC total. That makes sense, it's the two datastore calls, a get and a query. But where does the remaining API time come from? I thought API times only came from RPC calls. It seems something else must be responsible for that. I'm curious what. BTW, can't you merge the memcache gets, sets and deletes in only three RPC calls? E.g. memcache.get_multi()? On Jun 27, 11:54 pm, Lenny Rachitsky wrote: > I'm having trouble understanding where the usage costs are coming > from, specifically arround the CPU/API usage. I'm looking at the > following app stats example: > > http://img.skitch.com/20100627-x6s7s988y2kg2at3kg3t6984ig.png > > To me this looks rather well optimized, except when you get to the > "Grand Total" which has a huge cpu+api cost. Can anyone help me > understand what could be causing this? My costs are growing quickly, > and I'm considering some major revamping to deal with this situtation > I can't explain. > > Any help would be much appreaciated, > Lenny -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Investigating usage costs, profiling
I'm having trouble understanding where the usage costs are coming from, specifically arround the CPU/API usage. I'm looking at the following app stats example: http://img.skitch.com/20100627-x6s7s988y2kg2at3kg3t6984ig.png To me this looks rather well optimized, except when you get to the "Grand Total" which has a huge cpu+api cost. Can anyone help me understand what could be causing this? My costs are growing quickly, and I'm considering some major revamping to deal with this situtation I can't explain. Any help would be much appreaciated, Lenny -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Datastore design question
I'm using JDO to connect to the datastore and wondering how I should design my tables. I have users who sign up. Users have items. In the RDMS world I'd have a User table and a Items table and the Items table would have a field of user_id which would join with a primary key in the User table. However, with bigtable I thought you could append values to a cell. In this situation I might just have my User table with a column "items". For each item that gets added to the user over time I could then write something to that user's "items" column and it would be appended to the other items also written to that column. Is there an atomic way to do this in the AppEngine datastore? I could imagine doing it through a transactional read-write operation, but locking on every write like that seems like a bad idea. I'm looking to squeeze everything into the Users table because I assume the read will be much faster if I can do a lookup on a particular row as opposed to a table scan query on a separate items table, even if I indexed the items table on user_id. Thanks, Phil -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] AppEngineJS base on python ?
Hi All, Today I found AppEngineJS a wonderful framework for gae(which base on java). Is any framework like AppEngineJS but base on python ? Thanks in advance. Best Regards Tom Wu -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
