[google-appengine] Re: How does this CPU limit stuff really work?
http://code.google.com/p/googleappengine/issues/detail?id=240 is an issue related to the issue you raised in your post. On Jan 16, 1:31 pm, boson dan.kam...@gmail.com wrote: After a URL in the request log are the following fields (example): 200 1584ms 2571ms-cpu 4kb Am I correct in believing these are: HTTP response code, real time, CPU time, and response size? Now I get various orange and red warnings depending on the ms-cpu value (almost always when I create an entity, and often just for writing one - even simple ones). But I haven't ever seen any increase in my High CPU Requests on the quota page. Is this normal? I've read these pages, but I'm still not sure what's going on here.http://code.google.com/appengine/docs/quotas.html#Resourceshttp://code.google.com/appengine/articles/quotas.html The first page says Datastore CPU time does not count towards the high CPU threshold, but apparently it IS included in the value in the request log where the warning comes from. Is there a breakdown anywhere of whether this time is in user CPU or DS CPU? Also with the amount of CPU time reported in the request log, it seems that even modest traffic will kill the CPU Time quota, as even the most basic DS write operations with a handful of properties seem to often take ~1000ms-cpu (even when the request itself takes less time). I'm not even sure what questions to ask here, so if you can answer the obvious unasked ones too please do so. I'm rather confused by all this, but am hoping it's more obvious than it seems right now... Thank you. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Hiring contract programmers with Google App Engine / Python / AJAX experience
A number of them yes. http://codecrafter.wordpress.com/tag/appengine/ . That link doesn't go to a single post that sums up all of the things to consider, but rather a tag I put on all of my blog posts that speak to the issue. On Dec 6, 12:00 pm, mytemp [EMAIL PROTECTED] wrote: Yes, it should be well suited... do you have specific things I should consider? Thx! On Dec 5, 5:30 pm, Josh Heitzman [EMAIL PROTECTED] wrote: Are you certain that your app is well suited to app engine? If not, you may want to first hire a consultant to evaluate your apps suitability to app engine. On Dec 5, 3:59 pm, mytemp [EMAIL PROTECTED] wrote: I want to implement a website on Google App engine, front end is AJAX with some web services/API calls, back-end is python on app-engine with some math/algorithms/DB access and constant back and forth interaction on a timer with minimal user interaction. Prefer everything be app engine and not host any servers ;-) Please mail me experience, example sites, and expected rate. Location is flexible, but preference is bay area. Thanks!- Hide quoted text - - Show quoted text - --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: How far can Google take this thing?
I think the answer really depends on how much Google is willing to invest in GAE. Eventually could be two years from now or ten years from now, but if its the later they may miss the boat considering there are numerous big names and no names all moving into this space. One of those big names is Microsoft, and while they are just getting into this space, one thing they do really well is getting application developers to target their platforms. In my opinion Google has a long way to go in that respect. After using GAE's platform for the last few months I can honestly say that the only thing I like about it is that I haven't had to pay for the hosting of my app. Sure getting an app going that fits with-in the quite limited confines of what GAE supports well is quite easy, but it doesn't take much to smack into GAE's walls and then it becomes very much a square peg in round whole experience where doing things that would be quite simple on other platforms is quite difficult on GAE or even impossible without getting shutdown due to going over CPU quota. Josh On Dec 5, 10:42 am, rvjcallanan [EMAIL PROTECTED] wrote: I am about to take the GAE plunge (at least in the experimentation sense). I understand the current irritations and I am hopeful that these will be overcome in due course But I am very curious how far Google can take this thing... A key question on everyone's mind: Can we assume that GAE developers will eventually be able to produce GAE apps with similar complexity, reliability, scalability and performance ballparks as Gmail, subject of course to hosting fees? If the answer to that question is YES, then I am am convinced that GAE will eventually be able to host sophisticated financial applications that are not currently in the GAE sweetspot, e.g. accounts, payroll, etc Or would it be more realistic to assume that GAE developers will never really be able to leverage what Gmail's developers can leverage? Looking beyond the Gmail comparison, I see lots of problems with the GAE datastore for financial applications e.g. the absence of joins, aggregation, etc. I understand that these limitations are inherent to the BigTable paradigm, yet I already see posts by developers showing how these limitations can be overcome. Solutions tend to revolve around de-normalisation and other forms of data redundancy together with a sizable smattering of code trickery. All very, very botchy and alien to the GAE philosophy of removing much of the the tedium of web development. I am wondering if it will ever be possible to write an abstraction layer that will present the underlying GAE datastore as an SQL database albeit at a cost in terms of data efficiency, CPU cycles and bandwidth...or is this completely missing the point? Bear in mind that I am thinking a few years down the road. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Why limit request time (as opposed to cpu usage)?
Apparently GAE's infrastructure will just continue to spawn more processes as needed, so your app actually should continue to scale even if it waits 1 minute instead of 10 seconds for URLFetches, but operating your app will be more costly as memory usage time will be increased by a factor of 6. Currently google isn't tracking memory usage time though (at least that we know of), so they couldn't actually charge for it at present. On Dec 1, 11:14 pm, JM Ibanez [EMAIL PROTECTED] wrote: On Tue, Dec 2, 2008 at 3:08 PM, Amir Michail [EMAIL PROTECTED] wrote: Hi, When doing a urlfetch, it may take quite a long time. As this is not a CPU usage issue, what's the point of limiting the request time anyway? Why not allow the request to go on for a minute or so? As I understand it, the thread doing the fetch will be blocked and cannot service any other requests until it finishes the fetch. Now, if a lot of threads are blocked waiting for that resource to be fetched, you will not be able to scale -- as it would be effectively an inadvertent denial of service for your app (and the whole infrastructure). -- JM Ibanez -- The worst government is often the most moral. One composed of cynics is often very tolerant and humane. But when fanatics are on top there is no limit to oppression. -- H. L. Mencken -http://www.livejournal.com/~jmibanez/http://www.mycgiserver.com/~butiki/ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Observing GAE Scalability (or trying to ...)
The lack of any long running server side process also precludes doing any real time MMOGs. With many turn based web based MMOGs there is no way to partition the game state into entities such that executing all player commands only requires updating entities in one entity group during the execution of the command, so its necessary to implement a cross-entity group transaction layer on top of the GAE datastore, but its currently unclear to me that this can actually be done while staying under GAEs quotas. On Nov 16, 11:52 am, Ben Nevile [EMAIL PROTECTED] wrote: Hi Ross, Well, Google App Engine would be a terrible place to serve most massively multiplayer games. I hope that's not what got you interested in GAE. Pretend I'm dumb... no, it's okay, I *am* dumb when it comes to MMOG. Why would GAE be a terrible place to serve these? The type of communication required? Ben --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] CapabilityDisabledError: Datastore writes are temporarily disabled.
Just started seeing this exception thrown from any attempt to write to the datastore. Anyone else seeing this at present, or is it just me? Thanks, Josh Heitzman --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: CapabilityDisabledError: Datastore writes are temporarily disabled.
Good call. Looks like there scheduled maintenance now per http://groups.google.com/group/google-appengine-downtime-notify/browse_frm/thread/455b69c7bf3b0de Josh On Nov 15, 2:24 pm, russ.au [EMAIL PROTECTED] wrote: I'm getting this exception in my app engine app : CapabilityDisabledError: Datastore writes are temporarily disabled. A search of this group suggest this only happens during scheduled maintenance?http://groups.google.com/group/google-appengine/browse_thread/thread/... The data viewer is also giving me a Server Error. Are other people seeing the same errors? Russ --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Tips on Optimizing Writes?
Jeff you said: The CPU warning in the log is based on runtime CPU, while the displayed CPU warning in the log is based on runtime and datastore CPU. but you wrote in the log is based on both times. I'd assume one of those is type, so could you clarify which if the mcycle consumption value shown in the per request logs the runtime mcycles or the runtime and datastore mcycles? If the mcycle value shown in the per request logs is only the runtime mcycles then issue 786 is not a duplicate of 814 as the values listed in 786 were taken from the per request logs. Thanks, Josh Heitzman On Oct 29, 11:27 am, Jeff S [EMAIL PROTECTED] wrote: Hi Waldemar, On Oct 28, 3:22 pm, Waldemar Kornewald [EMAIL PROTECTED] wrote: Hi Jeff, On 28 Okt., 21:11, Jeff S [EMAIL PROTECTED] wrote: Our queries and indices are rather simple. I should clarify: The particular problem I'm talking about is a view that currently has a response time of about 360ms and almost all of the time (280ms) is spent on datastore put()/fetch()/get(). IOW, runtime CPU usage is low. If you only read data then 360ms would probably consume less than 1500mcycles, but since this view primarily writes to the datastore it consumes more than 10,000mcycles! That's really bothering me because I already get yellow warnings for views that consume more than 1000mcycles and here I have red warnings. How am I supposed to implement a simple write operation if it eats mcycles so quickly? Can I just ignore the warning since only runtime cpu counts and we primarily consume datastore cpu in this case? From your description, it sounds like the datastore CPU usage need not concern you. If the CPU Usage listed on the admin console is at an acceptable level and the logs do not contain large amounts of high CPU request warnings, then your app should be okay. Well, that's the problem. The dashboard shows a warning sign (!) and very high Avg CPU usage for that URL, but the logs don't mention anything at all for that URL. So, which is more correct? The dashboard or the logs? :) The two are currently measuring different things. The CPU warning in the log is based on runtime CPU, while the displayed CPU warning in the log is based on runtime and datastore CPU. Thank you, Jeff Normally, if I consume too much CPU I get an explicit warning *message* in the logs: 10-25 06:48AM 43.525 / 200 802ms 2754mcycles ! 3kb [...snip...] This request used a high amount of CPU, and was roughly 2.6 times over the average request CPU limit. High CPU requests have a small quota, and if you exceed this quota, your app will be temporarily disabled. As I said, that warning message doesn't appear. But if I print something to the logs I can see in the header of the log entry that the request consumes a huge amount of CPU (e.g., 15000mcycles !, please note the ! warning *sign*). Why do I *not* get a warning *message* although I can provoke a warning *sign* if (and only if) I print something to the logs? Also, why does the CPU quota usage go up and down so quickly? A few minutes ago I had 40gcycles used and after I visited that high-CPU URL four times it went up to 85gcycles and a few minutes later it went back to 0. I thought this was a *24h* moving window? Don't get me wrong. I don't mind quota usage going back to zero every few minutes (or seconds would be fine, too). :) Thanks! Bye, Waldemar Kornewald --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Tips on Optimizing Writes?
If you can't reduce the number of entities, then all you have left is to reduce the number of indexed fields on the entities and the number of elements in any list fields on the entities. That said you can't put two entities with only a single non-indexed property each in less then 1000 mcycles. See http://code.google.com/p/googleappengine/issues/detail?id=786 On Oct 23, 6:57 pm, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: Hi, My application get a lot of high-CPU usage warnings on the production server. I have spent the past few weeks profiling and optimizing my production code as suggested. From profiler data, I know that 99% of processor time is spent in Model.put or db.put. As a result I have been aggressively eliminating unnecessary or duplicate write operations. For instance in my most processor intensive http request, I have cut down the number of entities I update from 5 to a varying number between 0 and 3 depending on what absolutely must be updated. I also batch writes by using db.put() instead of separately calling Model.put(). As a result, I have cut the number of mcycles down from 7000+ to an average of around 2000 mcycles. However, I still get high cpu warnings on one http request, because it absolutely needs to update 3 separate entities (two node entities in a directed graph, and a sharded counter entity). So my question is, what else can I do to further optimize this specific section of code? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High CPU Warning on module load
As I understand it loaded modules are cached, so if your have two scripts importing the same modules for the handling of two different requests the modules will only get loaded once so long as that particular process instance is running. On Oct 21, 2:22 pm, Adam [EMAIL PROTECTED] wrote: So given that I need modules a,b and c for all of my request handlers, I'm better off having fewer larger modules rather than a script for each page? A On Oct 21, 4:59 pm, Marzia Niccolai [EMAIL PROTECTED] wrote: Hi Adam, The biggest thing you can do to in this case is don't import large modules until you need them. If your app has been idle for awhile, it will take some CPU to warm up the Python interpreter in terms of import costs, but not loading big modules until/unless they are needed, you will only incur this cost when necessary. It should also be noted that sometimes it's expected that a request might take more CPU than average. The key is to limit those requests overall. If you are interested in optimizing your app, I would suggest watching 'Building Scalable Web Applications', one of our Google I/O talks:http://sites.google.com/site/io/building-scalable-web-applications-wi... -Marzia On Tue, Oct 21, 2008 at 1:54 PM, Adam [EMAIL PROTECTED] wrote: I'm getting high cpu warnings when I hit the main page of my app (www.fyood.com) for the first time after being idle for 10-20 minutes. Given that the warning isn't there the second time (250 v.s. 2500 mega cycles), and there's no datastore activity, it seems like this is the cost of creating the objects, loading imports etc. Anyone have a good strategy for dealing with this, or a good handle on which pieces are more expensive. e.g. should I create smaller classes, fewer classes, less but larger modules, more smaller modules etc. do less regex compiles etc. Also, any pointers to specifics on the high cpu quota? e.g. 3 times/ hour, 3 times/minute etc. A --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: High CPU usage for put()
It is not just you. An issue has be opened http://code.google.com/p/googleappengine/issues/detail?id=786 but the GAE team has yet to acknowledge it or response to other posts on the subject on this discussion group. On Oct 19, 10:18 pm, MajorProgamming [EMAIL PROTECTED] wrote: Is it just me or do puts take up a tremendous amount of CPU power. I was looking over my logs and found that many of my requests which updated a record in the database were marked as using a high amount of cpu. (such as 2139/2044/2283/etc mcycles). Is this normal or am I doing something wrong? Just to give you a overview of what this particular request does (although I noticed this on a lot of requests on another application I am developing), here is the basic outline of the code: (excerpts) #start off with get or insert (variable p is passed from the request query string) r = Users.get_or_insert(key_name='p:%s' % p, number=p) #the app then goes through about 8 if/elif statements, and depending will set: r.aproperty=True #(or false) #and then we store the change r.put() So how much more can it be optimized!?! Thanks for your help --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: ROR
Not at present. Currently only Python is supported. On Oct 17, 11:54 pm, amshuhu [EMAIL PROTECTED] wrote: Its possible Ruby coding in Google App Engine ? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Feature Request?
I wouldn't want the item deleted until I had done whatever processing I needed to do with its data to ensure it didn't get lost due to a raised exception or crashed machine. On Oct 17, 9:18 am, Ethan Post [EMAIL PROTECTED] wrote: A common task (queueing) is fetchitem...delete the item once it has been fetched. Right now that requires a get/fetch and a call to db.delete at some point. It would be nice if there was a particular attribute that could be set in the get which would automatically delete the item once it has been fetched. Google could figure out the most efficient mechanism for doing this on the backend and we end up saving mcycles by not having to call the delete for this type of db request. Stupid? Not stupid? Would anyone else like this feature? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Datastore storage recommendation...
Haven't tried it, but I found that a class derived from Model with one attribute of type BlobProperty has the same baseline time as one with an empty StringListProperty. Further both of those are slightly faster then a Model with an single IntegerProperty. This makes sense as I wouldn't expect the BlobProperty to do anything with its data and I would expect StringListProperty to get no further then checking the number of items in the list in its processing. On Oct 17, 7:25 am, David koblas [EMAIL PROTECTED] wrote: That's at least interesting from an update standpoint. If I read that correctly the biggest overhead is the DS Put operation, so the optimization of moving everything to single entry ID/Key/Value stores would cause problems during updates. Though as a difference I'm using the Entity object directly not the Django Model abstraction. Was there any noted performance differences with that test (eg. Are the *Property bits using a lot of time) Josh Heitzman wrote: Seehttp://groups.google.com/group/google-appengine/browse_thread/thread/... regarding the mcycle consumption impact of having lots of fields. On Oct 16, 9:11 pm, David koblas [EMAIL PROTECTED] wrote: I'm storing application preferences in the datastore, they come in a variety of types. As I see it I've got three choices for implementation: * Name / Blob -- general has a json encoded object of all of the sub-settings * Name / ID / Lots of Fields -- Where ID = general or user (etc.) and different fields in each entity * Name / ID / Key / Value -- Where you query out all of the user settings and re-construct an object from it. My gut feel is that #2 is preferable, since it avoids double encoding the data and doing a query out of a large number of records in the later case. Thanks, --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: 1000 mcycles to update a single entity
I've experimented with pickling and found that in about 810 mcycles
are consumed getting the entity (just one BlobProperty), unpickling,
pickling, and then putting the entity. Of those mcycles most (~730)
go into the put operation.
This only leaves 190 mcycles to actually do the work of parsing the
request, processing data, and constructing a result. Even if you can
manage to stay under 1000 mcycles you can only ever put one entity per
request and stay under 1000 mcycles and I found the putting a second
entity basically doubles the get, pickling, and put times.
The sad thing is the actual runtime of putting multiple entities isn't
very long. I've seen 15000 mcycle be consumed by a request that took
less then two seconds to actually get rendered in my browser.
So this 1000 mcycle soft cap seems out of whack with the reality of
how many mcycles are consumed by put data into the store.
While the soft cap can be gotten around by breaking up the processing
into multiple requests behind the scenes, this will result in higher
mcycle consumption to process the user command as well as longer
response time which just isn't good for my bottom line (since we'll be
paying for overall mcycle consumption) or my user satisfaction as
anyone using a high latency connection see it many seconds longer to
render a page. For example someone on using satellite will probably
see about a 4 second delay for every chunk the processing is broken
into.
On Oct 16, 12:20 am, Andy Freeman [EMAIL PROTECTED] wrote:
Does this do what you'd like?
import copy
import pickle
# should work on any pickle able datatype
# tested with both dicts and lists, works with unicode keys, data,
elements
# empty() is arguably wrong, but what's correct?
# choices probably doesn't make any sense here, but
# readable=True is for folks who want to look at pickle encoded data.
class PickleProperty(db.Property):
def __init__(self, readable=False, *args, **kwds):
if readable:
self._readable = 0
self.data_type = db.Text
else:
self._readable = -1
self.data_type = db.Blob
super(PickleProperty, self).__init__(*args, **kwds)
# as with db.ListProperty, don't want any static sharing
def default_value(self):
return copy.deepcopy(self.default)
# if value is true or the same type as the default value, it is
# assumed to be reasonable even if required is set.
def empty(self, value):
return not (value or (isinstance(value, type(self.default
def get_value_for_datastore(self, model_instance):
v = super(PickleProperty,
self).get_value_for_datastore(model_instance)
r = pickle.dumps(v, self._readable)
return self.data_type(r)
def make_value_from_datastore(self, value):
v = super(PickleProperty,
self).make_value_from_datastore(value)
r = pickle.loads(str(v))
return r
class XX(db.Model):
data = PickleProperty(default={})
xx = XX()
xx.data['a'] = 7
xx.put()
yy = XX.get(xx.key())
assert xx.data == yy.data
On Oct 15, 12:25 am, Josh Heitzman [EMAIL PROTECTED] wrote:
There are no indexes in index.yaml for these entity kinds and not very
many of the properties are being changed at one time (no idea if that
matters or not).
If updating the implicit indexes is the majority of the cost of doing
these updates, then I definitely agree that either--
1) an attribute for disabling the implicit indexing of properties
should be added, or
2) native serialization needs to be provided as part of the runtime so
we can quickly (de)serialize our data (from)into a blob.
On Oct 15, 12:04 am, djidjadji [EMAIL PROTECTED] wrote:
For this entity at least 44 (10+1+2+15+1+15) index updates have to be done
in 16 different index tables (10+1+2+1+1+1). Every attribute has its
implicit index
and you get an implicit index for the 'product' of the property lists.
Not to mention the index tables mentioned in the index.yaml file that
this entity uses.
It can grow big when you have the ListProperties used in the
index.yaml file, 15 extra updates
for every mention of the string list property.
I'm sure not every property of an entity is used in a query to retrieve
objects.
To reduce the number of index updates it could be useful to have a
non-index version of every property type. Just like we have for the
StringProperty. The TextProperty does not have an index to be updated.
A possible syntax to tell AppEngine NOT to create and update an index for
a
property would be to add an attribute to the Property constructor.
The default value of the attribute is True.
def MyModel(db.Model):
id = db.IntegerProperty(required=True)
num1 = db.IntegerProperty(need_index=False)
This would also help not to often hit the entity-index-update-limit
('exploding' index).
Are the index updates counted
[google-appengine] Re: Is there a way to get mcycles in code?
I'm not aware of anyway to get the mcycle in code either. As a result I've been breaking out little chunks of my processing and testing how long each on takes and I've found that updating an existing entity consumes no less then an average of 718 mcycles. And that's for an entity with an empty StringListProperty or an EmptyBlobProperty. Starting adding properties or items to a list and the mcycle consumption of an update starts jumping up. For example an entity with a single StringLIstProperty that has 40 elements each 14 characters long takes and average 2141 mcycles to update. See http://groups.google.com/group/google-appengine/browse_thread/thread/a29a4b1dd606f52e for details. On Oct 16, 10:07 am, Ethan Post [EMAIL PROTECTED] wrote: I have profiled the code. The problem with profiling is that the top CPU consuming calls appear to be part of the appengine framework, nothing I can do anything about. Also I don't understand how to correlate the information from profiling directly to mcycles. Just putting debug calls and looking at the timestamps tells me that most of time is taking place when I call put. I would like to record mcycles for each logging.debug call. Maybe I can add some calls using the profiler but I haven't figured out how to do that. On Thu, Oct 16, 2008 at 11:29 AM, Mahmoud [EMAIL PROTECTED] wrote: You might want to profile your code: http://code.google.com/appengine/kb/commontasks.html#profiling On Oct 16, 12:04 pm, Ethan Post [EMAIL PROTECTED] wrote: I would like to include the # of mcycles consumed in some of my logging. At the moment it a put to two entities has occasional timeouts and the number of mcycles appears to fluctuate for essentially the same put. I would like to add some logging to see if this is the case and if so what the variations are. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Is there a way to get mcycles in code?
Just opened http://code.google.com/p/googleappengine/issues/detail?id=789 - API to get current mcycle consumption during request processing. On Oct 16, 9:04 am, Ethan Post [EMAIL PROTECTED] wrote: I would like to include the # of mcycles consumed in some of my logging. At the moment it a put to two entities has occasional timeouts and the number of mcycles appears to fluctuate for essentially the same put. I would like to add some logging to see if this is the case and if so what the variations are. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Datastore storage recommendation...
See http://groups.google.com/group/google-appengine/browse_thread/thread/a29a4b1dd606f52e regarding the mcycle consumption impact of having lots of fields. On Oct 16, 9:11 pm, David koblas [EMAIL PROTECTED] wrote: I'm storing application preferences in the datastore, they come in a variety of types. As I see it I've got three choices for implementation: * Name / Blob -- general has a json encoded object of all of the sub-settings * Name / ID / Lots of Fields -- Where ID = general or user (etc.) and different fields in each entity * Name / ID / Key / Value -- Where you query out all of the user settings and re-construct an object from it. My gut feel is that #2 is preferable, since it avoids double encoding the data and doing a query out of a large number of records in the later case. Thanks, --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: 1000 mcycles to update a single entity
There are no indexes in index.yaml for these entity kinds and not very
many of the properties are being changed at one time (no idea if that
matters or not).
If updating the implicit indexes is the majority of the cost of doing
these updates, then I definitely agree that either--
1) an attribute for disabling the implicit indexing of properties
should be added, or
2) native serialization needs to be provided as part of the runtime so
we can quickly (de)serialize our data (from)into a blob.
On Oct 15, 12:04 am, djidjadji [EMAIL PROTECTED] wrote:
For this entity at least 44 (10+1+2+15+1+15) index updates have to be done
in 16 different index tables (10+1+2+1+1+1). Every attribute has its
implicit index
and you get an implicit index for the 'product' of the property lists.
Not to mention the index tables mentioned in the index.yaml file that
this entity uses.
It can grow big when you have the ListProperties used in the
index.yaml file, 15 extra updates
for every mention of the string list property.
I'm sure not every property of an entity is used in a query to retrieve
objects.
To reduce the number of index updates it could be useful to have a
non-index version of every property type. Just like we have for the
StringProperty. The TextProperty does not have an index to be updated.
A possible syntax to tell AppEngine NOT to create and update an index for a
property would be to add an attribute to the Property constructor.
The default value of the attribute is True.
def MyModel(db.Model):
id = db.IntegerProperty(required=True)
num1 = db.IntegerProperty(need_index=False)
This would also help not to often hit the entity-index-update-limit
('exploding' index).
Are the index updates counted in the mcycles used?
2008/10/15 Josh Heitzman [EMAIL PROTECTED]:
Regarding the first question, those mcycle numbers are from logs on
GAE, not from local profiling. But if you mean are lots of people
using, no. I was the only user with any data when I did the test.
Regarding the second question, the entities are not what I would
consider large. For example, one has 10 integer properties, 1 string
property, 2 datetime properties, one string list property (15 strings
with none more 30 characters long), and one int list property (only 1
value at the moment).
The entity group had 4 entities in it when I generated those numbers.
There is no contention involved, as the data is user specific and I
was the only user with data when I did the test.
On Oct 14, 8:31 pm, David Symonds [EMAIL PROTECTED] wrote:
On Wed, Oct 15, 2008 at 1:50 PM, Josh Heitzman [EMAIL PROTECTED] wrote:
Actually, I'm it take about 1500 mcycle to update one entity and then
an about an additional 1000 mcycle per additional entity (each a
different kind in this case) that is updated via the same db.put call.
Is this in production? What size is the entity? Is it in a large
entity group? How much contention do you think is involved?
Dave.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] 754 mcycles to update an entity (sole in group) with one integer property
As we didn't see anyone from the GAE team confirm or deny that the excessive mcycle consumption for doing an entity update was due to the number of properties on the entity (see this thread http://groups.google.com/group/google-appengine/browse_thread/thread/2901a4ddd1f671d9 ) I decided to do a little experiment. Follows are the average mcycle consumptions (on GAE not locally) to update 4 different types of entities. Each entity has no parent and or children (i.e. its the one and only entity in its group). There was no contention as these entities were created solely for this test. 754 - one IntegerProperty 864 - One each of IntegerProperty, DateTimeProperty, FloatProperty, and StringProperty 718 - one StringListProperty with no elements 2141 - one StringListProperty with 40 elements each with 14 characters. Has anyone else done a similar experiment? If so did you get similar results? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: 1000 mcycles to update a single entity
Opened http://code.google.com/p/googleappengine/issues/detail?id=786
regarding this issue.
On Oct 15, 12:25 am, Josh Heitzman [EMAIL PROTECTED] wrote:
There are no indexes in index.yaml for these entity kinds and not very
many of the properties are being changed at one time (no idea if that
matters or not).
If updating the implicit indexes is the majority of the cost of doing
these updates, then I definitely agree that either--
1) an attribute for disabling the implicit indexing of properties
should be added, or
2) native serialization needs to be provided as part of the runtime so
we can quickly (de)serialize our data (from)into a blob.
On Oct 15, 12:04 am, djidjadji [EMAIL PROTECTED] wrote:
For this entity at least 44 (10+1+2+15+1+15) index updates have to be done
in 16 different index tables (10+1+2+1+1+1). Every attribute has its
implicit index
and you get an implicit index for the 'product' of the property lists.
Not to mention the index tables mentioned in the index.yaml file that
this entity uses.
It can grow big when you have the ListProperties used in the
index.yaml file, 15 extra updates
for every mention of the string list property.
I'm sure not every property of an entity is used in a query to retrieve
objects.
To reduce the number of index updates it could be useful to have a
non-index version of every property type. Just like we have for the
StringProperty. The TextProperty does not have an index to be updated.
A possible syntax to tell AppEngine NOT to create and update an index for a
property would be to add an attribute to the Property constructor.
The default value of the attribute is True.
def MyModel(db.Model):
id = db.IntegerProperty(required=True)
num1 = db.IntegerProperty(need_index=False)
This would also help not to often hit the entity-index-update-limit
('exploding' index).
Are the index updates counted in the mcycles used?
2008/10/15 Josh Heitzman [EMAIL PROTECTED]:
Regarding the first question, those mcycle numbers are from logs on
GAE, not from local profiling. But if you mean are lots of people
using, no. I was the only user with any data when I did the test.
Regarding the second question, the entities are not what I would
consider large. For example, one has 10 integer properties, 1 string
property, 2 datetime properties, one string list property (15 strings
with none more 30 characters long), and one int list property (only 1
value at the moment).
The entity group had 4 entities in it when I generated those numbers.
There is no contention involved, as the data is user specific and I
was the only user with data when I did the test.
On Oct 14, 8:31 pm, David Symonds [EMAIL PROTECTED] wrote:
On Wed, Oct 15, 2008 at 1:50 PM, Josh Heitzman [EMAIL PROTECTED] wrote:
Actually, I'm it take about 1500 mcycle to update one entity and then
an about an additional 1000 mcycle per additional entity (each a
different kind in this case) that is updated via the same db.put call.
Is this in production? What size is the entity? Is it in a large
entity group? How much contention do you think is involved?
Dave.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: 754 mcycles to update an entity (sole in group) with one integer property
Oh, and: 756 - one StringListProperty with 1 element with 490 characters. So each of the three trials had just one property was under 800 mcycles regardless of the size of the property. On Oct 15, 9:17 pm, Josh Heitzman [EMAIL PROTECTED] wrote: As we didn't see anyone from the GAE team confirm or deny that the excessive mcycle consumption for doing an entity update was due to the number of properties on the entity (see this threadhttp://groups.google.com/group/google-appengine/browse_thread/thread/... ) I decided to do a little experiment. Follows are the average mcycle consumptions (on GAE not locally) to update 4 different types of entities. Each entity has no parent and or children (i.e. its the one and only entity in its group). There was no contention as these entities were created solely for this test. 754 - one IntegerProperty 864 - One each of IntegerProperty, DateTimeProperty, FloatProperty, and StringProperty 718 - one StringListProperty with no elements 2141 - one StringListProperty with 40 elements each with 14 characters. Has anyone else done a similar experiment? If so did you get similar results? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Unexpected timeouts
Timeouts can happen on even the simplest of datastore operations. As I understand it when a timeout does occur on a transaction (including a single put as there is a transaction there underneath) there is a lot of extra processing due to retry attempts. On Oct 13, 11:40 pm, Roy Leban [EMAIL PROTECTED] wrote: This might be bug 764 (http://code.google.com/p/googleappengine/issues/ detail?id=764) but I'm not sure. Sometimes, I get timeouts that don't make sense. For example, refreshing my home page (sometimes, when it happens to include a single database write). 0-13 05:58PM 18.196 / 500 4180ms 10883mcycles 5kb File myfile.py ... some_entity.put() File /base/python_lib/versions/1/google/appengine/ext/db/ __init__.py, line 618, in put return datastore.Put(self._entity) File /base/python_lib/versions/1/google/appengine/api/ datastore.py, line 162, in Put raise _ToDatastoreError(err) File /base/python_lib/versions/1/google/appengine/api/ datastore.py, line 1627, in _ToDatastoreError raise errors[err.application_error](err.error_detail) Timeout It's looking like the 10883 megacycles is 20-50x a typical load of my home page. Anybody have any ideas? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] 1000 mcycles to update a single entity
I've been digging into where may app is spending its mcycles. By using memcache I've gotten all of my requests that only read data down well under the 1000 mcycle overall average request processing limit; however, I'm seeing that it takes about 1000 mcycles to update a single entity (not indexed, not very large either). Is this in line with what other folks are seeing? --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: 1000 mcycles to update a single entity
Regarding the first question, those mcycle numbers are from logs on GAE, not from local profiling. But if you mean are lots of people using, no. I was the only user with any data when I did the test. Regarding the second question, the entities are not what I would consider large. For example, one has 10 integer properties, 1 string property, 2 datetime properties, one string list property (15 strings with none more 30 characters long), and one int list property (only 1 value at the moment). The entity group had 4 entities in it when I generated those numbers. There is no contention involved, as the data is user specific and I was the only user with data when I did the test. On Oct 14, 8:31 pm, David Symonds [EMAIL PROTECTED] wrote: On Wed, Oct 15, 2008 at 1:50 PM, Josh Heitzman [EMAIL PROTECTED] wrote: Actually, I'm it take about 1500 mcycle to update one entity and then an about an additional 1000 mcycle per additional entity (each a different kind in this case) that is updated via the same db.put call. Is this in production? What size is the entity? Is it in a large entity group? How much contention do you think is involved? Dave. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Does the datastore cache?
Just tried my app and I'm seeing changes show up immediately as expected. I was having some issue with page caching until I added: meta http-equiv=Cache-Control content=no-cache / to the head element of my pages. If you can reproduce the problem yourself, you can check your logs right after seeing the problem to see if the request actually got to the server or if a cached version was actually rendered. On Oct 12, 7:58 pm, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: I'm having the same problem. On Sep 19, 10:13 am, Andreas Krohn [EMAIL PROTECTED] wrote: Some clarifications from me as well... It doesn't matter how many times I force reload withoutcache, the new content doesnt show up (even if I can see it in the datastore). The delay is not just a few minutes (which I could live with) but several hours (and counting). Any ideas of how to get around this would be greatly appreciated! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: 3000 mcycles and This request used a amount of CPU and may soon exceed its quota
I've been wondering the same thing. When they start letting us pay for this, will our apps still have to stay under some arbitrary average number of CPU cycles? On Oct 13, 3:21 am, Matija [EMAIL PROTECTED] wrote: Hi, I have some init stage imports to app-engine application and combining with some ajax calls I was able to use maximum of 3000 mcycles per request. When there is request that consumes more the 1000 mcycles I have warning 'This request used a amount of CPU and may soon exceed its quota.' in my application log. Should I be worried because of that or not. Is this general warning for me to inform me that with this average CPU use i will use all my CPU quota eventually soon, or that app-engine quota permits this CPU usage only few times let's say per minute and that they will stop my application for several hours if I continue with this usage. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: how to create a plugable system and API without pkg_resources
It isn't impossible to use pkg_resources and setuptools with GAE. Pylons makes use of these and appengine-monkey ( http://code.google.com/p/appengine-monkey/ ) was created to get Pylons working on GAE. If you aren't using Pylons appengine-monkey still may prove useful to you. On Oct 11, 10:51 am, sevenseeker [EMAIL PROTECTED] wrote: I have become used to / spoiled with entry points from pkg_resources and setuptools. What is a similar system that avoids too much reinventing of the wheel? The goal is to facilitate a plugable system where directories or files (much like with eggs) can be dropped in the appropriate directory and extend the system through existing extension points as well as define new ones. I am most familiar with the Trac use of this and really like it. That said, I am very open to suggestions on how to use another system (hopefully not homegrown as I am sure many others do this). Thank you, Jason --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: datastore transaction question
In this scenario Update both the User and the Thing using a single db.put([thing, user]) the thing and the user won't be updated in the same transaction, so the put of the thing could succeed while the put of the user failed. On Oct 9, 9:29 pm, johnP [EMAIL PROTECTED] wrote: Why not simply have the reference in the Thing, pointing to the User? Then use the idiom: x = user.thing_set.fetch(1000) to get the User's Things? Update both the User and the Thing using a single db.put([thing, user]) johnP On Oct 9, 2:19 pm, Josh Heitzman [EMAIL PROTECTED] wrote: I ran into a similar problem with the strategy game I'm working on. The game world is divided up into areas and each area can have a more then one player's troops in it at a time. I've modeled these with three entities, player, area, and areaplayer. When a player has his troops do something in an area it can potentially affect the area state, the associated areaplayer states, and the associated player states. If all of those things were in the same entity group that would mean the entire game state would be in the same entity group resulting in massive contention. Instead each player and their home area is an entity group and each non-home area is an entity group. The areaplayer is in the entity group with the area, which is the same as the player for the case of the home area. So when a player orders troops to do something in their home area when no other player has troops there, its no problem as the player, the area, and the sole areaplayer are all in the same entity group so a GAE DS transaction can be used. However, orders given to troops away from home or at home other players have troops there have to make updates to multiple entity groups. I'm dealing with this by via a reliable multi-entity group update mechanism (no rollback, but will continue processing the update across requests and server process instances) that I hope can be enhance into true multi-entity group transactions if that can actually be done will still staying under quotas. I'm getting close to deploying my prototype with the reliable multi-entity group update mechanism, so I can find out how I'm doing on quotas as it is. Then I'll know if there is possibly create a cross-entity group transaction mechanism. I really think Google needs to provide cross-entity group transactions though, as they'll be able to do it in native code, and thus faster then we'll be able to do it in Python on at the app level. On Oct 9, 1:20 pm, David [EMAIL PROTECTED] wrote: thanks for the replies. as i mentioned in the question, the Thing will become a product and could be sold to another user at a later date. A list of keys is also what ive been using, as a referenece property in the Thing causes the blobs to pulled from the data store (they can be quite large, well less then a mb aye ;) ) when i want to list the users current things (by title). i guess another way of looking at it is really the user is creating a new product. and then is trying to buy it. if the product is no longer wanted by anyone it can be deleted. i was trying to work out a was to safly trasact on the money so the Thing never touches the datastore if there isnt enough cash. But this is a unuaul case, things will be sold between users more then created. any thoughts? its all a learning process for me. On Oct 9, 8:49 pm, Bill [EMAIL PROTECTED] wrote: Have you seen: Entity Groups, Ancestors and Paths ?http://code.google.com/appengine/docs/datastore/keysandentitygroups.html Good point. If a Thing will never be used for another User (e.g., it's not a product), you could make the Thing part of a User's entity group. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Why python?
Python has been around for over decade so I really wouldn't call it a new style of grammar. There are numerous programming languages out there that don't use c-like grammar, so Python is hardly unique in that respect. Python started life without objects and none of its procedural aspects were removed when objects were added. On Oct 9, 11:50 pm, Leo(雷傲) [EMAIL PROTECTED] wrote: I hate python's strange grammer! It's not neccesary to build a new style of grammer since the c-like grammer is so successful ! And I think the OOP is not so great. The procedure programming language is still powerful and will continue its power! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] get returning None (no exception raised) on server, but not locally
I just uploaded the prototype I've been working on for the last few weeks and I'm seeing different and unexpected behavior on the server. The following method is run at the beginning of a transaction to ensure that the transaction will operate on the most current state of the entities modified. All its doing is getting the key for an entity and then using the key to get the current sate of the entity. This has been working fine locally for weeks. def __FreshenEntities(self, names, toSetNone): for attrName in names: attr = getattr(self, attrName) if attr is None: raise AttributeError(HelperMethods.method_name(self) + attribute + attrName + not expected) key = attr.key() if key is None: raise Exception(FGBeingRootInterface.__FreshenEntities + attrName + key is none) entity = attr.get(key) if entity is None: entity = db.get(key) if entity is None: entity = db.get(str(key)) if entity is None: raise Exception(FGBeingRootInterface.__FreshenEntities + attrName + was none after being gotten from DS using key + str(key)) setattr(self, attrName, entity) if getattr(self, attrName) is None: raise Exception(FGBeingRootInterface.__FreshenEntities self. + attrName + was none after being set) toSetNone.remove(attrName) After seeing an exception thrown on the server due to an attribute with a None value I added the two extra attempts to get the entity in my efforts to determine what is going on, but all three return None without throwing an exception. The attributes were gotten the first time just before this method was called, so I don't understand how their keys can all of the sudden not be valid nor why get is not throwing some kind of an exception. Follows is the generated exception test for one occurrence and the key string looks legitimate: Exception: FGBeingRootInterface.__FreshenEntities _ranking was none after being gotten from DS using key ag53ZWJnYW1lc2J5am9zaHIjCxIMRkcwUmFua0dyb3VwGOoHDAsSCkZHMFJhbmtpbmcYAQw Anyone encountered anything similar? Any ideas on how to figure out what is failing? Thanks, Josh Heitzman --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: unique identifier for server process
The state of the process is not of interest, just which instance of many processes it is (i.e. there is no process specific state to store and query). I've got side entities all setup so that I can reliably make multi-entity group changes (i.e. processing will continue across requests, even from different clients). I'm currently just using a random number as a process instance identity, to determine if a change was left hanging by the same process so its write lock (custom not GAE) can be expired immediately, as the processes are not threaded, rather then waiting the full timeout period. Rather then a random number I could use an entity key path, but that's more time consuming to generate and then it needs to periodically have a datetime field updated so the staleness of it can be used determined if the entity can be deleted. I figure GAE already got some kind of process identifier around (likely an entity in the datastore) as part of the infrastructure, so its just a matter of providing an API to access it (I don't care if its hashed if there some concern about security, but there shouldn't be since one app is never supposed to be able to get any data outside its domain). On Oct 8, 9:27 pm, pr3d4t0r [EMAIL PROTECTED] wrote: On Oct 8, 6:03 pm, Josh Heitzman [EMAIL PROTECTED] wrote: As os.getpid is not supported by GAE, is there another API that provides a unique (preferably across all of GAE) identifier for the process? I'd like to use this to identify if an incomplete update sequence (spanning entity groups, so no it can't be done in a transaction) was left hanging by the current process or another, as if it was the current it can be immediately restarted, where as if it is another, it'll have to wait a time period before continuing the sequence. Because of the distributed nature of App Engine, about the only way you have of ensuring that something committed/finished to completion is to use transactions and entities. You could create an entity group that tracks your process(es) state and updates it as various stages are completed. Remember that there is no guarantee that your process will run on any given machine across subsequent calls. Cheers, pr3d4t0rhttp://www.istheserverup.comhttp://www.teslatestament.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: datastore transaction question
I ran into a similar problem with the strategy game I'm working on. The game world is divided up into areas and each area can have a more then one player's troops in it at a time. I've modeled these with three entities, player, area, and areaplayer. When a player has his troops do something in an area it can potentially affect the area state, the associated areaplayer states, and the associated player states. If all of those things were in the same entity group that would mean the entire game state would be in the same entity group resulting in massive contention. Instead each player and their home area is an entity group and each non-home area is an entity group. The areaplayer is in the entity group with the area, which is the same as the player for the case of the home area. So when a player orders troops to do something in their home area when no other player has troops there, its no problem as the player, the area, and the sole areaplayer are all in the same entity group so a GAE DS transaction can be used. However, orders given to troops away from home or at home other players have troops there have to make updates to multiple entity groups. I'm dealing with this by via a reliable multi-entity group update mechanism (no rollback, but will continue processing the update across requests and server process instances) that I hope can be enhance into true multi-entity group transactions if that can actually be done will still staying under quotas. I'm getting close to deploying my prototype with the reliable multi-entity group update mechanism, so I can find out how I'm doing on quotas as it is. Then I'll know if there is possibly create a cross-entity group transaction mechanism. I really think Google needs to provide cross-entity group transactions though, as they'll be able to do it in native code, and thus faster then we'll be able to do it in Python on at the app level. On Oct 9, 1:20 pm, David [EMAIL PROTECTED] wrote: thanks for the replies. as i mentioned in the question, the Thing will become a product and could be sold to another user at a later date. A list of keys is also what ive been using, as a referenece property in the Thing causes the blobs to pulled from the data store (they can be quite large, well less then a mb aye ;) ) when i want to list the users current things (by title). i guess another way of looking at it is really the user is creating a new product. and then is trying to buy it. if the product is no longer wanted by anyone it can be deleted. i was trying to work out a was to safly trasact on the money so the Thing never touches the datastore if there isnt enough cash. But this is a unuaul case, things will be sold between users more then created. any thoughts? its all a learning process for me. On Oct 9, 8:49 pm, Bill [EMAIL PROTECTED] wrote: Have you seen: Entity Groups, Ancestors and Paths ?http://code.google.com/appengine/docs/datastore/keysandentitygroups.html Good point. If a Thing will never be used for another User (e.g., it's not a product), you could make the Thing part of a User's entity group. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Definition of Google's Infrastructure
Regarding Should we design our system so that if the datacenters where an app is deployed are vaporized, then the app keeps serving? No, this is a much thornier issue. does this mean that an app's data is stored in one and only one data center with no off-site backups? I could understand that the off-site backup of an app isn't automatically started up when there is a burp in service from a datacenter, but if all of the data was lost from a datacenter would it just be gone? On Oct 7, 9:26 pm, Jon McAlister [EMAIL PROTECTED] wrote: Should we allow control over where the app is serving from? Sure, that sounds very reasonable, and is notably important for certain legal reasons amongst our European customers. Should we design our system so that if the datacenters where an app is deployed are vaporized, then the app keeps serving? No, this is a much thornier issue. Notably, I disagree with the claim that true inter-continental deployment of an app is a basic premise of modern cloud computing, mostly because this is really hard, and few systems actually get this right. Think about it from the view of a datastore write. When you write an entity, should that entity be immediately available on every continent? The reasonable answer is no, because if we guaranteed that, then the write latency would skyrocket. But if we don't guarantee that, what do we guarantee instead? If the app is presently serving from two continents, but we do not guarantee strong write behavior, how are conflicting writes then to be merged? If one datacenter disappears and then later comes back online, what happens to the writes that were halfway applied but not yet fully merged? Do we permit data to be dropped or do we try to reconcile this data, in spite of the fact that it may be hours or days stale? The answer to the above questions rely heavily on the specifics of the data and the behavior of the application, and most apps are happy to avoid this issue and are fine serving from one or a small number of locations. It's not a trivial thing to design one (or a handful of) generic APIs that support true inter-continental application presence, but this doesn't mean we'll give up trying to do so. We also welcome any technical suggestions you have. For instance, how would you presently solve this issue outside of Google App Engine? On Oct 7, 1:39 pm, Andrew Badera [EMAIL PROTECTED] wrote: That's the point of the cloud -- if you're going to make your resources external, remote, you need to provide a means for assuring uptime. For some people, different geophysical locations are required for their service. Obviously GAE beta shouldn't see a true NEED for this while still in beta, but like SSL and everything else GAE lacks, there IS a need, it IS a basic premise of modern cloud computing. On Tue, Oct 7, 2008 at 4:35 PM, Sal [EMAIL PROTECTED] wrote: So you want to be assured that if all the Google data centers in the U.S. (over 12) go down (I wonder the probability of this), your GAE application will still be up? On Oct 7, 11:35 am, Andrew Badera [EMAIL PROTECTED] wrote: Ahh ... availability and assurance? That's half the point of the cloud. On Tue, Oct 7, 2008 at 2:34 PM, Sal [EMAIL PROTECTED] wrote: Honestly, why would anyone need to deploy their GAE applications to international data centers? On Oct 7, 10:48 am, dleifker [EMAIL PROTECTED] wrote: What exactly does the term Google's Infrastructure imply? Once deployed does an application get deployed to regional (ie international) data centers? If not, from what general geographical area are the applications being served from? (US only?) And are there plans to allow an application to be deployed to international locations? -- Thanks- - Andy Badera - [EMAIL PROTECTED] - (518) 641-1420 -http://higherefficiency.net -http://changeroundup.com/ -http://flipbitsnotburgers.blogspot.com/ -http://andrew.badera.us/ - Google me:http://www.google.com/search?q=andrew+badera --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Why Google App Engine is broken and what Google must do to fix it.
Assuming you can actually work around all of the limitation simultaneously, considering the more things you work around the close you come to the CPU time quota. It also is not completely accurate to state that the limitations are there to make apps safe and scalable. For example transactions being limited to a single entity group makes it very complicated (i.e. not safe) to write code that needs to reliably update entities from different groups (it isn't always possible to structure entity groups such that everything that is needs to be updated for a request can be all be in one entity group). Also given the roundness of the quotas, I find it very unlikely that the quota numbers were choosen based on an in depth analysis or a broad sampling of data, rather the being choosen fairly arbitrarily (possibly based to some extent on what works for google's own apps). On Oct 7, 4:16 pm, Greg [EMAIL PROTECTED] wrote: Davew said it way back at the top - appengine's killer feature is scalability. That is what sets it apart from the other cloud systems out there, and it is also the root cause of most complaints (except the quotas, which will disappear when you get to pay for the service). For the application I'm working on, I'm happy to trade off lack of a relational database for the future gain of scalability. My guess is that most of you haven't had the nightmare of an application that suddenly became popular, and you had to become an expert at database replication, load balancing and multi-system maintenance overnight. It's a very stressful situation. So my advice is that if you don't need scalability, get a normal hosting account or EC3. Then you can have PHP, Ruby, MySQL, cron jobs, anything you want - problem solved. Oh, yes you are going to have to shell out a few buck a month. But if you do need scalability, then appengine is a godsend. The limitations are there to make it safe and scalable, not because Google wants to annoy you. You spend a little more time now working around the limitations, and save endless time later managing systems and capacity. And lastly, I believe that many of the complaints come from people just wanting a free hosting service, and not finding what they are used to. It would be a crying shame if Google listened to these people and turned appengine into a vanilla PHP/MySQL hosting service. Appengine is so much more... On Oct 7, 3:54 pm, Ross Ridge [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] wrote: One thing you have to remember it is not what Guido or the engineers want. If Google App Engine is to succeed it is what the customers want. If it is designed as you have stated it will never recoup what Google has spent so far let alone down the road. Google App Engine has so many many limitations. Regardless if the limitations are by design or not it is virtually unusable by 99% of all developers. Can Google make a business off the remaining 1%? The question of whether Google can turn Google App Engine into a profitable business doesn't depend on what percentage of developers find it useful, but whether Google exploit a competive advantage. Google could've started up a tradtional web hosting service using popular SQL databases and other techonologies and created something that would have had a much broader appeal. Any one could. That's the problem. Google might be able to grab market share, but without anything to distiguish themselves from their competitors, a best they only get a marginal return on their investment. We can only speculate on what Google business plan for GAE is, but it seems pretty obvious to me that leveraging Google's own internal technologies is at the heart of it. A number of limitations and problems with GAE stem from technologies like Big Table, Google Frontend and Google Apps. Another part of their plan appears to be keeping support costs low, so you're not given much rope to hang yourself (or others). If, in the long term, Google can't make a business following this plan, if it doesn't give them enough a competive advanage, then there's probably no way they can make the kind profits from a hosting service that Google's investors expect. (While it's not terribly relevent to this discussion, I suspect Google has some other goals for GAE that don't deal directly with its viability as a business. One is to educate programmers in the Google way of doing things. I'm sure Google has been fustrated with tons of amazing job applicants with advanced degrees, 10+ years of WWW experience, and the inability work with anything but PHP and SQL. Another is that they want to make even easier for people to create WWW sites, the sort of small little sites that through AdWords/AdSense, Google has made billions.) Ultimately, what matters is what you want and what Google is willing give you. It doesn't matter what 99% developers want. The are number of
[google-appengine] Re: Why Google App Engine is broken and what Google must do to fix it.
Not all applications' data can be modeled such that that each piece of data belongs to one and only one user. On Oct 7, 10:24 pm, Ian Lewis [EMAIL PROTECTED] wrote: The documentation says that an entity group could be as large as a single user's data. It seems to me that to avoid the problem you are describing that it would be a good idea to do so. I have a feeling that the number of cases where you would need to update more than one user's data in a single transaction would be limited. 2008/10/8 Josh Heitzman [EMAIL PROTECTED] Assuming you can actually work around all of the limitation simultaneously, considering the more things you work around the close you come to the CPU time quota. It also is not completely accurate to state that the limitations are there to make apps safe and scalable. For example transactions being limited to a single entity group makes it very complicated (i.e. not safe) to write code that needs to reliably update entities from different groups (it isn't always possible to structure entity groups such that everything that is needs to be updated for a request can be all be in one entity group). Also given the roundness of the quotas, I find it very unlikely that the quota numbers were choosen based on an in depth analysis or a broad sampling of data, rather the being choosen fairly arbitrarily (possibly based to some extent on what works for google's own apps). On Oct 7, 4:16 pm, Greg [EMAIL PROTECTED] wrote: Davew said it way back at the top - appengine's killer feature is scalability. That is what sets it apart from the other cloud systems out there, and it is also the root cause of most complaints (except the quotas, which will disappear when you get to pay for the service). For the application I'm working on, I'm happy to trade off lack of a relational database for the future gain of scalability. My guess is that most of you haven't had the nightmare of an application that suddenly became popular, and you had to become an expert at database replication, load balancing and multi-system maintenance overnight. It's a very stressful situation. So my advice is that if you don't need scalability, get a normal hosting account or EC3. Then you can have PHP, Ruby, MySQL, cron jobs, anything you want - problem solved. Oh, yes you are going to have to shell out a few buck a month. But if you do need scalability, then appengine is a godsend. The limitations are there to make it safe and scalable, not because Google wants to annoy you. You spend a little more time now working around the limitations, and save endless time later managing systems and capacity. And lastly, I believe that many of the complaints come from people just wanting a free hosting service, and not finding what they are used to. It would be a crying shame if Google listened to these people and turned appengine into a vanilla PHP/MySQL hosting service. Appengine is so much more... On Oct 7, 3:54 pm, Ross Ridge [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] wrote: One thing you have to remember it is not what Guido or the engineers want. If Google App Engine is to succeed it is what the customers want. If it is designed as you have stated it will never recoup what Google has spent so far let alone down the road. Google App Engine has so many many limitations. Regardless if the limitations are by design or not it is virtually unusable by 99% of all developers. Can Google make a business off the remaining 1%? The question of whether Google can turn Google App Engine into a profitable business doesn't depend on what percentage of developers find it useful, but whether Google exploit a competive advantage. Google could've started up a tradtional web hosting service using popular SQL databases and other techonologies and created something that would have had a much broader appeal. Any one could. That's the problem. Google might be able to grab market share, but without anything to distiguish themselves from their competitors, a best they only get a marginal return on their investment. We can only speculate on what Google business plan for GAE is, but it seems pretty obvious to me that leveraging Google's own internal technologies is at the heart of it. A number of limitations and problems with GAE stem from technologies like Big Table, Google Frontend and Google Apps. Another part of their plan appears to be keeping support costs low, so you're not given much rope to hang yourself (or others). If, in the long term, Google can't make a business following this plan, if it doesn't give them enough a competive advanage, then there's probably no way they can make the kind profits from a hosting service that Google's investors expect. (While it's not terribly relevent
[google-appengine] Re: No module named os
Can't speak for Mac or this specific SDK, but on Windows when I
upgraded to an earlier SDK release, I couldn't get my app working
again on the new release until I re-imaged my hard disk to GAE SDK
free image. This was after having uninstalled the older SDK before
installing the new one.
The problem I encountered was also module loading related.
On Oct 6, 9:51 am, Marzia Niccolai [EMAIL PROTECTED] wrote:
Hi Ilia,
What python installation have you set in your preferences for the
MacLauncher? Are you using the standard Mac python? Is this a fresh install
of the launcher or did you just download the SDK?
Sorry for the lame advice but have you tried uninstalling and
re-installing? On 10.5.3 I'm able to call 'import os' with no errors (using
/usr/local/bin/python), so consider me officially flummoxed this morning.
-Marzia
On Sat, Oct 4, 2008 at 9:52 AM, Ilia Lobsanov [EMAIL PROTECTED] wrote:
oh and this is on mac os x 10.5.5
On Oct 4, 12:43 pm, Ilia Lobsanov [EMAIL PROTECTED] wrote:
As of the 1.1.5 upgrade, I am getting this error when I try to access
my app using the dev server:
type 'exceptions.ImportError': No module named os
args = ('No module named os',)
message = 'No module named os'
Any ideas?
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Please include CPU quotas better db support in your simulation
What would be really nice is if there was an API that could be called to determine how close you were to the CPU quota so you could stop processing, persist the current state, and return a Server busy, please try again. type of page to the user that has a link to a URL that would pickup were the last request had to stop. Josh On Oct 6, 10:10 am, Amir Michail [EMAIL PROTECTED] wrote: Hi, Running into hours of downtime because of several requests taking too much CPU makes development almost impossible. If you really want to keep this penalty in place for high CPU usage, then please at least simulate it so people can do development locally. Moreover, please provide an efficient local database -- one whose performance would be similar to the deployed app with one user at a time. Amir --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Why Google App Engine is broken and what Google must do to fix it.
From near the bottom of http://codecrafter.wordpress.com/2008/09/16/light-web-strategy-game-data-model-smacks-into-gae-limitations/ : After some thought I decided my requirements for an enhanced GAE data model layer were: 1. Transactions spanning entity groups 2. [redacted - more of a nice to have] 3. Schema version management And from earlier tonight http://codecrafter.wordpress.com/2008/10/03/google-app-engine-scalability-that-doesnt-just-work/ * [redacted - not really broken if other issues are fixed] * [redacted - not really broken if other issues are fixed] * [redacted - not really broken if other issues are fixed] * GAE can’t not tell you how many total entities there are of a specific type and it can’t count more then 1000 entities. * GAE limits the entities that can be processed in a single transaction to those in the same entity group and only one request processing instance can write to an entire entity group at a time. * [redacted - not really broken if other issues are fixed] * [redacted - not really broken if other issues are fixed] * GAE’s version of Python does not include marshall or cpickle, only the slow pickle Of these issues, the two I'm finding most annoying is that transactions can't span entity groups (combined with needing to keep entity groups small to avoid put contention so the app will actually scale) and the lack of cpickle (or something more peformant then pickle) so I could efficiently serialize collections of small data structures associated with another entity for which I have need to query. Josh Heitzman On Oct 3, 11:52 am, Aral Balkan [EMAIL PROTECTED] wrote: I just wrote up a blog post summarizing the biggest issues I have with App Engine:http://aralbalkan.com/1504 If you are developing real-world/commercial apps with App Engine, please add your thoughts to the discussion. Thanks, Aral --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Why Google App Engine is broken and what Google must do to fix it.
If the entity-groups have a (single) home and there is not any global coordination for transactions that means that there is a single machine in charge of that entity group. Allowing cross entity group transactions does not require that a single machine be in charge of all entities, it just means that the request processing process will need to coordinate with the machines in charge of each entity or that the machines in charge of each entity need would need to coordinate with each other as peers. That said I'm doubtful that entity groups have a single home as they would need to be mirrored around to actually scale, so it seems likely to me there is already coordination between the nodes mirroring the same entity group. On Oct 4, 2:53 am, David Symonds [EMAIL PROTECTED] wrote: On Sat, Oct 4, 2008 at 6:00 PM, Josh Heitzman [EMAIL PROTECTED] wrote: As far as transactions go, the global coordination is already there at the entity group level (if it wasn't, then there wouldn't be any transactions at all). Sure it will be more costly at the entity level then at the group level, but it will still be less costly then layering the equivalent on top of the data store API in the python application code. Allowing transactions spanning entity groups, doesn't mean that entity groups and non-spanning transactions need be eliminated. Leave it the application developer to choose the appropriate transaction type as needed. You're right, application-wide transactions don't have to subsume entity-group transactions; you could, after all, put every entity in your application in a single entity group. They are, however, still inherently unscalable. Entity groups mean that global coordination is *not* required, only that there is a home for each entity group. As long as the entity groups are small, this responsibility can be split across machines quite easily. Dave. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Why Google App Engine is broken and what Google must do to fix it.
This is a good point. This isn't something I care about for the web app I writing at the moment, but I've already there are other apps I've though about doing that I definitely would not want be willing upload non-obfuscated source code for, and I was figuring I'd just have to host those elsewhere. On Oct 4, 2:48 am, Davide Rognoni [EMAIL PROTECTED] wrote: The business is the main issues. Search the word obfuscatehttp://evolvingtrends.wordpress.com/2008/07/23/google-app-engine-thre... On Oct 3, 8:52 pm, Aral Balkan [EMAIL PROTECTED] wrote: I just wrote up a blog post summarizing the biggest issues I have with App Engine:http://aralbalkan.com/1504 If you are developing real-world/commercial apps with App Engine, please add your thoughts to the discussion. Thanks, Aral --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Lifestreaming app on GAE?
You would need to do the data mining and machine learning as either a side task processed a little bit at a time with each page request or you would need a URL whose only purpose in being requested is to allow your algorithm to process to for a bit and then you would need to have a client that just keeps hitting that URL to keep your algorithm processing. You are also strictly limited to Python code removing which remove any possibility of doing low level optimization for CPU intensive routines. On Oct 4, 11:46 am, Carlos Hellín [EMAIL PROTECTED] wrote: I'm building a lifestreaming app that will involve pulling down lots of feeds for lots of users, and performing data-mining, and machine learning algorithms on the results. It's a good idea to use GAE or has many limitations? for example: lack of cron jobs is a nuisance or for CPU intensive apps I don't know if it's good. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Entity Group and Transactions
Your understanding is correct based on my experience. As far as determining entity group I think you'll just have to examine the ancestry of each entity in your transaction to see if they actually have a common instance. On Sep 25, 11:33 pm, Jamesz [EMAIL PROTECTED] wrote: Hi everyone, I just want to check my general understanding of app engine datastore concepts with you guys. After reading documentations on entity group and transactions, my understanding is that entities of different kind can still be in the same group provided that they have the same ancestors(parents or parents of parents). This means I can perform transactions on entities of different kinds. For clarity, this is my understanding of app engine concepts that roughly translates to its RDBMS equivalent 1. Kind = Table (this is roughly the case, though entities of the same kind doesn't have to have the same fields) 2. Property = Field 3. Entity = A record 4. Parent and Ancestors = no equivalent but is used to group entities into a entity group which will be stored in the same nodes which enables transactional executions. But when i applied this in practice, I get an error with Cannot operate on different entity groups in a transaction. So I wonder if I've misunderstood something, Is there a way to check what entity group an entity actually belongs to? Thanks James --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] SDK uninstaller / upgrade process bug
I don't know what exactly the problem is but I upgraded from the 1.1.1 SDK to 1.1.3 SDK today. I uninstalled 1.1.1 first before installed 1.1.3, but when I tried starting my app after installing 1.1.3 SDK it failed extremely early on in the Pylons loading code. I put up a couple of messages on the Pylons group, but no one else encountered the issue nor did anyone know what would be causing the behavior I was seeing with regard to site.py failing to load. After spending the better part of the day debugging and trying different things (including uninstalling 1.1.3 and reinstalling 1.1.1 and then reversing that again), I finally threw up my hands and restored my hard drive to an earlier image and reinstalled 1.1.1 and my app worked fine. I then uninstalled 1.1.1 then deleted what the uninstaller didn't remove and installed 1.1.3 and my app again worked fine. While searching through this group for 'load_module' I ran across a message that also indicated someone ran into a module loading related problem when moving from 1.1.1 to 1.1.2, so the issue doesn't seem to have occured just on my machine. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
