If this is some fixed data that needs to be searched almost every request, how about breaking it into a lot of chunks (e.g., if it's words, you could have 26 dicts, one for each starting letter) and hard- coding it as a python variable declarations in static files -- one dict per file. That would help you around the 1MB limits on variable and file size, right?
When you read in each file as a module, the variable declarations would be automatically cached between requests, which would really decrease the load. On Dec 7, 10:10 am, adrian <[EMAIL PROTECTED]> wrote: > My app needs some fixed data that I need to search during almost every > request to my app. It's over 1 MB as a dict in a python file so I > can't do that. > > So I've been reading a binary picked file but this is taking almost 2 > CPU seconds and 900k function calls according to profile. I get a > CPU usage warning on every request. > > I tried replacing pickle with marshal and it reduced the function > calls to 15k for the same data, and was much faster on the dev > server. So I uploaded it to the production server, but it dies > saying that marshal has no load method. It appears that marshal.load > () is not supported on AppEngine, but if so, why is it available on > the dev server? > > I realize that marshal may change with python version updates, so this > is not an optimal solution, so I'm open for other solutions to try. > > Thanks --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
