On Mar 22, 3:48 pm, "Nick Johnson (Google)" <nick.john...@google.com> wrote > On Mon, Mar 22, 2010 at 8:45 PM, homunq <jameson.qu...@gmail.com> wrote: > > OK, after hashing it out on IRC, I see that I have to erase my data > > and start again. > > Why is that? Wouldn't updating the data be a better option?
Because everything about it is wrong for saving space - the key names, the field names, the indexes, and even in one case the fact of breaking a string out into a list. (something I did for better searching in several cases, one of which is not worth it now I realize that 10X is easy to hit.) And because the data import runs smoothly, and I have code for that already. .... Watching my deletion process start to get trapped in molasses, as Eli Jones mentions above, I have to ask two things again: 1. Is there ANY ANY way to delete all indexes on a given property name? Without worrying about keeping indexes in order when I'm just paring them down to 0, I'd just be running through key names and deleting them. It seems that would be much faster. (If it's any help, I strongly suspect that most of my key names are globally unique across all of Google). 2. What is the reason for the slowdown? If I understand his suggestion to delete every 10th record, Eli Jones seems to suspect that it's because there's some kind of resource conflict on specific sections of storage, thus the solution is to attempt to spread your load across machines. I don't see why that would cause a gradual slowdown. My best theory is that write-then-delete leaves the index somehow a little messier (for instance, maybe the index doesn't fully recover the unused space because it expects you to fill it again) and that when you do it on a massive scale you get massively messy and slow indexes. Thus, again, I suspect this question reduces to question 1, although I guess that if my theory is right a compress/garbage-collect/degunking call for the indexes would be (for me) second best after a way to nuke them. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
