Totally agree what Sudhir is suggesting. I think there are at least 2 things the index building can be improved
1. currently the index building is still a little bit buggy, sometimes I (and I believe some others) would have indexes being stuck in "building" state, when I believe there were errors there but for some unknown reasons they can't be transited to "Error" state. And we, as programmer, can do nothing to prevent or fix this. What we can do is to send an email to google and ask them help changing the indexes to "Error" state so that we can vacuum it. 2. putting index building tasks into the global queue is totally understandable, but could we have a little smart feature that if the system knows that the indexes to be built are on 'empty' model (or the required CPU is very minimal if there are only a few entries), this could be executed immediately without putting on the queue for waiting. (or there can be multiple queues for light-weight or heavy-duty tasks) - eric 2010/1/3 Sudhir <sudhi...@gmail.com>: > Absolutely... which is the problem. Indexes on *empty* datastores took > 7 hours last night. I understand fully that 7 hours might be expected > for millions of objects and gigabytes of data, but not for a 76kb > datastore with three objects, 80% of which is the datastore statistics > and metadata. My app is still in development and there's nothing in > there expect "test1", "test2" and "test3". > > I've seen on the docs that index building might take 12h to 24h just > waiting on the queue, irrespective of how much data actually needs to > be indexed. While I appreciate the fact that the GAE team is being > very open and honest about this, this will be a deal breaker for lots > of people. > > I love the GAE idea, and I'm basing all my hopes of writing the next > killer app and making a billion dollars on this platform, but stuff > like this is a big turn off for anybody, I think. It completely takes > the 'instant' and 'quick' out of the process, and I think it needs to > be fixed before it gives GAE a bad rep. > > Sudhir > > On Jan 3, 4:53 am, Barry Hunter <barrybhun...@googlemail.com> wrote: >> A index built against a empty datastore (or at least a Model without >> any Entities) should always be available immediately. Or at least that >> is my experience. This makes sense because such an index doesnt >> actully need to go though the 'bulk build' pipeline (which can be a >> cause of delay if the pineline is busy - the index builder is a >> background process) >> >> of course adding Entities to this empty index will happen in realtime, >> so just add the data after deploying the index. >> >> 2010/1/2 Sudhir <sudhi...@gmail.com>: >> >> >> >> > Does anyone know what efforts are being taken to speed up index >> > building and what their status is? >> >> > I'm finding it very difficult to be a champion for GAE in my company - >> > one of the (implicitly) advertised features of GAE is the whole click >> > to deploy and start running feature. Things don't bode very well if >> > I'm doing a demo for the clients / bosses and we all need sit and wait >> > 12 to 24 hours for GAE to index two strings that say "Hello World" and >> > "test". >> >> > Its especially frustrating when I need to deploy and test during >> > development (stuff like custom search doesn't work on localhost). >> >> > I've been hitting refresh the last six hours. >> >> > The production issues pilot is a great step in the right direction, >> > and I filed an issue and all, but is the GAE team actually thinking of >> > responding to 100,000 (and counting) developers? >> >> > Just wondering, how on earth do you guys do demos at I/O? I don't >> > think any of the presenters hit deploy and said "okay, let's all come >> > back tomorrow, or maybe the day after, and we'll see if this >> > works" :-/ >> >> > Sudhir >> >> > -- >> >> > You received this message because you are subscribed to the Google Groups >> > "Google App Engine" group. >> > To post to this group, send email to google-appeng...@googlegroups.com. >> > To unsubscribe from this group, send email to >> > google-appengine+unsubscr...@googlegroups.com. >> > For more options, visit this group >> > athttp://groups.google.com/group/google-appengine?hl=en. > > -- > > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appeng...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
