I've been tinkering with the indexing system and found a few bugs, and added some better functionality plus a new hook for custom systems. Basically, my site was getting so large it was not handling the indexing very well and performance was taking a hit whenever content got written to a page (any page). I also figured out why some pages were not getting indexed consistently.
The system I'm going to try is a custom autoIndex function which takes the list of modified pages (generated automatically by BoltWire) and then filter through the list and record any pages I want to index to special site.myindex pages. The rest get dropped. Then another function (cron based) checks all the site.myindex pages periodically and creates a custom index page for each one. So members all get indexed in a member index. My blog posts in a blog index. And forum posts in a forum index. This should put everything in the background without any performance loss, and allow for much zippier searches of specific indexes. I'll lose site wide searching--but I'm not sure I need that for my site... Oh, and I also set up a data miner to allow me to reindex all the existing sections of my site gradually over the next few days according to the new scheme... I'm wondering if anyone would be interested in this kind of plugin, or if you have other ideas/feedback on how you have dealt with indexing and performance? Is there any reasonable way to rapidly search an index of several megabytes in size in real time? For a really large site, would a database index be the only really viable course? Or should we just leave it to google to index such a site? Curious what your thoughts are... I hoped to release the new version today, but may wait till tomorrow as I want to test things some more before I release it... Cheers, Dan -- You received this message because you are subscribed to the Google Groups "BoltWire" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/boltwire?hl=en.
