cc to lucy-dev... On Tue, Jan 12, 2010 at 07:50:46PM -0600, Peter Karman wrote: > one-to-one field names is cool as long as FullText is sortable.
OK, glad that'll work. > Of course, my next question is predictable: when will 0.30 be fully cooked? The goal has been to get the KS dev branch file format and API stable before displacing the stable branch. However, there are still difficult file format problems to solve, particularly with regards to term dictionaries and posting lists -- such as support for multi-stream posting formats and indexing of non-text field types. I haven't really been working on those problems too much lately. After reaching our goals for index opening speed and integration of memory mapped sort caches last year, I could have gone back to that -- but instead, I've gone to work on Lucy. Lucy isn't that far off at this point: N months. There are some people who are using stable branch KS and who would be disrupted if we simply clobber the stable branch by releasing the dev branch on top of it, e.g. the MojoMojo folks (<http://mojomojo.org/features#Searching>). I'm reluctant to do that, since we haven't reached our goals for file format and API stability. Yeah, they were warned by the "alpha" label, but KS has also been promising a level of stability which we have yet to deliver. A one-time painful switch might have been OK, but forcing them back into an ongoing dev cycle isn't. To avoid disrupting such users, we could take one of two paths: * Fork the current stable release under "KinoSearch0" and expect existing users to switch. * Move the dev branch (svn trunk) under "KinoSearch2" and release it as an alpha. (I lean towards this option because it sets a precedent for how I think we'll need to handle versioning in Lucy.) If we'd managed to launch Lucy by now, this question would be academic, because Lucy would have become the successor to the KS dev branch. And I've kind of been working on Lucy with that in mind. Lucy remains my main goal. From a marketing perspective, I'm not sure that it's ideal to launch "KinoSearch2" as an alpha, then deprecate it in favor of Lucy a few months later. And once Lucy is launched in earnest and people outside our small circle start contributing, KS will have to be deprecated because licensing issues will eventually prevent us from backporting some important chunk of Lucy code to KS. So that's why I've been kind of keeping my head down and working feverishly on Lucy. I figured we'd get Lucy out as an alpha, grow its user base by releasing Ruby and Python bindings, then harness the excitement from that to work on the difficult problems that have held back KS. Designing a pluggable indexing framework is hard; it's almost impossible without a large user base, since only a small subset of users will be in a situation where they can test drive the pluggability features and help us refine the API. > And, how can I help? You and Nate have been very helpful with regards to code and API review. If we go down the current path towards Lucy, I'd ask you to continue exploring new areas and providing feedback about how it went. If we decide to make a formal CPAN release of dev branch somehow, there will be some mechanical work to do. If you wanted to do that, you could -- but I'm under the impression that you don't have that much time (compared with the 60 or so hours I've been putting in each week) and I don't want to squander a limited resource. If I could go back in time, I would have released the KS 0.20 branch under the namespace "KinoSearch2" and the 0.30 branch under "KinoSearch3". Maybe that points the way forward. Whatever we do, though, I'm determined not to let progress towards Lucy flag again. Marvin Humphrey
