Dear Wiki user, You have subscribed to a wiki page or wiki category on "Lucy Wiki" for change notification.
The "LucyIncubatorProposal" page has been changed by MarvinHumphrey. The comment on this change is: Flesh out "An Excessive Fascination...". http://wiki.apache.org/lucy/LucyIncubatorProposal?action=diff&rev1=22&rev2=23 -------------------------------------------------- ## page was renamed from GraduationPlan For reference, see the original template at [http://incubator.apache.org/guides/proposal.html] + == Preface == - Lucy is a sub-project which is being spun off from the Lucene TLP but is not yet ready for graduation. We propose to address certain needs of the project by assimilating the KinoSearch code base, and enter the Incubator on a top-level-project track. == Abstract == @@ -17, +17 @@ In the last year, the work has come to fruition: major technical milestones have been achieved and Lucy's underpinnings have been completed. Additionally, other developers from the KinoSearch community have taken an interest in Lucy and have begun to ramp up their contributions. The next steps for Lucy were articulated by the Lucene PMC in a recent review: make releases, acquire users, grow community. - To implement the Lucene PMC's recommendations and get to a release as quickly as possible, the Lucy community proposes to assimilate the KinoSearch code base, which has been retrofitted to use Lucy's core. Lucy still lacks a number of important indexing and search classes; we wish to flesh these out via IP clearance work rather than software development. + To implement the Lucene PMC's recommendations and get to a release as quickly as possible, the Lucy community proposes to assimilate the KinoSearch code base, which has been retrofitted to use Lucy's core. Lucy still lacks a number of important indexing and search classes; we wish to flesh these out via IP clearance work rather than software development. Since the Lucene PMC will not be responsible for Lucy much longer, it is more appropriate for the software grant to take place within the context of the Incubator than the Lucene TLP. As none of our current members have Apache PMC experience, we also seek to take advantage of the Incubator environment to prepare ourselves for responsible self-governance. @@ -30, +30 @@ * Expose a public C API. == Current Status == - === Meritocracy === Our initial committer list includes two individuals (Peter Karman and Nathan Kurz) who started off as KinoSearch users, demonstrated merit through constructive forum participation, adept negotiation, consensus building, and submission of high-quality contributions, and were invited to become committers. Peter now rolls most releases. @@ -42, +41 @@ Lucy's chief challenge is growing its community, which it hopes to achieve through efforts in two areas: reaching a 1.0 release, and actively reaching out to its target audience, users and developers in the dynamic language communities who want a fast, scalable full-text search solution in their native language. === Core Developers === - * Marvin Humphrey is the project founder of KinoSearch, and co-founded the existing Lucy subproject. He is presently employed by Eventful, Inc. * Peter Karman has contributed to several open source projects since 2001, including being a committer at http://swish-e.org/ (a search engine), http://code.google.com/p/rose/ (an ORM) and http://catalyst.perl.org/ (web framework). He is employed by American Public Media. * Nathan Kurz has participated in numerous open source projects and has been a KinoSearch committer since 2007. He is currently Chief Flavor Engineer of Scream Sorbet, and writes software in his copious free time. @@ -55, +53 @@ A third priority of ours is to be bound by existing Apache institutions, for the protection of all our stakeholders. == Known Risks == - === Orphaned products === All initial committers have been associated with the project for several years across multiple jobs. However, at this time, the project would probably not survive the departure of Marvin Humphrey, so there is a risk of being orphaned. Marvin has no plans to leave, but we have been actively working to disperse his knowledge of the code base and administrative responsibilities in order to make him dispensable. Having staggered badly after Dave Balmain's departure, we are keenly aware of this vulnerability and highly motivated to eliminate it. === Inexperience with Open Source === - The initial committers have all have significant experience with open source development, and include one present Apache committer. We recognize that we lack PMC experience and seek to address that deficiency by going through the Incubator. In retrospect, Marvin wishes that Lucy had gone through the Incubator during its first inception. === Homogenous Developers === @@ -74, +70 @@ Still, Lucy is a loose port and its core differs in fundamental ways from that of Lucene. The biggest difference is that for Lucy, "the OS is our JVM": Lucene Searcher objects build up optimized data structures at search-time in process RAM, while Lucy writes its data structures to disk at index-time and reads them via memory-mapped IO at search-time. This affords Lucy several advantages, such as fast process launch, low process RAM requirements, and OO design flexibility because Lucy's "cheap Searcher" objects are lightweight, thin wrappers around the system IO cache. - === A Excessive Fascination with the Apache Brand === + === An Excessive Fascination with the Apache Brand === - Lucy's past sins: + Our desire to maintain Lucy's affiliation with Apache has much less to do with the brand and much more to do with our conviction that developing the project The Apache Way under Apache institutions is in Lucy's best interests. However, we have to acknowledge that during its time as a Lucene subproject, Lucy has not always fulfilled certain key requirements for an Apache project. In particular, it has failed to "release early, release often", and it has made only limited progress in expanding its community. + We attribute some of our difficulties to the what may have been excess ambition in the original Lucy plan, given the scope of the project and the size of the initial committer list: - * Failed to release early and often. - * Failed to build community. - Proposal is intended to address those deficiencies. + . [http://www.apache.org/foundation/how-it-works.html#incubator] + + . The basic requirements for incubation are: + * a working codebase -- over the years and after several failures, the foundation came to understand that without an initial working codebase, it is generally hard to bootstrap a community. + + By rebooting the project with a working codebase, we expect to avoid the trap that ensnared Lucy's first incarnation: we will release early, release often, accumulate users, nurture contributors, and grow our community. == Documentation == - * Subversion repository: [http://www.rectangular.com/svn/kinosearch/] * Perl API documentation: [http://www.rectangular.com/kinosearch/docs/devel/] * Discussion list: [http://www.rectangular.com/mailman/listinfo/kinosearch/] @@ -97, +96 @@ == External Dependencies == The Perl bindings for KinoSearch depend on a few CPAN modules which do not have Apache-compatible licenses. It will be possible to address all of these during incubation: - * JSON::XS can be eliminated by writing our own JSON parser. + * JSON::XS can be eliminated by writing our own JSON parser. * Lingua::Stem::Snowball and Lingua::StopWords are both wrappers around Snowball, which is itself BSD licensed. We can eliminate them by writing our own wrappers and bundling the Snowball files. * The build currently requires Parse::RecDescent. However, the compilation stage which requires Parse::RecDescent can be moved up into release preparation, so it can be eliminate as an end-user dependency. The build also requires modules which are now bundled with core Perl as of 5.10.0 (Module::Build, ExtUtils::CBuilder), but were not with Perl 5.8.x. Guidance is sought as to whether they are acceptable. == Required Resources == - === Mailing lists === - * lucy-dev * lucy-private (with moderated subscriptions) * lucy-commits @@ -128, +125 @@ == Sponsors == === Champion === - TBD === Nominated Mentors === - TBD === Sponsoring Entity === - Lucy is currently sponsored by Lucene as a sub-project. This proposal advocates changing Lucy's relationship with Apache from developing all new code as a Lucene sub-project, to instead assimilating existing code (KinoSearch) under the sponsorship of the Incubator.
