Hi all, I've just added first draft of proposal for search improvements. So, let's discuss it :)
I suggest that we agree on functional requirements and then pass to possible implementation. Regards, Andrej On 22 November 2012 11:32, Apache Bloodhound < [email protected]> wrote: > Page "Proposals/BEP-0004" was added by andrej > Content: > -------8<------8<------8<------8<------8<------8<------8<------8<-------- > > = BEP 4 : Improved search architecture #overview > > [[PageOutline]] > > || '''PEP''' || 4 || > || '''Title''' || Improved search architecture || > || '''Version''' || || > || '''Last-Modified''' || || > || '''Author''' || The Bloodhound project || > || '''Status''' || Draft || > || '''Type''' || Standards Trac || > || '''Content-Type''' || [wiki:PageTemplates/Proposals text/x-trac-wiki] || > || '''Created''' || || > || '''Post-History''' || || > > ---- > > == Abstract #abstract > This document is meant to gather community consensus on the implementation > of the improved search functionality. Simultaneously, we want to merge > search and custom query functionality under the same umbrella. This is the > place to discuss it and make proposals. Main discussion must be placed in > mail list with [BEP-0004] subject. > > == Motivation == > > Nowadays modern issue tracking system should provide advanced free text > search functionality combine with flexible query mechanism. There are a lot > of user requests asking for better and customizable search. > > The term project is very generic and may be confusing considering the > context. Therefore in this specification the word product is used instead. > > == Rationale #rationale > > === Quick search box > > Quick search box should be just frontend for adhoc query. For example, if > user searches for “bla” string, user receives Search Result view with the > following query: text~=”bla”. User can refine query in Query Result view. > > Search box should accept the following search types: > - free-text search: “bla” > - free-text + query: bla status!=closed > - query only: status!=closed > > Other nice to have features: > - Did you mean... > - Suggestions during typing > > === Query Result view > > Query Result view represents consistent view of query result for different > resources. Result may be represented as > - All resources result tab - default, common for all resources columns > e.g. name and description > - Resource result tabs - resource specific fields are shown e.g. id, > status, summary for ticket. > - Query Result view can be instructed to limit result to specific > resource type e.g. show only tickets in result. In this case, resource tabs > can be omitted. > > User can refine search conditions either by editing query or by using > Query Builder. > User can specify what fields should be selected and in what results order > should be applied. A new order type is introduced for free-text search: > score > > Query Result view should support facets ( > http://searchhub.org/2009/09/02/faceted-search-with-solr/) for example: > - All resources result tab can show facets for resource type and product > - Ticket result tab can show facets for products and ticket status > > Other nice to have features in no particular order: > - Possibility to save a query for specific user and sharing of saved > queries > - Search highlighting > > === Resource Query > > New Resource Query should provide the following functionality: > - free-text search support > - facet support > - it is a superset of TracQuery functionality > - basic query expressions AND, OR, NOT, search by specific field, search > [FROM TO] - TBD (can be similar to SQL or lucene/solr like) > - search through different resources not only tickets: wiki, milestones, > changesets and other pluggable resources > - search through all resource fields > - search through attachments, history and comments > - multi-product aware - apply security context etc > - order by free-text score. Score calculation can be configured, for > example if found in summary: id: score:100, score*10, in keywords: score*5, > in components: score*3 … > > Nice to have features > - Support search in attachments of specific types e.g word, exel etc. > > === Resource Query macro > > Similar to existing TracQuery, New macro must be introduced to enable > access to Resource Query from Wiki. > > === Query Builder > > A lot improvements may be introduced here, TBD > > == Proposal #proposal > > === Implementation steps > > We can start with small step by implementing search box functionality > similar to trac:wiki:AdvancedSearch and then add support for query, wiki > macro and query builder/wizard. > > === Possible free text platforms > > A few possible solutions were discussed on the mail list: > > ==== Whoosh > > Whoosh links: > - Product page: http://pypi.python.org/pypi/Whoosh/ > - Discussion on implementing Trac search using Whoosh: > https://groups.google.com/forum/?fromgroups=#!msg/trac-dev/sbU-g0C6kvk/1_juL29aAtQJ > > ==== !PyLucen > > [http://lucene.apache.org/pylucene/ PyLucene] is not a Lucene port but a > Python wrapper around Java Lucene. !PyLucene embeds a Java VM with Lucene > into a Python process. > > ==== Java-based servers. > Solr or !ElasticSearch > > == Rejected ideas > > == Backwards Compatibility #backwards-compatibility > > TracQuery implementation must be changed to use Resource Query for data > retrieval. > > Existing TracSearch syntax will be mapped to Resource Query. > > == Reference Implementation #reference-implementation > > == Resources #resources > > == References #references > Proposals/BEP-0003 - Multi-product architecture > > === Trac proposals on this subject > - trac:wiki:AdvancedSearch > - trac:wiki:SearchRefactoring > > === Plugins on this subject > - http://trac-hacks.org/wiki/FullTextSearchPlugin from Logica > - http://trac-hacks.org/wiki/TracAdvancedSearchPlugin - uses Solr, > claims to be designed with extensions points > > === Existing search and query implementation > - TracQuery > - TracSearch > > == Copyright #copyright > > Copyright © 2009-2012 The [http://www.apache.org Apache Software > Foundation] [[BR]] > Licensed under the [http://www.apache.org/licenses/LICENSE-2.0 Apache > License, Version 2.0]. > > Apache Bloodhound, Apache, the Apache feather logo, and the Apache > Bloodhound project logo are trademarks of The Apache Software Foundation. > -------8<------8<------8<------8<------8<------8<------8<------8<-------- > > -- > Page URL: <https://issues.apache.org/bloodhound/wiki/Proposals/BEP-0004> > Apache Bloodhound <https://issues.apache.org/bloodhound/> > The Apache Bloodhound (incubating) issue tracker > > This is an automated message. Someone added your email address to be > notified of changes on 'Proposals/BEP-0004' page. > If it was not you, please report to . >
