[ https://issues.apache.org/jira/browse/SOLR-769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Brad Giaccio updated SOLR-769: ------------------------------ Attachment: clustering-componet-shard.patch Okay I've rewritten the patch, as I suggested. Now the clustering happens in finishStage for distributed queries and it happens in process for non-distributed both by calling the new method clusterResults . To make this happen I had to convert the interfaces and supporting code to use SolrDocumentList rather than DocList. I've added a unit test which extends TestDistributedSearch, I had to modify TestDistributedSearch and make a bunch of things protected. This allowed me to write a very small test case (just had to override doTest) and leave all the logic for creating shards, distributing docs, and comparing responses in TestDistributedSearch. I felt this made for a very clean way to test a single distributed component. > Support Document and Search Result clustering > --------------------------------------------- > > Key: SOLR-769 > URL: https://issues.apache.org/jira/browse/SOLR-769 > Project: Solr > Issue Type: New Feature > Reporter: Grant Ingersoll > Assignee: Grant Ingersoll > Priority: Minor > Fix For: 1.4 > > Attachments: clustering-componet-shard.patch, clustering-libs.tar, > clustering-libs.tar, SOLR-769-analyzerClass.patch, SOLR-769-lib.zip, > SOLR-769.patch, SOLR-769.patch, SOLR-769.patch, SOLR-769.patch, > SOLR-769.patch, SOLR-769.patch, SOLR-769.patch, SOLR-769.patch, > SOLR-769.patch, SOLR-769.patch, SOLR-769.patch, SOLR-769.tar, SOLR-769.zip > > > Clustering is a useful tool for working with documents and search results, > similar to the notion of dynamic faceting. Carrot2 > (http://project.carrot2.org/) is a nice, BSD-licensed, library for doing > search results clustering. Mahout (http://lucene.apache.org/mahout) is well > suited for whole-corpus clustering. > The patch I lays out a contrib module that starts off w/ an integration of a > SearchComponent for doing clustering and an implementation using Carrot. In > search results mode, it will use the DocList as the input for the cluster. > While Carrot2 comes w/ a Solr input component, it is not the same as the > SearchComponent that I have in that the Carrot example actually submits a > query to Solr, whereas my SearchComponent is just chained into the Component > list and uses the ResponseBuilder to add in the cluster results. > While not fully fleshed out yet, the collection based mode will take in a > list of ids or just use the whole collection and will produce clusters. > Since this is a longer, typically offline task, there will need to be some > type of storage mechanism (and replication??????) for the clusters. I _may_ > push this off to a separate JIRA issue, but I at least want to present the > use case as part of the design of this component/contrib. It may even make > sense that we split this out, such that the building piece is something like > an UpdateProcessor and then the SearchComponent just acts as a lookup > mechanism. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.