Rewrite causes BooleanQuery to loose required terms

[Nick Burch]

Hi All

I'm using lucene from CVS, and I've discovered the rewriting a 
BooleanQuery created with the old style (Query,boolean,boolean) method,
the rewrite will cause the required parameters to get lost.

Using old style (Query,boolean,boolean):
query = +contents:test* +(class:1.2 class:1.2.*)
rewritten query = (contents:tester contents:testing contents:tests) 
  (class:1.2 (class:1.2.3 class:1.2.4))

Using new style (Query,BooleanClause.Occur.MUST):
query = +contents:test* +(class:1.2 class:1.2.*)
rewritten query = +(contents:tester contents:testing contents:tests) 
  +(class:1.2 (class:1.2.3 class:1.2.4))

Attached is a simple RAMDirectory test to show this. I know that the 
(Query,boolean,boolean) method is depricated, but should it also be 
broken?

Thanks
Nick

/*
 * For testing to see if there are problems with rewriting
 *
 * Should show
 *    +contents:test* +(class:1.2 class:1.2.*)
 * Goes to with old style
 *    (contents:tester contents:testing contents:tests) (class:1.2 (class:1.2.3 
class:1.2.4))
 * Goes to (correctly) with new style
 *    +(contents:tester contents:testing contents:tests) +(class:1.2 
(class:1.2.3 class:1.2.4))
 *
 * Nick Burch <nick at torchbox dot com>
 */

import org.apache.lucene.index.*;
import org.apache.lucene.store.*;
import org.apache.lucene.search.*;
import org.apache.lucene.document.*;

import org.apache.lucene.analysis.SimpleAnalyzer;
import org.apache.lucene.analysis.Analyzer;

import java.io.IOException;

public class rewritingTest {
public static void main(String args[]) throws IOException {
        // Create a ram directory with a few test entries in it
        RAMDirectory dir = new RAMDirectory();

        // Create an analyzer
        Analyzer a = new SimpleAnalyzer();

        // Open an index writer
        IndexWriter writer = new IndexWriter(dir,a,true);

        // Add some test docs
        Document doc;
        Field f;

        // Doc 1
        doc = new Document();
        f = new 
Field("contents","testing",Field.Store.YES,Field.Index.TOKENIZED);
        doc.add(f);
        f = new Field("title","testing",Field.Store.YES,Field.Index.TOKENIZED);
        doc.add(f);
        f = new Field("class","1.2.3",Field.Store.YES,Field.Index.UN_TOKENIZED);
        doc.add(f);
        f = new Field("class","1.2.4",Field.Store.YES,Field.Index.UN_TOKENIZED);
        doc.add(f);
        writer.addDocument(doc);

        // Doc 2
        doc = new Document();
        f = new Field("contents","tests",Field.Store.YES,Field.Index.TOKENIZED);
        doc.add(f);
        f = new Field("tilte","tests",Field.Store.YES,Field.Index.TOKENIZED);
        doc.add(f);
        f = new Field("class","1.3.3",Field.Store.YES,Field.Index.UN_TOKENIZED);
        doc.add(f);
        f = new Field("class","1.3.4",Field.Store.YES,Field.Index.UN_TOKENIZED);
        doc.add(f);
        writer.addDocument(doc);

        // Doc 3
        doc = new Document();
        f = new 
Field("contents","tester",Field.Store.YES,Field.Index.TOKENIZED);
        doc.add(f);
        f = new Field("tilte","tester",Field.Store.YES,Field.Index.TOKENIZED);
        doc.add(f);
        f = new Field("class","1.4",Field.Store.YES,Field.Index.UN_TOKENIZED);
        doc.add(f);
        f = new Field("class","1.5",Field.Store.YES,Field.Index.UN_TOKENIZED);
        doc.add(f);
        writer.addDocument(doc);

        // Now get a searcher
        writer.close();
        IndexReader reader = IndexReader.open(dir);
        IndexSearcher search = new IndexSearcher(reader);

        // Construct a nest of queries
        BooleanQuery overallQueryOld = new BooleanQuery();
        BooleanQuery overallQueryNew = new BooleanQuery();

        // Contents query
        Term contentsT = new Term("contents","test*");
        WildcardQuery contents = new WildcardQuery(contentsT);
        overallQueryOld.add(contents,true,false);
        overallQueryNew.add(contents,BooleanClause.Occur.MUST);

        // Classifcation Query
        BooleanQuery classQ = new BooleanQuery();
        Term justClassT = new Term("class","1.2");
        Term classChildrenT = new Term("class","1.2.*");
        TermQuery justClass = new TermQuery(justClassT);
        WildcardQuery classChildren = new WildcardQuery(classChildrenT);
        classQ.add(justClass,false,false);
        classQ.add(classChildren,false,false);
        overallQueryOld.add(classQ,true,false);
        overallQueryNew.add(classQ,BooleanClause.Occur.MUST);

        System.out.println(overallQueryOld);
        Query rewrittenOld = overallQueryOld.rewrite(reader);
        System.out.println(rewrittenOld);

        System.out.println("");
        System.out.println(overallQueryNew);
        Query rewrittenNew = overallQueryNew.rewrite(reader);
        System.out.println(rewrittenNew);
}
}

---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]