[ 
https://issues.apache.org/jira/browse/LUCENE-3450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13113281#comment-13113281
 ] 

Karl Wright commented on LUCENE-3450:
-------------------------------------

Apologies for not finishing this ticket promptly, but minutes after creating it 
I lost internet and phone service for seven hours.

Some notes:
(1) The filter-based plugin passes the tests, the query-based one fails two out 
of the four tests.  I've been concentrating initially on the failure that comes 
from ManifoldCFSecurityFilterTest.java line 100.  Here's the error:

{code}
[junit] ------------- Standard Error -----------------
    [junit] +((-allow_token_share:* -deny_token_share:*) 
allow_token_share:token1 -deny_token_share:token1) +((-allow_token_document:* 
-deny_token_document:*) allow_token_document:token1 -deny_token_document:token1)
    [junit] 22/09/2011 08:26:50 ? org.apache.solr.SolrTestCaseJ4 assertQ
    [junit] SEVERE: REQUEST FAILED: xpath=//*[@numFound='3']
    [junit]     xml response was: <?xml version="1.0" encoding="UTF-8"?>
    [junit] <response>
    [junit] <lst name="responseHeader"><int name="status">0</int><int 
name="QTime">116</int><lst name="params"><str name="echoParams">all</str><str 
name="fl">id</str><str name="q">*:*</str><str name="qt">/mcf</str><str 
name="UserTokens">token1</str><str name="mcf">true</str></lst></lst><result 
name="response" numFound="0" start="0"></result>
    [junit] </response>
{code}

This output includes the actual query I am generating for this test, which 
looks correct to me.

(2) The test indexes 5 documents into four access token fields.  These 
documents are summarized below.

{code}
    //             |     share    |   document
    //             |--------------|--------------
    //             | allow | deny | allow | deny
    // ------------+-------+------+-------+------
    // da12        |       |      | 1, 2  |
    // ------------+-------+------+-------+------
    // da13-dd3    |       |      | 1,3   | 3
    // ------------+-------+------+-------+------
    // sa123-sd13  | 1,2,3 | 1, 3 |       |
    // ------------+-------+------+-------+------
    // sa3-sd1-da23| 3     | 1    | 2,3   |
    // ------------+-------+------+-------+------
    // notoken     |       |      |       |
    // ------------+-------+------+-------+------

{code}


(2) Initially the query-based plugin directly corresponded to the filter based 
one. Still failed in exactly the same way.  Here's the diff between the 
filter-based one and the initial query-based one:

{code}
Index: src/java/org/apache/solr/mcf/ManifoldCFSecurityFilter.java
===================================================================
--- src/java/org/apache/solr/mcf/ManifoldCFSecurityFilter.java  (revision 
1173895)
+++ src/java/org/apache/solr/mcf/ManifoldCFSecurityFilter.java  (working copy)
@@ -150,7 +150,8 @@
       userAccessTokens = getAccessTokens(authenticatedUserName);
     }
 
-    BooleanFilter bf = new BooleanFilter();
+    BooleanQuery bq = new BooleanQuery();
+    //bf.setMaxClauseCount(100000);
     
     if (userAccessTokens.size() == 0)
     {
@@ -159,28 +160,26 @@
       // (fieldAllowShare is empty AND fieldDenyShare is empty AND 
fieldAllowDocument is empty AND fieldDenyDocument is empty)
       // We're trying to map to:  -(fieldAllowShare:*) , which should be 
pretty efficient in Solr because it is negated.  If this turns out not to be 
so, then we should
       // have the SolrConnector inject a special token into these fields when 
they otherwise would be empty, and we can trivially match on that token.
-      bf.add(new FilterClause(new QueryWrapperFilter(new WildcardQuery(new 
Term(fieldAllowShare,"*"))),BooleanClause.Occur.MUST_NOT));
-      bf.add(new FilterClause(new QueryWrapperFilter(new WildcardQuery(new 
Term(fieldDenyShare,"*"))),BooleanClause.Occur.MUST_NOT));
-      bf.add(new FilterClause(new QueryWrapperFilter(new WildcardQuery(new 
Term(fieldAllowDocument,"*"))),BooleanClause.Occur.MUST_NOT));
-      bf.add(new FilterClause(new QueryWrapperFilter(new WildcardQuery(new 
Term(fieldDenyDocument,"*"))),BooleanClause.Occur.MUST_NOT));
+      bq.add(new WildcardQuery(new 
Term(fieldAllowShare,"*")),BooleanClause.Occur.MUST_NOT);
+      bq.add(new WildcardQuery(new 
Term(fieldDenyShare,"*")),BooleanClause.Occur.MUST_NOT);
+      bq.add(new WildcardQuery(new 
Term(fieldAllowDocument,"*")),BooleanClause.Occur.MUST_NOT);
+      bq.add(new WildcardQuery(new 
Term(fieldDenyDocument,"*")),BooleanClause.Occur.MUST_NOT);
     }
     else
     {
       // Extend the query appropriately for each user access token.
-      bf.add(new 
FilterClause(calculateCompleteSubfilter(fieldAllowShare,fieldDenyShare,userAccessTokens),BooleanClause.Occur.MUST));
-      bf.add(new 
FilterClause(calculateCompleteSubfilter(fieldAllowDocument,fieldDenyDocument,userAccessTokens),BooleanClause.Occur.MUST));
+      
bq.add(calculateCompleteSubquery(fieldAllowShare,fieldDenyShare,userAccessTokens),BooleanClause.Occur.MUST);
+      
bq.add(calculateCompleteSubquery(fieldAllowDocument,fieldDenyDocument,userAccessTokens),BooleanClause.Occur.MUST);
     }
 
     // Concatenate with the user's original query.
-    //FilteredQuery query = new FilteredQuery(rb.getQuery(),bf);
-    //rb.setQuery(query);
     List<Query> list = rb.getFilters();
     if (list == null)
     {
       list = new ArrayList<Query>();
       rb.setFilters(list);
     }
-    list.add(new ConstantScoreQuery(bf));
+    list.add(new ConstantScoreQuery(bq));
   }
 
   @Override
@@ -193,28 +192,27 @@
   * ((fieldAllowShare is empty AND fieldDenyShare is empty) OR fieldAllowShare 
HAS token1 OR fieldAllowShare HAS token2 ...)
   *     AND fieldDenyShare DOESN'T_HAVE token1 AND fieldDenyShare DOESN'T_HAVE 
token2 ...
   */
-  protected Filter calculateCompleteSubfilter(String allowField, String 
denyField, List<String> userAccessTokens)
+  protected Query calculateCompleteSubquery(String allowField, String 
denyField, List<String> userAccessTokens)
   {
-    BooleanFilter bf = new BooleanFilter();
+    BooleanQuery bq = new BooleanQuery();
+    bq.setMaxClauseCount(1000000);
     
     // Add a clause for each token.  This will be added directly to the main 
filter (as a deny test), as well as to an OR's subclause (as an allow test).
-    BooleanFilter orFilter = new BooleanFilter();
+    BooleanQuery orQuery = new BooleanQuery();
+    orQuery.setMaxClauseCount(1000000);
+
     // Add the empty-acl case
-    BooleanFilter subUnprotectedClause = new BooleanFilter();
-    subUnprotectedClause.add(new FilterClause(new QueryWrapperFilter(new 
WildcardQuery(new Term(allowField,"*"))),BooleanClause.Occur.MUST_NOT));
-    subUnprotectedClause.add(new FilterClause(new QueryWrapperFilter(new 
WildcardQuery(new Term(denyField,"*"))),BooleanClause.Occur.MUST_NOT));
-    orFilter.add(new 
FilterClause(subUnprotectedClause,BooleanClause.Occur.SHOULD));
+    BooleanQuery subUnprotectedClause = new BooleanQuery();
+    subUnprotectedClause.add(new WildcardQuery(new 
Term(allowField,"*")),BooleanClause.Occur.MUST_NOT);
+    subUnprotectedClause.add(new WildcardQuery(new 
Term(denyField,"*")),BooleanClause.Occur.MUST_NOT);
+    orQuery.add(subUnprotectedClause,BooleanClause.Occur.SHOULD);
     for (String accessToken : userAccessTokens)
     {
-      TermsFilter tf = new TermsFilter();
-      tf.addTerm(new Term(allowField,accessToken));
-      orFilter.add(new FilterClause(tf,BooleanClause.Occur.SHOULD));
-      tf = new TermsFilter();
-      tf.addTerm(new Term(denyField,accessToken));
-      bf.add(new FilterClause(tf,BooleanClause.Occur.MUST_NOT));
+      orQuery.add(new TermQuery(new 
Term(allowField,accessToken)),BooleanClause.Occur.SHOULD);
+      bq.add(new TermQuery(new 
Term(denyField,accessToken)),BooleanClause.Occur.MUST_NOT);
     }
-    bf.add(new FilterClause(orFilter,BooleanClause.Occur.MUST));
-    return bf;
+    bq.add(orQuery,BooleanClause.Occur.MUST);
+    return bq;
   }
{code}

After this I experimented for a bit, flattening out one level of BooleanQuery, 
commenting out the ConstantScoreQuery wrapper, and adding output to stderr of 
the generated query.  None of this fixed things, although the test behavior did 
change somewhat, which was another worrisome sign.  The failure at 
ManifoldCFSecurityFilterTest.java line 100 still occurred just the same, 
however.

This is as far as I've gotten so far.  If you need a more condensed test case, 
I'm happy to try and produce one, but I am unfamiliar with the Lucene-side 
testing infrastructure so I'd need to be pointed at an example to emulate.


> BooleanQuery seems broken on trunk
> ----------------------------------
>
>                 Key: LUCENE-3450
>                 URL: https://issues.apache.org/jira/browse/LUCENE-3450
>             Project: Lucene - Java
>          Issue Type: Bug
>            Reporter: Karl Wright
>         Attachments: filter.diff, query.diff
>
>
> Converting a Solr SearchComponent from using filters to using equivalent 
> queries causes its test cases to fail.  The reasons are unclear, and I've 
> tried several things to isolate the problem, but with no luck so far.
> The failure manifests itself by the constructed query returning no documents 
> whatsoever, at least within the Solr context it's being used in.  The 
> equivalent filter version of the component works properly (as you can see by 
> the passed test).
> To run the test:
> cd solr/contrib/mcf
> ant test
> I will attach both filter and query versions of the code, as patches.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org

Reply via email to