jpountz commented on a change in pull request #1866:
URL: https://github.com/apache/lucene-solr/pull/1866#discussion_r489311556
##########
File path: lucene/core/src/java/org/apache/lucene/document/ShapeQuery.java
##########
@@ -265,10 +265,20 @@ private Scorer getSparseScorer(final LeafReader reader,
final Weight weight, fin
final DocIdSetIterator iterator = new BitSetIterator(result, cost[0]);
return new ConstantScoreScorer(weight, boost, scoreMode, iterator);
}
- final DocIdSetBuilder docIdSetBuilder = new
DocIdSetBuilder(reader.maxDoc(), values, query.getField());
- values.intersect(getSparseVisitor(query, docIdSetBuilder));
- final DocIdSetIterator iterator = docIdSetBuilder.build().iterator();
- return new ConstantScoreScorer(weight, boost, scoreMode, iterator);
+ if (values.getDocCount() << 2 < values.size()) {
+ // we use a dense structure so we can skip already visited documents
+ final FixedBitSet result = new FixedBitSet(reader.maxDoc());
+ final long[] cost = new long[]{reader.maxDoc()};
Review comment:
We just need one long?
```suggestion
final long[] cost = new long[]{1};
```
##########
File path: lucene/core/src/java/org/apache/lucene/document/ShapeQuery.java
##########
@@ -265,10 +265,20 @@ private Scorer getSparseScorer(final LeafReader reader,
final Weight weight, fin
final DocIdSetIterator iterator = new BitSetIterator(result, cost[0]);
return new ConstantScoreScorer(weight, boost, scoreMode, iterator);
}
- final DocIdSetBuilder docIdSetBuilder = new
DocIdSetBuilder(reader.maxDoc(), values, query.getField());
- values.intersect(getSparseVisitor(query, docIdSetBuilder));
- final DocIdSetIterator iterator = docIdSetBuilder.build().iterator();
- return new ConstantScoreScorer(weight, boost, scoreMode, iterator);
+ if (values.getDocCount() << 2 < values.size()) {
Review comment:
I think we should be careful with overflows, maybe divide instead of
multiplying, ie.
```suggestion
if (values.getDocCount() < (values.size() >>> 2)) {
```
##########
File path: lucene/core/src/java/org/apache/lucene/document/ShapeQuery.java
##########
@@ -265,10 +265,20 @@ private Scorer getSparseScorer(final LeafReader reader,
final Weight weight, fin
final DocIdSetIterator iterator = new BitSetIterator(result, cost[0]);
return new ConstantScoreScorer(weight, boost, scoreMode, iterator);
}
- final DocIdSetBuilder docIdSetBuilder = new
DocIdSetBuilder(reader.maxDoc(), values, query.getField());
- values.intersect(getSparseVisitor(query, docIdSetBuilder));
- final DocIdSetIterator iterator = docIdSetBuilder.build().iterator();
- return new ConstantScoreScorer(weight, boost, scoreMode, iterator);
+ if (values.getDocCount() << 2 < values.size()) {
+ // we use a dense structure so we can skip already visited documents
+ final FixedBitSet result = new FixedBitSet(reader.maxDoc());
Review comment:
I wonder if we should use SparseFixedBitSet to avoid allocating so much
memory at once.
##########
File path: lucene/core/src/java/org/apache/lucene/document/ShapeQuery.java
##########
@@ -340,7 +351,8 @@ public Relation compare(byte[] minTriangle, byte[]
maxTriangle) {
};
}
- /** create a visitor that adds documents that match the query using a sparse
bitset. (Used by INTERSECT) */
+ /** create a visitor that adds documents that match the query using a sparse
bitset. (Used by INTERSECT
+ * when the number of points <= 4 * number of docs ) */
Review comment:
```suggestion
* when the number of docs <= 4 * number of points ) */
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@lucene.apache.org
For additional commands, e-mail: issues-h...@lucene.apache.org
