rajat315315 commented on PR #16431:
URL: https://github.com/apache/lucene/pull/16431#issuecomment-5114543243
Thanks for raising this crucial point regarding monotonicity!
I have updated the PR to handle both **monotonically increasing** and
**monotonically decreasing** functions while maintaining strict safety for
arbitrary expressions:
### 1. 100% Sound & Safe Fallback (`Float.POSITIVE_INFINITY`) for Arbitrary
Expressions
In `DoubleValues.java`, the default interface implementation of
`getMaxScore` is conservative:
```java
public float getMaxScore(int upTo) throws IOException {
return Float.POSITIVE_INFINITY;
}
```
Any `DoubleValuesSource` (such as `ExpressionValueSource`, script functions,
or un-analyzed custom functions) that does **not** override `getMaxScore` will
automatically return `Float.POSITIVE_INFINITY`.
In `FunctionScoreQuery`:
```java
float valueMaxScore = scores.getMaxScore(upTo);
if (Float.isInfinite(valueMaxScore)) {
return Float.POSITIVE_INFINITY;
}
```
When `valueMaxScore` is `Float.POSITIVE_INFINITY`, Block-Max WAND dynamic
block pruning is safely bypassed. This guarantees **zero correctness bugs** for
non-monotonic or arbitrary expression sources.
---
### 2. Support for Both Increasing AND Decreasing Monotonic Functions
Because `DocValuesSkipper` provides **both** `skipper.minValue(0)` and
`skipper.maxValue(0)` for every segment block, I added an `increasing` boolean
parameter to `DoubleValuesSource.fromField()`:
```java
DoubleValuesSource.fromField(String field, LongToDoubleFunction decoder,
boolean increasing)
```
* **Increasing Functions** (`increasing = true`, default for $x, \log(x),
\sqrt{x}$):
$$\text{BlockMaxScore} =
\text{decoder.applyAsDouble}(\text{skipper.maxValue}(0))$$
* **Decreasing Functions** (`increasing = false`, for $-x$, $1/x$,
reciprocal decay, distance decay $e^{-\lambda \cdot x}$):
$$\text{BlockMaxScore} =
\text{decoder.applyAsDouble}(\text{skipper.minValue}(0))$$
For a monotonically decreasing function, the maximum output score occurs at
the **smallest** raw field value (`skipper.minValue(0)`), giving exact and
sound block upper bounds for WAND pruning.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
