kaivalnp commented on PR #14178:
URL: https://github.com/apache/lucene/pull/14178#issuecomment-2622946390
> FAISS with this vector dimension does seem about 20% faster at search
I should add here that Lucene was using vectorized instructions via Panama,
but the C_API of Faiss was not..
I tweaked the offline build to use AVX512 instructions from Faiss as well
(basically link it to `libfaiss_avx512.so` instead of `libfaiss.so`):
Lucene:
```
recall latency (ms) nDoc topK fanout maxConn beamWidth quantized
index s index docs/s force merge s num segments index size (MB) vec disk
(MB) vec RAM (MB)
0.812 1.424 200000 100 50 32 200 no
145.30 1376.49 0.01 1 236.93
228.882 228.882
```
Faiss:
```
recall latency (ms) nDoc topK fanout maxConn beamWidth quantized
index s index docs/s force merge s num segments index size (MB) vec disk
(MB) vec RAM (MB)
0.811 1.127 200000 100 50 32 200 no
129.18 1548.20 0.01 1 511.20
228.882 228.882
```
..and we do see slightly faster indexing times
> number of vectors it must visit when searching the graph
Faiss has an
[`HNSWStats`](https://github.com/facebookresearch/faiss/blob/1334d169b9279a8c581835ef6025f4752b246ce8/faiss/impl/HNSW.h#L234)
struct exposed via a [global
variable](https://github.com/facebookresearch/faiss/blob/1334d169b9279a8c581835ef6025f4752b246ce8/faiss/impl/HNSW.h#L256)
-- I'll try to access this from Java somehow
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
