RKSPD commented on PR #14892:
URL: https://github.com/apache/lucene/pull/14892#issuecomment-3026030820
## Initial Benchmark Results
### Small Corpus Testing (Wikipedia Cohere 768, 200k docs)
```
Results: Lucene
recall latency(ms) netCPU avgCpuCount nDoc topK fanout maxConn
beamWidth quantized index(s) index_docs/s force_merge(s) num_segments
index_size(MB) overSample vec_disk(MB) vec_RAM(MB) indexType
0.803 2.800 2.343 0.837 200000 100 300 12
16 7 bits 8.46 23646.25 7.58 1
736.49 1.000 733.185 147.247 HNSW
0.822 2.486 2.286 0.920 200000 100 300 12
20 7 bits 7.33 27273.97 7.45 1
736.76 1.000 733.185 147.247 HNSW
0.857 2.657 2.429 0.914 200000 100 300 12
28 7 bits 13.64 14658.46 8.97 1
737.15 1.000 733.185 147.247 HNSW
0.831 2.771 2.514 0.907 200000 100 300 16
16 7 bits 6.44 31075.20 7.42 1
736.61 1.000 733.185 147.247 HNSW
0.846 2.857 2.571 0.900 200000 100 300 16
20 7 bits 7.19 27812.54 8.42 1
736.86 1.000 733.185 147.247 HNSW
0.869 3.029 2.657 0.877 200000 100 300 16
28 7 bits 8.47 23626.70 10.04 1
737.17 1.000 733.185 147.247 HNSW
0.847 2.829 2.486 0.879 200000 100 300 20
16 7 bits 6.11 32717.16 7.05 1
736.68 1.000 733.185 147.247 HNSW
0.862 2.743 2.429 0.885 200000 100 300 20
20 7 bits 6.92 28893.38 8.13 1
736.88 1.000 733.185 147.247 HNSW
0.883 3.086 2.743 0.889 200000 100 300 20
28 7 bits 7.94 25176.23 8.90 1
737.26 1.000 733.185 147.247 HNSW
0.860 2.943 2.657 0.903 200000 100 300 24
16 7 bits 9.37 21342.44 7.21 1
736.69 1.000 733.185 147.247 HNSW
0.880 3.371 3.143 0.932 200000 100 300 24
20 7 bits 7.77 25749.97 8.38 1
736.92 1.000 733.185 147.247 HNSW
0.900 3.086 2.886 0.935 200000 100 300 24
28 7 bits 8.37 23900.57 9.70 1
737.29 1.000 733.185 147.247 HNSW
```
```
Results: JVector
recall latency(ms) netCPU avgCpuCount nDoc topK fanout maxConn
beamWidth quantized index(s) index_docs/s force_merge(s) num_segments
index_size(MB) vec_disk(MB) vec_RAM(MB) indexType
0.877 3.943 3.714 0.942 200000 100 300 12
16 7 bits 12.94 15458.34 101.28 1
1197.28 733.185 147.247 HNSW
0.901 3.771 3.629 0.962 200000 100 300 12
20 7 bits 13.89 14394.70 123.37 1
1197.28 733.185 147.247 HNSW
0.913 3.457 3.314 0.959 200000 100 300 12
28 7 bits 18.52 10802.05 136.17 1
1197.28 733.185 147.247 HNSW
0.915 3.743 3.571 0.954 200000 100 300 16
16 7 bits 15.16 13193.48 118.83 1
1200.28 733.185 147.247 HNSW
0.921 4.029 3.857 0.957 200000 100 300 16
20 7 bits 18.83 10620.22 134.91 1
1200.28 733.185 147.247 HNSW
0.931 3.886 3.714 0.956 200000 100 300 16
28 7 bits 22.87 8746.61 174.35 1
1200.28 733.185 147.247 HNSW
0.921 5.400 5.257 0.974 200000 100 300 20
16 7 bits 15.68 12758.36 126.82 1
1203.30 733.185 147.247 HNSW
0.929 4.229 4.057 0.959 200000 100 300 20
20 7 bits 19.68 10161.57 152.86 1
1203.30 733.185 147.247 HNSW
0.942 4.343 4.171 0.961 200000 100 300 20
28 7 bits 27.79 7197.35 212.50 1
1203.30 733.185 147.247 HNSW
0.930 4.257 4.086 0.960 200000 100 300 24
16 7 bits 17.47 11449.51 131.11 1
1206.33 733.185 147.247 HNSW
0.943 4.314 4.143 0.960 200000 100 300 24
20 7 bits 21.34 9371.63 162.54 1
1206.33 733.185 147.247 HNSW
0.940 4.914 4.743 0.965 200000 100 300 24
28 7 bits 29.75 6722.24 235.78 1
1206.33 733.185 147.247 HNSW
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
