Hi,
sometime ago I was asked some feedback about InstantiatedIndex, I lost
the identity of the person who asked this so I decided to post some info
on this list, sorry.
It is not a benchmark but just raw debug results, do not expect
scientific/usable results.
We index a small set of documents and performs few queries on the index.
We use our own query API so we do not use any Scorer nor Searcher from
lucene code. We rely only on IW and IR.
Internal indexation time : represents the time for addDocument on IW.
Optimize duration : represents the time of iterating over TermEnum for
the whole query tree.
next/exportForSort/export: (time in nano seconds)
- next represents the time for the next() call on
our query tree and on leaf loop over a termPositions or a
termDocs
- exportForSort represents the time for the IR.document() call
with a single Field in the FieldSelector.
- export represent the time for the document() call with all
needed field + access to TermPositionVector on one field.
We did the test with a RAMDirectory vs InstantiatedIndex.
While it is not a benchmark results vary upon calls but few trends
are revealed.
We noticed that indexation time for RAMDirectory is very stable and do
not vary between calls, we have a peak for the 10th doc.
With InstantiatedIndex indexation time vary between calls but we do not
have the peak and is globally slower (If you do not consider the peak).
On the queries InstantiatedIndex perform much better than RAMDirectory.
We use the fresh lucene 2.9 and a 1.6.0.16 32 bits Sun JVM on linux x64
system.
First the numbers for RAMDirectory (sorry it is not very readable):
3019 [main] DEBUG spotter - Internal indexation time 11 ms
3032 [main] DEBUG spotter - Internal indexation time 5 ms
3053 [main] DEBUG spotter - Internal indexation time 8 ms
3081 [main] DEBUG spotter - Internal indexation time 9 ms
3097 [main] DEBUG spotter - Internal indexation time 4 ms
3114 [main] DEBUG spotter - Internal indexation time 7 ms
3124 [main] WARN spotter - Internal indexation meta name titre has
null value, not indexing...
3136 [main] DEBUG spotter - Internal indexation time 8 ms
3152 [main] DEBUG spotter - Internal indexation time 7 ms
3159 [main] WARN spotter - Internal indexation meta name titre has
null value, not indexing...
3168 [main] DEBUG spotter - Internal indexation time 7 ms
3273 [main] DEBUG spotter - Internal indexation time 97 ms
3299 [main] DEBUG spotter - Internal indexation time 9 ms
3316 [main] DEBUG spotter - Internal indexation time 7 ms
3329 [main] DEBUG spotter - Internal indexation time 5 ms
3343 [main] DEBUG spotter - Internal indexation time 6 ms
3347 [main] DEBUG spotter - Optimize duration : 3ms
3349 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 2 797/424 029/428 340 ns, total
4 905 705, nb (tot/exp) 1/1
3350 [main] DEBUG spotter - Total time spent (1 result(s)) : 7ms
3352 [main] DEBUG spotter - Optimize duration : 1ms
3354 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 2 757/402 906/368 173 ns, total
2 249 306, nb (tot/exp) 1/1
3354 [main] DEBUG spotter - Total time spent (1 result(s)) : 3ms
3356 [main] DEBUG spotter - Optimize duration : 0ms
3366 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 136/272 985/338 887 ns, total
10 369 422, nb (tot/exp) 14/14
3367 [main] DEBUG spotter - Total time spent (14 result(s)) : 11ms
3368 [main] DEBUG spotter - Optimize duration : 1ms
3377 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 186/271 181/383 213 ns, total 9 510 161,
nb (tot/exp) 12/12
3378 [main] DEBUG spotter - Total time spent (12 result(s)) : 11ms
3379 [main] DEBUG spotter - Optimize duration : 1ms
3381 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 3 027/285 701/368 660 ns, total
1 925 851, nb (tot/exp) 1/1
3381 [main] DEBUG spotter - Total time spent (1 result(s)) : 3ms
3383 [main] DEBUG spotter - Optimize duration : 1ms
3396 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 131/258 265/497 030 ns, total
13 159 809, nb (tot/exp) 14/14
3396 [main] DEBUG spotter - Total time spent (14 result(s)) : 14ms
3398 [main] DEBUG spotter - Optimize duration : 1ms
3408 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 125/250 807/348 839 ns, total
10 683 871, nb (tot/exp) 14/14
3409 [main] DEBUG spotter - Total time spent (14 result(s)) : 12ms
3410 [main] DEBUG spotter - Optimize duration : 1ms
3411 [main] DEBUG spotter - Total time spent (0 result(s)) : 1ms
3412 [main] DEBUG spotter - Optimize duration : 1ms
3414 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 3 228/399 888/379 086 ns, total
1 942 673, nb (tot/exp) 1/1
3414 [main] DEBUG spotter - Total time spent (1 result(s)) : 3ms
3416 [main] DEBUG spotter - Optimize duration : 1ms
3420 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 596/266 276/337 983 ns, total 4 501 430,
nb (tot/exp) 4/4
3421 [main] DEBUG spotter - Total time spent (4 result(s)) : 6ms
3423 [main] DEBUG spotter - Optimize duration : 1ms
3427 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 536/284 111/369 779 ns, total 5 372 017,
nb (tot/exp) 5/5
3428 [main] DEBUG spotter - Total time spent (5 result(s)) : 7ms
3429 [main] DEBUG spotter - Optimize duration : 0ms
3433 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 15 444/371 502/370 394 ns, total
3 297 290, nb (tot/exp) 1/1
3434 [main] DEBUG spotter - Total time spent (1 result(s)) : 4ms
3436 [main] DEBUG spotter - Optimize duration : 0ms
3437 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 2 045/395 487/367 612 ns, total
2 375 419, nb (tot/exp) 1/1
3438 [main] DEBUG spotter - Total time spent (1 result(s)) : 3ms
3440 [main] DEBUG spotter - Optimize duration : 0ms
3446 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 778/263 756/352 647 ns, total 3 362 914,
nb (tot/exp) 3/3
3447 [main] DEBUG spotter - Total time spent (3 result(s)) : 8ms
3448 [main] DEBUG spotter - Optimize duration : 0ms
3453 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 552/286 027/378 822 ns, total 4 606 819,
nb (tot/exp) 5/5
3454 [main] DEBUG spotter - Total time spent (5 result(s)) : 5ms
3455 [main] DEBUG spotter - Optimize duration : 1ms
3465 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 141/264 997/370 457 ns, total
10 731 665, nb (tot/exp) 14/14
3466 [main] DEBUG spotter - Total time spent (14 result(s)) : 12ms
3467 [main] DEBUG spotter - Optimize duration : 0ms
3482 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 175/285 948/377 016 ns, total
14 349 331, nb (tot/exp) 14/14
3482 [main] DEBUG spotter - Total time spent (14 result(s)) : 15ms
Numbers for Instantiated Index :
3529 [main] DEBUG spotter - Internal indexation time 23 ms
3544 [main] DEBUG spotter - Internal indexation time 6 ms
3593 [main] DEBUG spotter - Internal indexation time 16 ms
3639 [main] DEBUG spotter - Internal indexation time 24 ms
3653 [main] DEBUG spotter - Internal indexation time 6 ms
3682 [main] DEBUG spotter - Internal indexation time 14 ms
3697 [main] WARN spotter - Internal indexation meta name titre has
null value, not indexing...
3718 [main] DEBUG spotter - Internal indexation time 17 ms
3741 [main] DEBUG spotter - Internal indexation time 11 ms
3750 [main] WARN spotter - Internal indexation meta name titre has
null value, not indexing...
3774 [main] DEBUG spotter - Internal indexation time 20 ms
3797 [main] DEBUG spotter - Internal indexation time 12 ms
3845 [main] DEBUG spotter - Internal indexation time 27 ms
3880 [main] DEBUG spotter - Internal indexation time 21 ms
3899 [main] DEBUG spotter - Internal indexation time 9 ms
3917 [main] DEBUG spotter - Internal indexation time 8 ms
3918 [main] DEBUG spotter - Optimize duration : 0ms
3920 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 1 774/93 110/260 321 ns, total
1 171 400, nb (tot/exp) 1/1
3921 [main] DEBUG spotter - Total time spent (1 result(s)) : 2ms
3922 [main] DEBUG spotter - Optimize duration : 1ms
3923 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 3 749/111 842/343 275 ns, total
1 370 942, nb (tot/exp) 1/1
3924 [main] DEBUG spotter - Total time spent (1 result(s)) : 3ms
3925 [main] DEBUG spotter - Optimize duration : 0ms
3932 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 247/77 550/310 593 ns, total 6 900 255,
nb (tot/exp) 14/14
3933 [main] DEBUG spotter - Total time spent (14 result(s)) : 8ms
3934 [main] DEBUG spotter - Optimize duration : 0ms
3940 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 292/77 183/312 467 ns, total 6 038 818,
nb (tot/exp) 12/12
3941 [main] DEBUG spotter - Total time spent (12 result(s)) : 7ms
3942 [main] DEBUG spotter - Optimize duration : 0ms
3944 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 3 935/108 603/334 062 ns, total
1 397 710, nb (tot/exp) 1/1
3944 [main] DEBUG spotter - Total time spent (1 result(s)) : 2ms
3945 [main] DEBUG spotter - Optimize duration : 0ms
3951 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 131/69 608/255 846 ns, total 5 903 471,
nb (tot/exp) 14/14
3952 [main] DEBUG spotter - Total time spent (14 result(s)) : 7ms
3953 [main] DEBUG spotter - Optimize duration : 0ms
3959 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 140/62 788/262 402 ns, total 5 793 178,
nb (tot/exp) 14/14
3960 [main] DEBUG spotter - Total time spent (14 result(s)) : 6ms
3960 [main] DEBUG spotter - Optimize duration : 0ms
3961 [main] DEBUG spotter - Total time spent (0 result(s)) : 1ms
3962 [main] DEBUG spotter - Optimize duration : 0ms
3963 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 1 965/93 425/268 487 ns, total
1 184 984, nb (tot/exp) 1/1
3964 [main] DEBUG spotter - Total time spent (1 result(s)) : 2ms
3965 [main] DEBUG spotter - Optimize duration : 0ms
3967 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 491/72 793/281 611 ns, total 2 350 723,
nb (tot/exp) 4/4
3968 [main] DEBUG spotter - Total time spent (4 result(s)) : 3ms
3969 [main] DEBUG spotter - Optimize duration : 0ms
3977 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 401/63 903/1 295 728 ns, total
7 711 169, nb (tot/exp) 5/5
3978 [main] DEBUG spotter - Total time spent (5 result(s)) : 9ms
3979 [main] DEBUG spotter - Optimize duration : 0ms
3981 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 14 021/111 567/338 183 ns, total
1 648 622, nb (tot/exp) 1/1
3982 [main] DEBUG spotter - Total time spent (1 result(s)) : 3ms
3983 [main] DEBUG spotter - Optimize duration : 0ms
3984 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 3 293/112 318/341 581 ns, total
1 415 179, nb (tot/exp) 1/1
3985 [main] DEBUG spotter - Total time spent (1 result(s)) : 2ms
3986 [main] DEBUG spotter - Optimize duration : 0ms
3989 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 1 136/87 271/318 269 ns, total
2 217 341, nb (tot/exp) 3/3
3989 [main] DEBUG spotter - Total time spent (3 result(s)) : 3ms
3991 [main] DEBUG spotter - Optimize duration : 0ms
3994 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 630/82 683/316 651 ns, total 3 105 836,
nb (tot/exp) 5/5
3995 [main] DEBUG spotter - Total time spent (5 result(s)) : 4ms
3996 [main] DEBUG spotter - Optimize duration : 0ms
4001 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 115/60 577/258 742 ns, total 5 592 868,
nb (tot/exp) 14/14
4002 [main] DEBUG spotter - Total time spent (14 result(s)) : 7ms
4003 [main] DEBUG spotter - Optimize duration : 0ms
4009 [main] DEBUG spotter - next/exportForSort/export
(MATCHES_WITH_OFFSET) average : 139/62 011/287 332 ns, total 6 125 691,
nb (tot/exp) 14/14
4010 [main] DEBUG spotter - Total time spent (14 result(s)) : 7ms
--
David Causse
Spotter
http://www.spotter.com/
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
