[
https://issues.apache.org/jira/browse/LUCENE-3892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13397228#comment-13397228
]
Han Jiang edited comment on LUCENE-3892 at 8/8/12 3:35 AM:
-----------------------------------------------------------
And result for PFor(blocksize=128):
{noformat}
Task QPS Base StdDev Base QPS PFor StdDev PFor Pct
diff
Phrase 4.87 0.36 3.39 0.18 -38% -
-20% (-47% - -25%)
AndHighMed 27.78 2.35 21.13 0.52 -31% -
-14% (-37% - -15%)
SpanNear 2.70 0.14 2.20 0.11 -26% -
-9% (-36% - -13%)
SloppyPhrase 4.17 0.15 3.77 0.21 -17% -
0% (-30% - -6%)
Respell 39.97 1.56 37.65 1.95 -14% -
3% (-15% - 2%)
Wildcard 19.08 0.77 18.33 0.92 -12% -
5% (-17% - 3%)
Fuzzy1 42.29 1.13 40.78 1.44 -9% -
2% (-11% - 1%)
AndHighHigh 7.61 0.55 7.45 0.08 -9% -
6% (-19% - 6%)
Fuzzy2 15.79 0.55 15.64 0.70 -8% -
7% (-11% - 6%)
PKLookup 86.71 2.13 88.92 2.24 -2% -
7% ( -2% - 7%)
TermGroup1M 13.04 0.23 14.03 0.40 2% -
12% ( 1% - 9%)
IntNRQ 3.97 0.48 4.35 0.61 -15% -
41% (-16% - 24%)
TermBGroup1M1P 21.04 0.35 23.20 0.60 5% -
14% ( 0% - 14%)
TermBGroup1M 19.27 0.47 21.28 0.84 3% -
17% ( 1% - 10%)
OrHighHigh 4.13 0.47 4.63 0.27 -5% -
34% (-14% - 27%)
OrHighMed 4.95 0.59 5.58 0.34 -5% -
35% (-14% - 27%)
Prefix3 30.33 1.36 34.26 2.14 1% -
25% ( -6% - 20%)
Term 41.99 1.19 50.75 1.72 13% -
28% ( 2% - 26%)
{noformat}
-It works, and it is quite interesting that StdDev for Term query is reduced
significantly.- (same as last comment, when comparing two versions
directly(method call vs. unfolded, the improvement is somewhat noisy))
was (Author: billy):
And result for PFor(blocksize=128):
{noformat}
Task QPS Base StdDev Base QPS PFor StdDev PFor Pct
diff
Phrase 4.87 0.36 3.39 0.18 -38% -
-20% (-47% - -25%)
AndHighMed 27.78 2.35 21.13 0.52 -31% -
-14% (-37% - -15%)
SpanNear 2.70 0.14 2.20 0.11 -26% -
-9% (-36% - -13%)
SloppyPhrase 4.17 0.15 3.77 0.21 -17% -
0% (-30% - -6%)
Respell 39.97 1.56 37.65 1.95 -14% -
3% (-15% - 2%)
Wildcard 19.08 0.77 18.33 0.92 -12% -
5% (-17% - 3%)
Fuzzy1 42.29 1.13 40.78 1.44 -9% -
2% (-11% - 1%)
AndHighHigh 7.61 0.55 7.45 0.08 -9% -
6% (-19% - 6%)
Fuzzy2 15.79 0.55 15.64 0.70 -8% -
7% (-11% - 6%)
PKLookup 86.71 2.13 88.92 2.24 -2% -
7% ( -2% - 7%)
TermGroup1M 13.04 0.23 14.03 0.40 2% -
12% ( 1% - 9%)
IntNRQ 3.97 0.48 4.35 0.61 -15% -
41% (-16% - 24%)
TermBGroup1M1P 21.04 0.35 23.20 0.60 5% -
14% ( 0% - 14%)
TermBGroup1M 19.27 0.47 21.28 0.84 3% -
17% ( 1% - 10%)
OrHighHigh 4.13 0.47 4.63 0.27 -5% -
34% (-14% - 27%)
OrHighMed 4.95 0.59 5.58 0.34 -5% -
35% (-14% - 27%)
Prefix3 30.33 1.36 34.26 2.14 1% -
25% ( -6% - 20%)
Term 41.99 1.19 50.75 1.72 13% -
28% ( 2% - 26%)
{noformat}
-It works, and it is quite interesting that StdDev for Term query is reduced
significantly. - (same as last comment, when comparing two versions
directly(method call vs. unfolded, the improvement is somewhat noisy))
> Add a useful intblock postings format (eg, FOR, PFOR, PFORDelta,
> Simple9/16/64, etc.)
> -------------------------------------------------------------------------------------
>
> Key: LUCENE-3892
> URL: https://issues.apache.org/jira/browse/LUCENE-3892
> Project: Lucene - Core
> Issue Type: Improvement
> Reporter: Michael McCandless
> Labels: gsoc2012, lucene-gsoc-12
> Fix For: 4.1
>
> Attachments: LUCENE-3892-BlockTermScorer.patch,
> LUCENE-3892-blockFor&hardcode(base).patch,
> LUCENE-3892-blockFor&packedecoder(comp).patch,
> LUCENE-3892-blockFor-with-packedints-decoder.patch,
> LUCENE-3892-blockFor-with-packedints-decoder.patch,
> LUCENE-3892-blockFor-with-packedints.patch,
> LUCENE-3892-direct-IntBuffer.patch, LUCENE-3892-for&pfor-with-javadoc.patch,
> LUCENE-3892-handle_open_files.patch,
> LUCENE-3892-pfor-compress-iterate-numbits.patch,
> LUCENE-3892-pfor-compress-slow-estimate.patch, LUCENE-3892_for_byte[].patch,
> LUCENE-3892_for_int[].patch, LUCENE-3892_for_unfold_method.patch,
> LUCENE-3892_pfor_unfold_method.patch, LUCENE-3892_pulsing_support.patch,
> LUCENE-3892_settings.patch, LUCENE-3892_settings.patch
>
>
> On the flex branch we explored a number of possible intblock
> encodings, but for whatever reason never brought them to completion.
> There are still a number of issues opened with patches in different
> states.
> Initial results (based on prototype) were excellent (see
> http://blog.mikemccandless.com/2010/08/lucene-performance-with-pfordelta-codec.html
> ).
> I think this would make a good GSoC project.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
