[
https://issues.apache.org/jira/browse/LUCENE-3099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13040681#comment-13040681
]
Martijn van Groningen commented on LUCENE-3099:
-----------------------------------------------
Also some initial performance indications. This data was created with the
runner provided in the patch. A ran it on a index of 30M documents that has
7502 unique groups.
{code}
===== First pass collectors execution =====
number |collector |query
|time (ms) |Current mem usage (MB)
1 |FirstPassGroupingCollector |country:es |888
|170
2 |ResearchFirstPassGroupingCollector |country:es |710
|170
3 |FirstPassGroupingCollector |country:es country:eg
|1448 |170
4 |ResearchFirstPassGroupingCollector |country:es country:eg
|1350 |170
5 |FirstPassGroupingCollector |country:it |2
|170
6 |ResearchFirstPassGroupingCollector |country:it |3
|170
7 |FirstPassGroupingCollector |country:gr |45
|170
8 |ResearchFirstPassGroupingCollector |country:gr |43
|170
9 |FirstPassGroupingCollector |country:it |3
|170
10 |ResearchFirstPassGroupingCollector |country:it |3
|170
11 |FirstPassGroupingCollector |country:it |2
|170
12 |ResearchFirstPassGroupingCollector |country:it |3
|170
13 |FirstPassGroupingCollector |country:eg |67
|170
14 |ResearchFirstPassGroupingCollector |country:eg |91
|171
15 |FirstPassGroupingCollector |country:es country:eg
|1387 |171
16 |ResearchFirstPassGroupingCollector |country:es country:eg
|1351 |171
17 |FirstPassGroupingCollector |country:e*
|2264 |173
18 |ResearchFirstPassGroupingCollector |country:e*
|1036 |176
19 |FirstPassGroupingCollector |country:it |2
|176
20 |ResearchFirstPassGroupingCollector |country:it |3
|176
21 |FirstPassGroupingCollector |*:* |526
|176
22 |ResearchFirstPassGroupingCollector |*:* |410
|176
23 |FirstPassGroupingCollector |country:gr |50
|177
24 |ResearchFirstPassGroupingCollector |country:gr |45
|177
25 |FirstPassGroupingCollector |country:es country:eg
|1397 |177
26 |ResearchFirstPassGroupingCollector |country:es country:eg
|1362 |177
27 |FirstPassGroupingCollector |*:* |544
|177
28 |ResearchFirstPassGroupingCollector |*:* |411
|177
29 |FirstPassGroupingCollector |country:tr |54
|177
30 |ResearchFirstPassGroupingCollector |country:tr |52
|177
31 |FirstPassGroupingCollector |*:* |531
|177
32 |ResearchFirstPassGroupingCollector |*:* |413
|177
33 |FirstPassGroupingCollector |*:* |534
|177
34 |ResearchFirstPassGroupingCollector |*:* |416
|177
35 |FirstPassGroupingCollector |country:eg |67
|177
36 |ResearchFirstPassGroupingCollector |country:eg |64
|177
37 |FirstPassGroupingCollector |country:gr |45
|177
38 |ResearchFirstPassGroupingCollector |country:gr |47
|177
39 |FirstPassGroupingCollector |country:e*
|1109 |181
40 |ResearchFirstPassGroupingCollector |country:e* |965
|185
41 |FirstPassGroupingCollector |country:us |56
|185
42 |ResearchFirstPassGroupingCollector |country:us |62
|185
43 |FirstPassGroupingCollector |country:gr |44
|185
44 |ResearchFirstPassGroupingCollector |country:gr |42
|185
45 |FirstPassGroupingCollector |country:es |872
|185
46 |ResearchFirstPassGroupingCollector |country:es |723
|185
47 |FirstPassGroupingCollector |country:es |804
|185
48 |ResearchFirstPassGroupingCollector |country:es |717
|185
49 |FirstPassGroupingCollector |country:es |747
|185
50 |ResearchFirstPassGroupingCollector |country:es |723
|185
===== All groups collectors execution =====
number |collector |query |time (ms)
|count |Current mem usage (MB)
1 |AllGroupsCollector |country:us |54
|11 |185
2 |ResearchAllGroupsCollector |country:us |53
|11 |169
3 |AllGroupsCollector |country:it |2
|9 |169
4 |ResearchAllGroupsCollector |country:it |2
|9 |169
5 |AllGroupsCollector |country:tr |64
|1700 |170
6 |ResearchAllGroupsCollector |country:tr |67
|1700 |170
7 |AllGroupsCollector |*:* |454
|7502 |170
8 |ResearchAllGroupsCollector |*:* |471
|7502 |170
9 |AllGroupsCollector |country:tr |51
|1700 |171
10 |ResearchAllGroupsCollector |country:tr |61
|1700 |171
11 |AllGroupsCollector |country:es |717
|3012 |171
12 |ResearchAllGroupsCollector |country:es |755
|3012 |171
13 |AllGroupsCollector |country:es |691
|3012 |172
14 |ResearchAllGroupsCollector |country:es |765
|3012 |172
15 |AllGroupsCollector |*:* |411
|7502 |172
16 |ResearchAllGroupsCollector |*:* |462
|7502 |173
17 |AllGroupsCollector |country:es |685
|3012 |173
18 |ResearchAllGroupsCollector |country:es |743
|3012 |173
19 |AllGroupsCollector |country:eg |57
|646 |173
20 |ResearchAllGroupsCollector |country:eg |63
|646 |173
21 |AllGroupsCollector |country:tr |51
|1700 |173
22 |ResearchAllGroupsCollector |country:tr |54
|1700 |174
23 |AllGroupsCollector |country:it |2
|9 |174
24 |ResearchAllGroupsCollector |country:it |3
|9 |174
25 |AllGroupsCollector |*:* |433
|7502 |174
26 |ResearchAllGroupsCollector |*:* |709
|7502 |175
27 |AllGroupsCollector |country:tr |50
|1700 |175
28 |ResearchAllGroupsCollector |country:tr |59
|1700 |175
29 |AllGroupsCollector |country:gr |44
|2061 |175
30 |ResearchAllGroupsCollector |country:gr |43
|2061 |175
31 |AllGroupsCollector |country:us |17
|11 |176
32 |ResearchAllGroupsCollector |country:us |27
|11 |176
33 |AllGroupsCollector |country:tr |55
|1700 |176
34 |ResearchAllGroupsCollector |country:tr |55
|1700 |176
35 |AllGroupsCollector |country:e* |941
|3658 |180
36 |ResearchAllGroupsCollector |country:e* |1087
|3658 |184
37 |AllGroupsCollector |country:es |706
|3012 |184
38 |ResearchAllGroupsCollector |country:es |747
|3012 |184
39 |AllGroupsCollector |country:eg |57
|646 |185
40 |ResearchAllGroupsCollector |country:eg |61
|646 |185
41 |AllGroupsCollector |country:es country:eg |1507
|3658 |170
42 |ResearchAllGroupsCollector |country:es country:eg |1512
|3658 |171
43 |AllGroupsCollector |country:gr |38
|2061 |171
44 |ResearchAllGroupsCollector |country:gr |41
|2061 |172
45 |AllGroupsCollector |country:us |17
|11 |172
46 |ResearchAllGroupsCollector |country:us |19
|11 |172
47 |AllGroupsCollector |country:es |708
|3012 |172
48 |ResearchAllGroupsCollector |country:es |742
|3012 |172
49 |AllGroupsCollector |country:us |17
|11 |172
50 |ResearchAllGroupsCollector |country:us |19
|11 |173
{code}
> Grouping module should allow subclasses to set the group key per document
> -------------------------------------------------------------------------
>
> Key: LUCENE-3099
> URL: https://issues.apache.org/jira/browse/LUCENE-3099
> Project: Lucene - Java
> Issue Type: Improvement
> Reporter: Michael McCandless
> Fix For: 3.2, 4.0
>
> Attachments: LUCENE-3099.patch
>
>
> The new grouping module can only group by a single-valued indexed field.
> But, if we make the 'getGroupKey' a method that a subclass could override,
> then I think we could refactor Solr over to the module, because it could do
> function queries and normal queries via subclass (I think).
> This also makes the impl more extensible to apps that might have their own
> interesting group values per document.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
