Hi
what about this
https://issues.apache.org/jira/browse/SOLR-8096
seems still unresolved issue?
With our migration from version 4 to 7 last year we experienced similar
problems.
Günter
On 08.09.19 06:09, Russell Bahr wrote:
Hi David and Toke,
Thank you both for your input. I will be in DC tomorrow evening and will
try your suggestions, and read the ref guide again on the parts that you
have pointed out. I will let you know the results, and will share your
feedback with my team to see what we can change and still bring back the
result sets that are needed for our system.
Thanks again,
Russ
*Manzama*a MODERN GOVERNANCE company
Russell Bahr
Lead Infrastructure Engineer
USA & CAN Office: +1 (541) 306 3271
USA & CAN Support: +1 (541) 706 9393
UK Office & Support: +44 (0)203 282 1633
AUS Office & Support: +61 (0) 2 8417 2339
543 NW York Drive, Suite 100, Bend, OR 97703
LinkedIn <http://www.linkedin.com/company/manzama> | Twitter
<https://twitter.com/ManzamaInc> | Facebook
<http://www.facebook.com/manzamainc> | YouTube
<https://www.youtube.com/channel/UCBo3QoqewyNoo7HiT_BFuRw>
On Sat, Sep 7, 2019 at 2:43 PM David Smiley <david.w.smi...@gmail.com>
wrote:
Also consider substituting grouping with expand/collapse (see the ref
guide). The latter performs much better in general, although grouping does
have certain options that are uniquely valuable like ensuring that facet
counts look at the aggregate (if you want that). I wish we could outright
remove grouping; it's a complexity weight on our codebase.
~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley
On Sat, Sep 7, 2019 at 5:15 PM David Smiley <david.w.smi...@gmail.com>
wrote:
10s of seconds to respond to a simple match-all query, especially to just
a single shard via using distrib=false, is very bizarre. What's the
"QTime" on one of these? -- also super long or sub-second?
I took a brief look at your schema with a hunch. I see you have
docValues=true on your ID field -- makes sense to me. You also have
version=1.5 on the schema instead of 1.6. Why did you not do 1.6? With
1.5 useDocValuesAsStored is false by default. try toggling the version
number to 1.6. And try your query with "fl=id" and see how that changes
the times.
I also took a look at your solrconfig.xml with a hunch, and now think I
found the smoking gun. I see you've modified the /select request handler
to add a bunch of defaults, including, of all things, grouping. Thus
when
you report to us your queries are simple *:* queries, the reality is far
different. I wish people would treat /select as immutable and instead
create request handlers for their apps' needs.
Nonetheless my investigation here only reveals that your test queries are
actually very complex and thus explains their overall slowness. We don't
know why Solr 8 performs slower than Solr 4 here. For that I think we've
given you some tips. Get back to a simple query and compare that. Try
certain features in isolation (e.g. *just* the grouping). Maybe it's
that. You might experiment with switching "fingerprint" (the string
field
you group on) from docValues=true to false to see if it's a docValues
perf
issue compared to uninverting.
~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley
On Sat, Sep 7, 2019 at 3:06 PM Russell Bahr <r...@manzama.com> wrote:
Hi David,
I ran the *:* query 10 times against all 30 servers and the results
(below)
were similar across all of them. I agree working against a single server
is
easier troubleshooting, but I do not know where to start.
Server shard replica, Matches, Time, Pass
16 1 n2 2989421 78800 1
20 1 n1 2989559 63246 1
23 1 n8 2989619 55141 1
28 1 n6 2989619 65536 1
17 1 n4 2989818 56694 1
26 2 n10 2990088 63485 1
21 2 n18 2990145 68077 1
11 2 n16 2990145 62271 1
13 2 n12 2990242 68564 1
27 2 n14 2990242 63739 1
10 3 n26 2988056 69117 1
25 3 n24 2988056 73750 1
12 3 n28 2988096 61948 1
6 3 n20 2988123 62174 1
19 3 n22 2988123 65826 1
1 4 n30 2985457 60404 1
29 4 n34 2985457 68498 1
30 4 n38 2985604 72034 1
9 4 n36 2902757 65943 1
15 4 n32 2985948 67208 1
7 5 n48 2992278 63098 1
5 5 n42 2992363 69503 1
8 5 n44 2992363 66818 1
4 5 n40 2992397 66784 1
14 5 n46 2883495 58759 1
3 6 n56 2878221 52265 1
22 6 n58 2878221 53768 1
24 6 n52 2878326 62174 1
2 6 n50 2878326 53143 1
18 6 n54 2878326 59044 1
Results from 10 passes
p-solr-8-16.obscured.com:8983/solr/content_shard1_replica_n2/ 69697.8
4599.8171896
Query time milliseconds [78800, 65549, 68045, 72151, 62774, 69168,
66459,
74336, 69028, 70668]
p-solr-8-20.obscured.com:8983/solr/content_shard1_replica_n1/ 58310.5
4531.23621224
Query time milliseconds [63246, 59626, 61001, 59366, 53028, 58693,
58832,
64633, 54659, 50021]
p-solr-8-23.obscured.com:8983/solr/content_shard1_replica_n8/ 57778.5
4659.23933348
Query time milliseconds [55141, 55194, 59100, 62614, 65425, 59261,
58961,
59259, 53799, 49031]
p-solr-8-28.obscured.com:8983/solr/content_shard1_replica_n6/ 64944.1
3382.61379705
Query time milliseconds [65536, 67825, 69829, 60059, 63616, 67588,
68443,
60853, 62666, 63026]
p-solr-8-17.obscured.com:8983/solr/content_shard1_replica_n4/ 58018.9
4821.9028851
Query time milliseconds [56694, 58900, 55404, 51590, 66034, 51256,
57109,
57515, 63530, 62157]
p-solr-8-26.obscured.com:8983/solr/content_shard2_replica_n10/ 59366.6
5036.84751936
Query time milliseconds [63485, 53315, 64845, 62077, 54313, 52607,
65389,
55977, 63486, 58172]
p-solr-8-21.obscured.com:8983/solr/content_shard2_replica_n18/ 61844.1
4623.13444537
Query time milliseconds [68077, 61117, 64284, 65393, 60580, 57495,
58068,
67454, 62370, 53603]
p-solr-8-11.obscured.com:8983/solr/content_shard2_replica_n16/ 61179.1
4224.86040401
Query time milliseconds [62271, 66059, 67076, 55706, 60905, 58617,
56561,
66308, 57100, 61188]
p-solr-8-13.obscured.com:8983/solr/content_shard2_replica_n12/ 69578.3
3986.83530998
Query time milliseconds [68564, 67411, 71644, 75938, 73772, 69780,
67438,
72479, 66368, 62389]
p-solr-8-27.obscured.com:8983/solr/content_shard2_replica_n14/ 59808.2
4896.04649579
Query time milliseconds [63739, 59873, 65775, 50280, 63009, 60955,
55516,
64130, 60016, 54789]
p-solr-8-10.obscured.com:8983/solr/content_shard3_replica_n26/ 66038.1
3363.29340743
Query time milliseconds [69117, 63766, 66189, 72059, 68751, 67644,
65027,
60859, 63306, 63663]
p-solr-8-25.obscured.com:8983/solr/content_shard3_replica_n24/ 60566.8
6390.11408001
Query time milliseconds [73750, 58849, 59947, 58810, 55752, 51790,
54871,
60592, 66853, 64454]
p-solr-8-12.obscured.com:8983/solr/content_shard3_replica_n28/ 63456.6
4591.56887252
Query time milliseconds [61948, 61372, 60570, 59961, 66818, 62124,
66880,
72592, 56477, 65824]
p-solr-8-6.obscured.com:8983/solr/content_shard3_replica_n20/ 60869.6
2619.82752613
Query time milliseconds [62174, 59369, 62098, 66974, 59552, 58757,
59728,
59413, 62414, 58217]
p-solr-8-19.obscured.com:8983/solr/content_shard3_replica_n22/ 57122.0
5111.33222469
Query time milliseconds [65826, 51309, 52595, 50432, 59987, 61371,
62022,
57667, 55639, 54372]
p-solr-8-1.obscured.com:8983/solr/content_shard4_replica_n30/ 61678.3
4091.35454478
Query time milliseconds [60404, 55604, 62858, 67807, 64320, 63962,
66846,
58085, 58951, 57946]
p-solr-8-29.obscured.com:8983/solr/content_shard4_replica_n34/ 64042.9
3646.86466403
Query time milliseconds [68498, 60725, 69115, 61942, 61398, 61874,
61255,
68909, 66059, 60654]
p-solr-8-30.obscured.com:8983/solr/content_shard4_replica_n38/ 63116.5
4609.50824685
Query time milliseconds [72034, 66459, 56206, 56873, 64180, 60061,
63122,
63786, 63754, 64690]
p-solr-8-9.obscured.com:8983/solr/content_shard4_replica_n36/ 63432.7
4300.8959803
Query time milliseconds [65943, 69316, 55002, 68917, 63472, 58904,
63211,
64688, 62819, 62055]
p-solr-8-15.obscured.com:8983/solr/content_shard4_replica_n32/ 61906.5
2902.66395843
Query time milliseconds [67208, 65850, 57653, 60862, 59523, 59620,
62713,
61529, 61265, 62842]
p-solr-8-7.obscured.com:8983/solr/content_shard5_replica_n48/ 59892.7
3490.10076423
Query time milliseconds [63098, 60924, 61086, 63765, 56105, 53150,
56886,
63446, 59949, 60518]
p-solr-8-5.obscured.com:8983/solr/content_shard5_replica_n42/ 62159.0
5601.61476719
Query time milliseconds [69503, 56827, 55902, 61201, 56940, 65581,
71914,
63370, 63051, 57301]
p-solr-8-8.obscured.com:8983/solr/content_shard5_replica_n44/ 63823.7
3858.87292267
Query time milliseconds [66818, 67892, 58788, 69172, 64827, 63549,
63268,
59215, 58640, 66068]
p-solr-8-4.obscured.com:8983/solr/content_shard5_replica_n40/ 58128.5
4130.89318429
Query time milliseconds [66784, 56183, 61574, 55183, 57776, 57367,
60974,
52964, 53819, 58661]
p-solr-8-14.obscured.com:8983/solr/content_shard5_replica_n46/ 61351.6
4037.90460511
Query time milliseconds [58759, 62324, 57665, 67046, 56590, 59296,
59116,
68687, 63815, 60218]
p-solr-8-3.obscured.com:8983/solr/content_shard6_replica_n56/ 56639.2
4509.2680559
Query time milliseconds [52265, 49244, 51974, 60207, 57325, 58683,
63489,
54787, 61009, 57409]
p-solr-8-22.obscured.com:8983/solr/content_shard6_replica_n58/ 57532.8
2690.59624949
Query time milliseconds [53768, 57966, 57145, 53659, 60496, 56692,
59881,
56240, 61887, 57594]
p-solr-8-24.obscured.com:8983/solr/content_shard6_replica_n52/ 65245.2
3561.3285536
Query time milliseconds [62174, 63386, 59160, 68495, 70150, 64167,
70341,
66024, 64306, 64249]
p-solr-8-2.obscured.com:8983/solr/content_shard6_replica_n50/ 58632.8
3571.57904077
Query time milliseconds [53143, 56298, 58081, 62260, 61433, 61427,
63434,
59773, 56407, 54072]
p-solr-8-18.obscured.com:8983/solr/content_shard6_replica_n54/ 62589.2
4958.34174341
Query time milliseconds [59044, 64196, 68406, 63014, 61170, 59517,
63885,
72458, 57918, 56284]
Thank you,
Russ
*Manzama*a MODERN GOVERNANCE company
Russell Bahr
Lead Infrastructure Engineer
USA & CAN Office: +1 (541) 306 3271
USA & CAN Support: +1 (541) 706 9393
UK Office & Support: +44 (0)203 282 1633
AUS Office & Support: +61 (0) 2 8417 2339
543 NW York Drive, Suite 100, Bend, OR 97703
LinkedIn <http://www.linkedin.com/company/manzama> | Twitter
<https://twitter.com/ManzamaInc> | Facebook
<http://www.facebook.com/manzamainc> | YouTube
<https://www.youtube.com/channel/UCBo3QoqewyNoo7HiT_BFuRw>
On Thu, Sep 5, 2019 at 8:50 PM David Smiley <david.w.smi...@gmail.com>
wrote:
I suggest first working with a single machine to see if it responds
substantially slower with the new version. Just find one of yours and
issue it a query that will resolve locally (distrib=false param).
Your
current collection level queries are internally issuing such queries,
and
so with a little bit of sleuthing, looking at logs, you can find a
shard
level query like this. If it's quick then there's some distributed
aspect
to investigate. But you'll probably see the slowness here, and the
problem
is better scoped and easier to diagnose. At this point look at
timings
with debug=timing to see information on each of the components. That
may
give you a strong clue. If it's in the QueryComponent which actually
executes the underlying search then you have some further digging to
do.
Use a profiler like JVisualVM.
~ David Smiley
Apache Lucene/Solr Search Developer
http://www.linkedin.com/in/davidwsmiley
