[
https://issues.apache.org/jira/browse/SOLR-3094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Erick Erickson updated SOLR-3094:
---------------------------------
Attachment: SOLR-3094.patch
OK, anyone with good javascript skills, this would be a good time to chime in...
This is a variant of SOLR-1931. The new UI calls Luke at the top level in such
a way that it enumerates all the terms in all the fields to gather the
histogram data, which takes a long time. Note, this is what the old admin
UI/Luke handler did when you clicked "schema browser" link.
Once that data is accumulated, then clicking on the individual fields and
showing that data is very fast since the data is local. But this data is
accumulated *before* any field is selected from the "schema browser" drop-down
and stored away.
I think this design is too costly, especially the "get all the data for all the
fields up-front" bit. The users pay a penalty (many minutes demonstrated) even
when they may only care about one field. So here's what I propose.
1> Tweak the LukeRequestHandler so it *requires* the fieldName parameter to
gather the historgram data. That fixes the initial display of the stats issue
that sparked this JIRA. I can do that in a few minutes, patch attached (do not
commit yet, though). Problem is there is then no way at all to get the stats
data.
2> Tweak the javascript to call the luke request handler to collect the data
for individual fields only when the user selects them from the drop-down,
stowing them away at that point so they can be revisited if desired. Here's
where I could use some help, my javascript skills are rudimentary at best. If
anyone could work the javascript I'd be happy to field test. Or even just put
some comments in the code pointing me to them. Any trunk code from after 6-Jan
will have the right Luke handler in it (see SOLR-1931).
There's also something wrong with the display of the histogram, the "bucket"
and count in each bucket are mashed together on the bottom. With non-trivial
indexes, this is largely unreadable since they're side-by-side...
Anyway, the attached patch makes it so you can get into the admin page without
paying the above penalties, but you *never* get histogram data when you go into
"schema browser". If someone applies this to work on the admin UI bit,
attaching "&fl=field1 field2" to the luke URL will cause the histogram data to
be returned for the field(s) specified.
If anyone has some spare cycles to help out here it would be outstanding.
I think something similar could be done for the old admin UI as well in terms
of only getting the fields when requested, otherwise the histogram data won't
be returned either...
> The statistics entry on the new admin UI is very slow
> -----------------------------------------------------
>
> Key: SOLR-3094
> URL: https://issues.apache.org/jira/browse/SOLR-3094
> Project: Solr
> Issue Type: Bug
> Components: Schema and Analysis
> Affects Versions: 4.0
> Environment: trunk only, all environments
> Reporter: Erick Erickson
> Assignee: Erick Erickson
> Attachments: SOLR-3094.patch
>
>
> Prompted by Robert Reynolds (SOLR-2667), the entry point in the new Admin UI
> core drill down (e.g. clicking "singlecore" takes a long time. 28-46
> *minutes* on a 13M-23M doc set.
> On an example Wikipedia index (11M) docs, I see 21 seconds, compared to less
> than 2 seconds in the old admin UI (I'm using the old admin UI linked to from
> the new UI page on trunk). I have a very simple index layout compared to a
> commercial site. Clearly something is not right. I suspect that all the terms
> are being walked.
> This is particularly an issue because this behavior happens when I click
> "singlecore", so getting to the really neat parts of the new UI is hard.
> Robert reports on a separate thread that the same behavior happens just
> hitting admin/luke in the URL which is also slow in the 3.x world, which
> hints at where the problem lies.
> I'm going to guess that the terms are being walked and we can use the tricks
> used in SOLR-1931 to deal with the fact that admin/luke takes a long time,
> and just change the call to the entry ("singlecore") for this issue.
> Robert: Thanks for pointing this out!
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
