Excellent news; as you guessed, my schema was (for some reason) set to
version 1.0. This also caused some of the problems I had with the
original SolrPHP (parsing the wrong response).
But better yet, the 800 seconds query is now running in 0.5-2 seconds!
Amazing optimization! I can now do faceting on journal title (17 000
different titles) and last author (>400 000 authors), + 12 date range
queries, in a very reasonable time (considering im on a test windows
desktop box and not a server).
The only problem is if I add first author, I get a
java.lang.OutOfMemoryError: Java heap space. I'm sure this problem will
get away on a server with more than the current 500 megs I can allocate
to Tomcat.
Michael Imbeault
CHUL Research Center (CHUQ)
2705 boul. Laurier
Ste-Foy, QC, Canada, G1V 4G2
Tel: (418) 654-2705, Fax: (418) 654-2212
Yonik Seeley wrote:
On 9/22/06, Michael Imbeault <[EMAIL PROTECTED]> wrote:
I upgraded to the most recent Solr build (9-22) and sadly it's still
really slow. 800 seconds query with a single facet on first_author, 15
millions documents total, the query return 180. Maybe i'm doing
something wrong? Also, this is on my personal desktop; not on a server.
Still, I'm getting 0.1 seconds queries without facets, so I don't think
thats the cause. In the admin panel i can still see the filtercache
doing millions of lookups (and tons of evictions once it hits the
maxsize).
The fact that you see all the filtercache usage means that the
optimization didn't kick in for some reason.
Here's the field i'm using in schema.xml :
<field name ="first_author" type="string" indexed="true" stored="true"/>
That looks fine...
This is the query :
q="hiv red
blood"&start=0&rows=20&fl=article_title+authors+journal_iso+pubdate+pmid+score&qt=standard&facet=true&facet.field=first_author&facet.limit=5&facet.missing=false&facet.zeros=false
That looks OK too.
I assume that you didn't change the fieldtype definition for "string",
and that the schema has version="1.1"? Before 1.1, all fields were
assumed to be multiValued (there was no checking or enforcement).
-Yonik
