[htdig] Slow Responses

[Mike Paradis]

I have a database that is as follows in size...

265942016   db.docdb
77686784db.docdb.work
11092992db.docs.index
12362752db.metaphone.db
9239552 db.soundex.db
474692399   db.wordlist
134800983   db.wordlist.work
377980928   db.words.db

Granted, this is probably a good sizes dbase.
The problem is the amount of time it takes for a return on a search. Searches can take
about 10 seconds or more to return and this is with only one user searching.

Are there some tips on how to speed these searches up? While this might or might not 
be a
large dbase to some, it is only a tiny fraction of what I will be indexing.

Any feedback would be most appreciated.

Mike





To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
List archives:  
FAQ:

Re: [htdig] Slow Responses

[christopher . murtagh]

On Mon, 1 Jan 2001, Mike Paradis wrote:
>I have a database that is as follows in size...
>
>265942016  db.docdb
>77686784   db.docdb.work
>11092992   db.docs.index
>12362752   db.metaphone.db
>9239552db.soundex.db  
>474692399  db.wordlist  
>134800983  db.wordlist.work
>377980928  db.words.db
>
>Granted, this is probably a good sizes dbase. The problem is the amount
>of time it takes for a return on a search. Searches can take about 10
>seconds or more to return and this is with only one user searching.
 
 My databases are a bit smaller (after manually thinning out thousands of
pages through my exclude file):

210227200  db.docdb
  6283264  db.docs.index  
 12396544  db.metaphone.db
  9428992  db.soundex.db
322641272  db.wordlist
265526272  db.words.db

 
 and I get results pretty quickly (the speed is really noticeable with a
text only browser - damn images slowing down the net :-). Perhaps your
machine needs more RAM and/or power? I'm running on a G4/400 with 512 MB
of RAM (running YellowDog Linux). Also, if you are using words that are
too frequent to be useful, try putting them in your bad words file. For
example, my search engine is for McGill University, so I put McGill in the
bad words file. Makes searches containing the word McGill much faster.

 Hope this helps.
  
Cheers,
  
Chris

P.S. My first net access account used to be through Newforce (then
'livewire.newforce.ca'). You guys still in Aylmer P.Q.?

-- 

Christopher Murtagh
Webmaster / Web Communications Group
McGill University
Montreal, Quebec
Canada





To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
List archives:  
FAQ:

Re: [htdig] Slow Responses

[Malcolm Austen]

On Mon, 1 Jan 2001, Mike Paradis wrote:

+ Granted, this is probably a good sizes dbase.
+ The problem is the amount of time it takes for a return on a search. Searches can 
+take
+ about 10 seconds or more to return and this is with only one user searching.

If you haven't any specific cause to keep the default, set this option for
htsearch:

backlink_factor:0

which is noted in the FAQ (5.10) as speeding up searches when there are
large numbers of hits (as is likely with a large database).

HNY, Malcolm.
 [EMAIL PROTECTED] http://users.ox.ac.uk/~malcolm/



To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
List archives:  
FAQ: