On Fri, 16 Mar 2001, Geoff Hutchison wrote:
> Date: Fri, 16 Mar 2001 18:47:45 -0500 (EST)
> From: Geoff Hutchison <[EMAIL PROTECTED]>
> To: "Joe R. Jah" <[EMAIL PROTECTED]>
> Cc: [EMAIL PROTECTED]
> Subject: Re: [htdig-dev] Re: Configure changes (Regex and Shared-Libs)
...
> On Fri, 16 Mar 2001, Joe R. Jah wrote:
...
> > tonight or early tomorrow morning. That means 3.2.0b4 has slowed down by
> > a factor of 5 from 3.2.0b3;(~5 hours vs. ~25 hours.)
>
> Yes, and I'd guess it's almost all due to the HtRegexList. It's very
> inefficient right now and needs some cleanup. But it's hard for me to do
> that when there still seems to be a bug too. :-(
I reniced the process to -20, the maximum priority, but it didn't make any
discernable difference.
________________________________Top_____________________________________
load averages: 1.16, 1.13, 1.16 16:05:15
59 processes: 2 running, 57 sleeping
CPU states: 99.6% user, 0.0% nice, 0.4% system, 0.0% interrupt, 0.0% idle
Memory: Real: 48M/81M Virt: 76M/483M Free: 133M
PID USERNAME PRI NICE SIZE RES STATE TIME WCPU CPU COMMAND
27385 jjah 37 -20 37M 38M run/0 16.9H 96.34% 96.34% htdig
7535 jjah 28 0 252K 492K run/0 0:00 0.00% 0.00% top
________________________________End_____________________________________
What does htdig process do, when it does not change anything in the
database or any other part of the file system that I can observe. For
example, I observe changes to the time stamp and or size of the database
only a fraction of the dig time. In about 90% of the dig time I do not
observe any disk write. What is the bottle neck? What is it doing? Is
it reading all that time? The entire dig is done on the same server that
htdig runs. Working in memory shouldn't take long; does it?
_____________________________DB______________________________________
ll ../DB/|grep db|grep -v work;ll ../DB/|grep work;date
-rw-r--r-- 1 jjah www 1441792 Mar 6 16:33 db.docdb
-rw-r--r-- 1 jjah www 655360 Mar 6 16:33 db.docs.index
-rw-r--r-- 1 jjah www 14344192 Mar 6 16:33 db.excerpts
-rw-r--r-- 1 jjah www 17839104 Mar 6 16:38 db.words.db
-rw-r--r-- 1 jjah www 16384 Mar 6 16:38 db.words.db_weakcmpr
-rw-r--r-- 1 jjah www 720896 Mar 16 15:41 db.docdb.work
-rw-r--r-- 1 jjah www 335872 Mar 16 13:29 db.docs.index.work
-rw-r--r-- 1 jjah www 7299072 Mar 16 15:41 db.excerpts.work
-rw-r--r-- 1 jjah www 11006976 Mar 16 15:41 db.words.db.work
-rw-r--r-- 1 jjah www 16384 Mar 15 22:05 db.words.db.work_weakcmpr
Fri Mar 16 16:05:57 PST 2001
$ ps
PID TT STAT TIME COMMAND
6633 p0 Ss 0:00.37 -ksh (ksh)
7521 p0 R+ 0:00.01 ps
27383 p0- I 0:00.02 /bin/sh ./rundig.sh (rundig.sh)
27385 p0- D< 1010:22.03 /Search/Bin/htdig -s -a -c /Search/Conf/htdig.conf
_____________________________END_____________________________________
In the above example from 15:41 to 16:06, 25 minutes, nothing changed in
the database, but htdig had been running;-/ There were times when it ran
close to one hour without any change in the database. Is this behavior
reflective of my bad configuration and/or system setup?
Any clues to clarify this puzzle will be greatly appreciated.
Regards,
Joe
--
_/ _/_/_/ _/ ____________ __o
_/ _/ _/ _/ ______________ _-\<,_
_/ _/ _/_/_/ _/ _/ ......(_)/ (_)
_/_/ oe _/ _/. _/_/ ah [EMAIL PROTECTED]
_______________________________________________
htdig-dev mailing list
[EMAIL PROTECTED]
http://lists.sourceforge.net/lists/listinfo/htdig-dev
