On Wed, 6 Dec 2000, Clint Gilders wrote:
> David Gewirtz wrote:
> >
> > I just love getting to know new software. There's always some form of
> > teething pain. Yesterday, I started running my first set of reasonably
> > large htdig/htmerge processes. Came in today to find the Linux server
> > (which is running nothing besides basic Mandrake processes and, of course,
> > htdig) was deader than a doornail (have to say "deader than" because saying
> > "hung more than" would just be too weird).
>
> I use Mandrake at home and love it, but have nothing but problems with
> it in Server environment. Our lone Linux Server (The rest are free BSD)
> has been crashing daily (hanging, not telnet, no ftp etc) since we
> installed apache/mod_ssl. Even before that it wasn't the most reliable
> box going. If you are going to continue to use it in a production
> environment I suggest not running X or KDE as these can eat up 60% of
> you CPU.
>
> We have indexed well over 200,000 documents with htdig running on a
> single Free BSD machine without as much as hiccup.
>
> >Almost makes me wish for NT.
> Be careful what you wish for! You just might get it. Ahhhhhh!!! The
> horror.
I can say from experience, the only times I've crashed a Linux box has
been due to faulty hardware or faulty admin. There might be times when the
system is so loaded that it might take 2 minutes to login, but login it
eventually does. The few times where even login wouldn't work have been
admin error, things like writing memory bombs accidently or letting
file systems get full.
Now, having also run htdig for quite a while, here are the things that
could cause a box to become overloaded and die:
* running htmerge where TMPDIR points to a file system that is too
small. When sort runs it fills the file system, which is bad. And
people usually run dig as root, which means the file system really gets full.
If this happens to be the / file system, well, things get very ugly
when / is full.
* running htdig against a large number of pages and filling up / .
First, I would verify the hardware. The test of choice is still compiling
the kernel, this really does exercise the system more than anything else
(to really have fun, compile several kernels at once or alter the -j
parameter for make in the Makefile). I had a machine that could not
compile a kernel but otherwise ran fine. Turned out the CPU was
overheating, but only when it was really pushed.
So, compile a kernel or two and then start looking at htdig again.
$.02
Bill Carlson
--
Systems Programmer [EMAIL PROTECTED] | Opinions are mine,
Virtual Hospital http://www.vh.org/ | not my employer's.
University of Iowa Hospitals and Clinics |
------------------------------------
To unsubscribe from the htdig mailing list, send a message to
[EMAIL PROTECTED]
You will receive a message to confirm this.
List archives: <http://www.htdig.org/mail/menu.html>
FAQ: <http://www.htdig.org/FAQ.html>
