Re: [aseek-users] Extremely slow search if many urls are returned

[Kir Kolyshkin]

Yes, it's quite possible, please send the mail to [EMAIL PROTECTED] and our sales rep. will contact you soon with the details. Please describe exactly what kind of service do you need; it would be great if it will fall upon one of the categories described at http://www.aspseek.org/order.php

Also, if OS choice is not a question I would suggest you to switch to Linux. FreeBSD threads can be a problem for ASPseek.

Max Lytvyn wrote:

Thank you for your detailed reply.

Could you or any other developer who is familiar enough with Aspseek do
such profiling/tweeking/fixing for some monetary compensation?
I do not think it is too much work, and for us it will be more efficient
than switching to other search engine.

Best Max

-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]] On Behalf Of Kir Kolyshkin
Sent: Friday, November 29, 2002 7:10 AM
To: [EMAIL PROTECTED]
Subject: Re: [aseek-users] Extremely slow search if many urls are
returned

Well, when I have received your mail with proper vmstat output, I
realized that I do not understand the output of FreeBSD's vmstat - it differs a
lot
from Linux's vmstat. So I'm unable to give you any practical advise,
sorry :(

But I can tell a bit of theory.

You look at vmstat (or iostat) and find the system's bottleneck.

For example, if you see much swap happening, it means you need more RAM,
or change memory-related settings of programs which eats much memory. One
more thing could be too much I/O, that together with low size of system I/O caches means the same - you experience RAM shortage.

If the CPU spends too much time in 'system' state (that means it runs kernel code), that means something is sum-optimal in kernel, and it probably can be fixed by some tuning of kernel settings (in FreeBSD some
of them are compile-type defaults, while others can be tweaked with
'sysctl').

As MySQL is running with ASPseek, you should look at its run-time
behavior, too. Probably by running 'top' and see how much CPU cycles are being
spent for searchd and for mysqld. If 'mysqld' eats much, MySQL needs some
tweaking.

If you have IDE disks, make sure you have turned DMA access mode on (on Linux it is done with 'hdparm' utility).

Also, make yourself acquainted with 'strace' (called 'truss' on some systems) and 'ltrace' tools. They mostly used for "what's happening?" analysys, and sometimes you can even visually see the reason of
slowdown.

You can use 'gprof' tool to get a profile of running program (you should

recompile it with special 'profiling' flags). See 'man gprof' and 'info gprof' for details.

Max Lytvyn wrote:

So is there any hope? I know that you are very busy, but maybe some
hints?
Can it be operating system related? Or some compiler glitch? How to
see what part of code takes all the processor time?

Best Max


-----Original Message-----
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED]] On Behalf Of Kir

Kolyshkin

Sent: Monday, November 25, 2002 10:42 PM
To: [EMAIL PROTECTED]
Subject: Re: [aseek-users] Extremely slow search if many urls are
returned

Oops, sorry, probably I drank too much coffee yesterday.
'vmstat 1 1' is not enough, smth like 'vmstat 1 10' is needed.

Max Lytvyn wrote:

Here is the 'vmstat 1 1' output:

procs      memory      page                    disks     faults

cpu

r b w     avm    fre  flt  re  pi  po  fr  sr ad0 md0   in   sy  cs us
sy id
1 7 0  415740 232820  335   1   1   1 323 195   0   0  341 2120 153 26
9 65

It was taken while executing a one word ('word') query with 38k

returned

ulrs (query took 6 seconds). Some apache requests might have been
processed in parallel.

--
== kir_at_asplinux.ru == 7551596_at_ICQ == 6722750_at_sms.beemail.ru ==

Dream like you'll live forever...Love like you've never been hurt...
Work like you don't need the money...and Dance like nobody is watching!
       -- Satchel Paige