On Sun, Jul 4, 2010 at 2:08 PM, Ian D <rewar...@hotmail.com> wrote:

> Mem:  74098512k total, 73910728k used,   187784k free,    96948k buffers
> Swap:  2104488k total,      208k used,  2104280k free, 63210472k cached
>
>   PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
> 17652 mysql     20   0 3553m 3.1g 5472 S   38  4.4 247:51.80 mysqld
> 16301 mysql     20   0 4275m 3.3g 5980 S    4  4.7   5468:33 mysqld
> 16006 mysql     20   0 4434m 3.3g 5888 S    3  4.6   5034:06 mysqld
> 12822 root      15  -5     0    0    0 S    2  0.0  22:00.50 scsi_wq_39

Is that 38% of one CPU or 38% of all CPU's?  How many CPU's does the
Linux box have?  I don't mean the number of sockets, I mean number of
sockets * number of cores * number of threads per core.  My
recollection of top is that the CPU percentage is:

(pcpu_t2 - pcpu_t1) / (interval * ncpus)

Where pcpu_t* is the process CPU time at a particular time.  If you
have a two socket quad core box with hyperthreading enabled, that is 2
* 4 * 2 = 16 CPU's.  38% of 16 CPU's can be roughly 6 CPU's running as
fast as they can (and 10 of them idle) or 16 CPU's each running at
about 38%.  In the "I don't have a CPU bottleneck" argument, there is
a big difference.

If PID 16301 has a single thread that is doing significant work, on
the hypothetical 16 CPU box this means that it is spending about 2/3
of the time on CPU.  If the workload does:

while ( 1 ) {
    issue I/O request
    get response
    do cpu-intensive work work
}

It is only trying to do I/O 1/3 of the time.  Further, it has put a
single high latency operation between its bursts of CPU activity.

One other area of investigation that I didn't mention before: Your
stats imply that the Linux box is getting data 32 KB at a time.  How
does 32 KB compare to the database block size?  How does 32 KB compare
to the block size on the relevant zfs filesystem or zvol?  Are blocks
aligned at the various layers?

http://blogs.sun.com/dlutz/entry/partition_alignment_guidelines_for_unified

-- 
Mike Gerdts
http://mgerdts.blogspot.com/
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

Reply via email to