Cameron Harr wrote:
Vlad,
Thanks for the suggestion. As I look via vmstat, my CSw/s rate is fairly
constant around 280K when scst_threads=1 (per Vu's suggestion) and pops
up to ~330-340K CSw/s when scst_threads is set to 8. I'm currently doing
512B writes, and this gives me about a 4:1 ratio of context switches to
IOPs with 1 SCST thread (70K IOPs) and around 4.5:1 when there are 8
SCST threads (75K IOPs).
This is still too high. Considering that each CS is about 1 microsecond
you can estimate how many IOPS's it costs you.
You say those numbers could be overkill - do
you know of a way to drop the number?
Sorry, I don't. I can only say that too many CSs problem is in SRPT
driver. With qla2x00t driver and BLOCKIO backstorage you will have 1
CS/sec or less in average.
I'm very interested in trying Vu's
other suggestions (multiple initiators and multiple QPs, but my other
initiator has been too busy all weekend to run on.
Debug, tracing, and all that was turned off in the SCST Makefiles.
-Cameron
Vladislav Bolkhovitin wrote:
Cameron Harr wrote:
I was able to get the latest scst code working with Vu's standalone
ib_srpt and the kernel IB modules, and dropped my ib_srpt thread
count to 2. However, I still get about the same IOP performance on
the target although interrupts on the "busy" cpu have gone up to
around 140K. Interesting, but now I'm at a bit of a loss as to where
the bottleneck could be. I figured it was Interrupts, but if the CPU
is handling more right now, perhaps the problem is elsewhere?
How many context switches per second do you have during your test on
the target?
Once in scst-devel mailing list there was a thread about observation
that SRP target driver produces 10 context switches per command. See
http://sourceforge.net/mailarchive/message.php?msg_id=e2e108260802070110q1fa084a1j54945d06c16c94f2%40mail.gmail.com
If it is so in your case as well, it would very well explain your
issue. 10 CS/cmd is a definite overkill, it should be 1 or, at max, 2
CS/cmd.
BTW, I suppose you don't use the debug SCST build, do you?
Vlad
_______________________________________________
general mailing list
[email protected]
http://lists.openfabrics.org/cgi-bin/mailman/listinfo/general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
