On Fri, Jan 8, 2010 at 11:39 PM, Chris Worley <worl...@gmail.com> wrote: > On Fri, Jan 8, 2010 at 3:17 PM, David Dillow <d...@thedillows.org> wrote: >> On Fri, 2010-01-08 at 14:40 -0700, Chris Worley wrote: >>> On Wed, Jan 6, 2010 at 6:57 PM, David Dillow <d...@thedillows.org> wrote: >>> > On Wed, 2010-01-06 at 17:16 -0700, Chris Worley wrote: >>> >> 1) I'm seeing small block random writes (32KB and smaller) get better >>> >> performance over SRP than they do as a local drive. I'm guessing this >>> >> is async behavior: once the written data is on the wire, it's deemed >>> >> complete, and setting a sync flag would disable this. Is this >>> >> correct? >> >>> >> If not, any ideas why SRP random writes would be faster than >>> >> the same writes locally? >>> > >>> > I would guess deeper queue depths and more cache available on the >>> > target, especially if you are using a Linux-based SRP target. >>> >>> I do set the ib_srp initiator "srp_sg_tablesize" to its maximum of 58. >> >> The max is 255, which will guarantee you can send up to a 1020 KB I/O >> without breaking it into two SCSI commands. In practice, you're likely >> to be able to send larger requests, as you will often have some >> contiguous runs in the data pages. > > I've tried a larger max... 58 is all I can get. Maybe getting more is > dependent on some other setting.
The SRP spec says that the target must specify the maximum message size in the SRP_LOGIN_RSP information unit. The largest value one can set the srp_sg_tablesize initiator parameter to is (max. SRP message size defined by the target - 68) / 16. With older SCST-SRPT revisions the maximum SRP message size was 996 bytes, hence a maximum of 58 for srp_sg_tablesize. With newer SCST-SRPT revisions the maximum message size defaults to 2116, which corresponds to a maximum of 128 for srp_sg_tablesize. The maximum message size can even be increased further via the module parameter srp_max_message_size of ib_srpt (see also srpt/src/README in the SCST source tree). Bart. -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
