On 16/01/2013 2:48 AM, Dieter BSD wrote:
Karim writes:
It is quite obvious that something is awfully slow on SAS drives,
whatever it is and regardless of OS comparison. We swapped the SAS
drives for SATA and we're seeing much higher speeds. Basically on par
with what we were expecting (roughly 300 to 400 times faster then what
we see with SAS...).
Major clue there! According to wikipedia: "Most SAS drives provide
tagged command queuing, while most newer SATA drives provide native
command queuing" [1]
Note that the driver says "Command Queueing enabled" without
specifying which. If the driver is trying to use SATA's NCQ but
the drive only speaks SCSI's TCQ, that could explain it. Or if
the TCQ isn't working for some other reason.
See if there are any error messages in dmesg or /var/log.
If not, perhaps the driver has extra debugging you could turn on.
Get TCQ working and make sure your partitions are aligned on
4 KiB boundaries (in case the drive actually has 4 KiB sectors),
and you should get the expected performance.
[1] http://en.wikipedia.org/wiki/Serial_attached_SCSI
Thanks for the wiki article reference it is very interesting and
confirms our current setup. I'm mostly thinking about this line:
"SAS controllers may connect to SATA devices, either directly connected
using native SATA protocol or through SAS expanders using SATA Tunneled
Protocol (STP)."
The systems is currently put in place using SATA instead of SAS although
its using the same interface and backplane connectors and the drives
(SATA) show as da0 in BSD _but_ with the SATA drive we get *much* better
performances. I am thinking that something fancy in that SAS drive is
not being handled correctly by the FreeBSD driver. I am planning to
revisit the SAS drive issue at a later point (sometimes next week).
Here is some trimmed and hopefully relevant information (from dmesg):
SAS drive:
mpt0: <LSILogic SAS/SATA Adapter> port 0x1000-0x10ff mem
0x99910000-0x99913fff,0x99900000-0x9990ffff irq 28 at device 0.0 on pci11
mpt0: MPI Version=1.5.20.0
mpt0: Capabilities: ( RAID-0 RAID-1E RAID-1 )
mpt0: 0 Active Volumes (2 Max)
mpt0: 0 Hidden Drive Members (14 Max)
...
da0 at mpt0 bus 0 scbus0 target 1 lun 0
da0: <IBM-ESXS HUC106030CSS60 D3A6> Fixed Direct Access SCSI-6 device
da0: 300.000MB/s transfers
da0: Command Queueing enabled
da0: 286102MB (585937500 512 byte sectors: 255H 63S/T 36472C)
...
GEOM: da0: the primary GPT table is corrupt or invalid.
GEOM: da0: using the secondary instead -- recovery strongly advised.
SATA drive:
mpt0: <LSILogic SAS/SATA Adapter> port 0x1000-0x10ff mem
0x9b910000-0x9b913fff,0x9b900000-0x9b90ffff irq 28 at device 0.0 on pci11
mpt0: MPI Version=1.5.20.0
mpt0: Capabilities: ( RAID-0 RAID-1E RAID-1 )
mpt0: 0 Active Volumes (2 Max)
mpt0: 0 Hidden Drive Members (14 Max)
...
da0 at mpt0 bus 0 scbus0 target 2 lun 0
da0: <ATA ST91000640NS SN03> Fixed Direct Access SCSI-5 device
da0: 300.000MB/s transfers
da0: Command Queueing enabled
da0: 953869MB (1953525168 512 byte sectors: 255H 63S/T 121601C)
...
GEOM: da0s1: geometry does not match label (16h,63s != 255h,63s).
Please let me know if there is anything you would like me to run on the
BSD 9.1 system to help diagnose this issue?
Thank you,
Karim.
_______________________________________________
freebsd-hackers@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "freebsd-hackers-unsubscr...@freebsd.org"
