No, that is definitely not expected.
One thing that can hose you is having a single disk that performs
really badly. I've seen disks as slow as 5 MB/sec due to vibration,
bad sectors, etc. To see if you have such a disk, try my diskqual.sh
script (below). On my desktop system, which has 8 drives, I get:
# ./diskqual.sh
c1t0d0 65 MB/sec
c1t1d0 63 MB/sec
c2t0d0 59 MB/sec
c2t1d0 63 MB/sec
c3t0d0 60 MB/sec
c3t1d0 57 MB/sec
c4t0d0 61 MB/sec
c4t1d0 61 MB/sec
The diskqual test is non-destructive (it only does reads), but to
get valid numbers you should run it on an otherwise idle system.
----------------------
#!/bin/ksh
disks=`format </dev/null | grep c.t.d | nawk '{print $2}'`
getspeed1()
{
ptime dd if=/dev/rdsk/${1}s0 of=/dev/null bs=64k count=1024 2>&1 |
nawk '$1 == "real" { printf("%.0f\n", 67.108864 / $2) }'
}
getspeed()
{
for iter in 1 2 3
do
getspeed1 $1
done | sort -n | tail -2 | head -1
}
for disk in $disks
do
echo $disk `getspeed $disk` MB/sec
done
----------------------
Jeff
On Tue, Apr 08, 2008 at 06:44:13AM -0700, Henrik Hjort wrote:
> Hi!
>
> I just want to check with the community to see if this is normal.
>
> I have used a X4500 with 500Gb disks and I'm not impressed by the copy
> performance.
> I can run several jobs in parallel and get close to 400mb/s but I need better
> performance
> from a single copy. I have tried to be "EVIL" as well but without success.
>
> Tests done with:
> Solaris 10 U4
> Solaris 10 U5 (B10)
> Nevada B86
>
> *Setup*
>
> # zpool status
> pool: datapool
> state: ONLINE
> scrub: none requested
> config:
>
> NAME STATE READ WRITE CKSUM
> datapool ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c0t0d0 ONLINE 0 0 0
> c1t0d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c4t0d0 ONLINE 0 0 0
> c6t0d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c7t0d0 ONLINE 0 0 0
> c0t1d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c1t1d0 ONLINE 0 0 0
> c4t1d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c5t1d0 ONLINE 0 0 0
> c6t1d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c7t1d0 ONLINE 0 0 0
> c0t2d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c1t2d0 ONLINE 0 0 0
> c4t2d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c5t2d0 ONLINE 0 0 0
> c6t2d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c7t2d0 ONLINE 0 0 0
> c0t3d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c1t3d0 ONLINE 0 0 0
> c4t3d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c5t3d0 ONLINE 0 0 0
> c6t3d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c7t3d0 ONLINE 0 0 0
> c0t4d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c1t4d0 ONLINE 0 0 0
> c4t4d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c6t4d0 ONLINE 0 0 0
> c7t4d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c0t5d0 ONLINE 0 0 0
> c1t5d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c4t5d0 ONLINE 0 0 0
> c5t5d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c6t5d0 ONLINE 0 0 0
> c7t5d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c0t6d0 ONLINE 0 0 0
> c1t6d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c4t6d0 ONLINE 0 0 0
> c5t6d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c6t6d0 ONLINE 0 0 0
> c7t6d0 ONLINE 0 0 0
> mirror ONLINE 0 0 0
> c0t7d0 ONLINE 0 0 0
> c1t7d0 ONLINE 0 0 0
>
> *Result* - Around 50-60mb/s read
>
> parsing profile for config: copyfiles
> Running
> /tmp/temp165-231.*.*.COM-zfs-readtest-Apr_8_2008-09h_09m_07s/copyfiles/thisrun.f
> FileBench Version 1.2.2
> 5109: 0.005: CopyFiles Version 2.3 personality successfully loaded
> 5109: 0.005: Creating/pre-allocating files and filesets
> 5109: 0.069: Fileset destfiles: 10000 files, avg dir = 20, avg depth = 3.1,
> mbytes=156
> 5109: 3.922: Removed any existing fileset destfiles in 4 seconds
> 5109: 3.952: Creating fileset destfiles...
> 5109: 3.952: Preallocated 0 of 10000 of fileset destfiles in 1 seconds
> 5109: 4.039: Fileset bigfileset: 10000 files, avg dir = 20, avg depth = 3.1,
> mbytes=158
> 5109: 4.071: Removed any existing fileset bigfileset in 1 seconds
> 5109: 4.098: Creating fileset bigfileset...
> 5109: 117.245: Preallocated 10000 of 10000 of fileset bigfileset in 114
> seconds
> 5109: 117.245: waiting for fileset pre-allocation to finish
> 5109: 117.245: Running '/opt/filebench/scripts/fs_flush zfs
> /export/transcoded'
> 'zpool export datapool'
> 'zpool import datapool'
> 5109: 127.338: Change dir to
> /tmp/temp165-231.*.*.COM-zfs-readtest-Apr_8_2008-09h_09m_07s/copyfiles
> 5109: 127.339: Starting 1 filereader instances
> 5287: 128.348: Starting 16 filereaderthread threads
> 5109: 131.358: Running...
> 5109: 134.378: Run took 3 seconds...
> 5109: 134.378: Per-Operation Breakdown
> closefile2 3312ops/s 0.0mb/s 0.0ms/op 3us/op-cpu
> closefile1 3312ops/s 0.0mb/s 0.0ms/op 4us/op-cpu
> writefile2 3312ops/s 52.3mb/s 0.1ms/op 59us/op-cpu
> createfile2 3312ops/s 0.0mb/s 0.2ms/op 100us/op-cpu
> readfile1 3312ops/s 52.3mb/s 0.1ms/op 27us/op-cpu
> openfile1 3312ops/s 0.0mb/s 0.6ms/op 53us/op-cpu
>
> 5109: 134.378:
> IO Summary: 60000 ops 19869.5 ops/s, (3312/3312 r/w) 104.6mb/s, 228us
> cpu/op, 0.4ms latency
> 5109: 134.378: Stats dump to file 'stats.copyfiles.out'
> 5109: 134.378: in statsdump stats.copyfiles.out
> 5109: 134.379: Shutting down processes
>
> Cheers,
> Henrik
>
>
> This message posted from opensolaris.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss@opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
