On 12/11/2013 10:27 AM, Jan Lambertz wrote:
I found dd to be a very bad/misleading tool for this case.
Problems are caches in different layers of the system, filesystem
behaviour, sector sizing of drives and arrays, kernel configurations, input
data loading, real world scenarios and driver implementation.
I had same issues on centos.
Not perfect but a lot better for my purpose is bonnie++. Even with bonnie++
i would not dare to say that same tests on same hardware with centos and
openbsd will show the real differences in performance.
Maybe that might help to get more comparable results
I installed new batteries for the RAM cache on my hardware RAID
controller (HP Smart Array 6404) yesterday and, following Jan's
recommendation, I ran some file-system performance tests with bonnie++.
Before getting to the bonnie++ test results, for comparison to earlier
emails in this thread, this is how dd performs with the RAID
controller's write-cache enabled:
## OpenBSD 5.4 GENERIC.MP amd64 (two-disk RAID0)
hanzer:/tmp:34$ dd if=/dev/zero of=test bs=1k count=524288
536870912 bytes transferred in 5.922 secs (90647246 bytes/sec)
## FreeBSD 9.2-RELEASE amd64 (two-disk RAID0)
hanzer@colossus:/tmp % dd if=/dev/zero of=test bs=1k count=524288
536870912 bytes transferred in 3.765968 secs (142558549 bytes/sec)
The OpenBSD performance actually decreased, significantly.
I ran the default bonnie++ test suite on a single disk (no RAID) then
again on a two-disk RAID0 for each, OpenBSD and FreeBSD. I ran these
test for various partition sizes. 32G is presented here for simplicity.
It's a bit cluttered; hopefully, it doesn't get mangled in transport.
-------------------------------------------------------------------
### No RAID (single disk, 32GB partition)
## OpenBSD 5.4 GENERIC.MP amd64
------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
16G 281 100 50412 24 8966 5 331 99 9020 3 162.1 26
Latency 30337us 1384ms 179ms 38470us 34736us 222ms
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 2349 17 +++++ +++ 5068 17 2640 15 +++++ +++ 5044 19
Latency 15962us 169us 360us 8410us 190us 396us
## FreeBSD 9.2-RELEASE amd64
------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
16G 326 99 70558 18 11927 10 627 99 45458 5 339.3 10
Latency 32462us 894ms 3812ms 18257us 238ms 293ms
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 28296 69 +++++ +++ +++++ +++ 28454 82 +++++ +++ +++++ +++
Latency 105ms 149us 177us 71759us 167us 190us
-------------------------------------------------------------------
### Two-disk RAID0 (32GB partition)
## OpenBSD 5.4 GENERIC.MP amd64
------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
16G 285 99 88245 40 9013 4 360 99 11377 4 185.9 29
Latency 37473us 460ms 157ms 34602us 29774us 214ms
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 2465 15 +++++ +++ 5106 19 2663 15 +++++ +++ 4957 20
Latency 18823us 190us 360us 8056us 190us 386us
## FreeBSD 9.2-RELEASE amd64
------Sequential Output------ --Sequential Input- --Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP
16G 320 99 121412 31 13937 11 625 99 61133 7 486.9 15
Latency 32956us 341ms 2104ms 18049us 399ms 266ms
------Sequential Create------ --------Random Create--------
-Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP
16 32225 85 +++++ +++ +++++ +++ 24767 87 +++++ +++ +++++ +++
Latency 71852us 193us 195us 72355us 187us 232us
-------------------------------------------------------------------
I am *very* concerned that something might be malformed in OpenBSD or
that there is a configuration mismatch between my hardware and the
default OpenBSD file-system. If the later is the case, hopefully all
that is required is some tuning. Any advice on how to proceed with
tuning or further experimentation is *very* welcome!
If this performance difference is simply due to OpenBSD's architecture
and implementation methods - if it's a well engineered file-system - and
maximum performance was a lower priority goal than robustness and
reliability, then the lower performance isn't a big deal. However, if my
system is poorly tuned and if there is a mismatch between the software
and the hardware, that is something that needs serious consideration.
Please don't hesitate with any advice, recommendations, quandaries or
queries.
