Hi,
I have setup AVS replication between two zvols on two opensolaris-2008.11
nodes. I have been seeing BIG performance issues, so I tried to setup the
system to be as fast as possible using a couple of tricks. The detailed
setup and performance data are below:
* A 100G zvol has been setup on each node of an AVS replicating pair
* A "ramdisk" has been setup on each node using the following command.
This functions as a very fast logging disk!
ramdiskadm -a ram1 10m
* The replication relationship has been setup using
sndradm -E pri /dev/zvol/rdsk/gold/myzvol /dev/rramdisk/ram1 sec
/dev/zvol/rdsk/gold/myzvol /dev/rramdisk/ram1 ip async
* The AVS driver was configured to *not* log the disk bitmap to disk,
rather to keep it in kernel memory and write it to disk only upon machine
shutdown. This is configured as such
# grep bitmap_mode /usr/kernel/drv/rdc.conf
rdc_bitmap_mode=2;
* The replication was configured to be in logging mode (To avoid any
possible network bottlenecks)
#sndradm -P
/dev/zvol/rdsk/gold/myzvol <- pri:/dev/zvol/rdsk/gold/myzvol
autosync: off, max q writes: 4096, max q fbas: 16384, async threads:
2, mode: async, state: logging
=== Testing ===
All tests were performed using the following command line
# dd if=/dev/zero of=/dev/zvol/rdsk/gold/xxVolNamexx oflag=dsync
bs=256M count=10
I usually ran a couple of runs initially to avoid caching effects.
=== Results ===
The results
The following Results were reported after initial couple of runs to avoid
cache effects
Run# dd count=N Native Vol Throughput Replicated Vol
Throughput (logging mode)
1 4 42.2 MB/s
4.9 MB/s
2 4 52.8 MB/s
5.5 MB/s
3 10 50.9 MB/s
4.6 MB/s
As you can see the performance is almost 10 times slower!! Although no disk
logging is done to disk (only ram disk if not driver memory), and no network
traffic ! Seems to me that the AVS kernel drivers slow the system A LOT for
simply hooking any write operation and flipping a kernel memory bit per 32k
written disk space?!!
Any suggestions as to why this is happening is most appreciated
Best Regards
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
