Hi, Using files as back-ends will always be slower than using a volume, DASD, SAN LUN, or iSCSI target. It's just a matter of the overhead. You're writing to a fake block device that's really a file sitting on a file system on a real block device. In the other cases, you're writing directly to a block device. If you don't have SAN storage, I would recommend one of the following configurations:
1. ZFS volumes on internal disks in the primary domain. 2. ZFS volumes as back-ends for iSCSI disks on another server with lots of storage. If you're going to use NAS storage, ZFS can be used as the back-end with sparse files. I would configure NFSv4, dedicated VLAN or LAN for NAS traffic, and make use of IPMP or Link Aggr for performance. Also, make sure to tune your NFS server to scale up, look here: http://www.solarisinternals.com/wiki/index.php/Application_Specific_Tuning#NFS_Server I hope that helps folks out there. *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-* Octave J. Orgeron Solaris Virtualization Architect and Consultant Web: http://unixconsole.blogspot.com E-Mail: [email protected] *-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-*-* ----- Original Message ---- From: "Miller, Vincent (Rick)" <[email protected]> To: [email protected] Sent: Thu, April 8, 2010 9:30:20 AM Subject: Re: [ldoms-discuss] T5120 Ldom Network Performance My apologies. The vdisk was a zfs file. I started setting a domain up with just a zfs zvol as the backend store not long ago with the intent of testing and comparing the performance between a zvol and file. As you've already stated, I expect performance to be better with the zvol. > -----Original Message----- > From: Joe Balenzano [mailto:[email protected]] > Sent: Thursday, April 08, 2010 10:21 AM > To: Miller, Vincent (Rick) > Cc: [email protected] > Subject: Re: [ldoms-discuss] T5120 Ldom Network Performance > > I am a bit confused. Is your vdidsk a ZFS file ? Before you > assign a physical LUN to the LDom, try using a ZFS zvol as > the backend, the performance may be good enough. With that > said, a physical LUN exported to the LDom will give you > better performance. > > Miller, Vincent (Rick) wrote: > > I thought I would provide some more information related to > the below > > email... > > > > After some exhaustive tests, I have identified the problem was not > > really network performance. It was moreso disk overhead related to > > the use of a zfs backend store. > > > > As I understand it, the process of copying the file > involved reading > > from network in primary. The guest domain then reads the data from > > its vnet and writes to its filesystem. This, in turn, > writes to the > > guest's vdisk. This write to the vdisk is then written to > the primary > > domain as another copy operation. All of this occurrs prior to the > > physical write to the disk happens. The issue is > compounded when the > > backend store is a file, as it is with zfs, instead of a device. > > > > In conclusion, I suppose that using a zfs backend store > with LDoms is > > simply not an optimal solution. I will likely reconfigure > the system > > to eliminate the zfs backend store in favor of a hardware RAID. > > > > > >> -----Original Message----- > >> From: Miller, Vincent (Rick) > >> Sent: Thursday, March 25, 2010 2:27 PM > >> To: '[email protected]' > >> Subject: T5120 Ldom Network Performance > >> > >> Hello all, > >> > >> I am hoping that I can gain an understanding based on some > things I > >> have been trying to do. > >> > >> I have a T5120 w/ Solaris 10 10/09 with the latest > available firmware > >> updates applied. Before installing and enabling LDoms 1.3, I get > >> respectable transfer rates over the e1000g interfaces (about 2 > >> minutes to transfer a 4GB file) over Gig-E. However, after > >> installing LDoms and creating the services within the > control domain, > >> then creating a guest LDom, I attempt to transfer the same > 4GB file > >> and now it takes 8 - 10 mins to transfer. I have a pretty typical > >> configuration, at least compared to the Sun/Oracle docs for the > >> software. > >> > >> The OS indicates, through dladm, that the link is up at > 1000fdx. The > >> switch on the other end of the link indicates the same. However, > >> upon visual inspection, the e1000g0 link on the rear of > the server is > >> orange, when I expect green. > >> Seems to indicate a less than optimal link. > >> > >> In reviewing the release notes for LDoms 1.3, I found a > known issue > >> identified as Bug ID 6486234. That seems to indicate that network > >> performance on T2 systems is considerably worse than on > systems where > >> LDoms > >> 1.3 is not configured. In fact, this bug seems to be > referenced in > >> the release notes for several releases. The release notes > go on to > >> explain that a workaround is to "assign a Network Interface Unit > >> (NIU) to the logical domain". > >> > >> Ok. Great...I'll try that. Except the instructions I found to > >> accomplish this task seem to indicate that the on-board e1000g > >> interfaces are not NIU capable interfaces, thus requiring the > >> addition of a PCI network card. I gleaned this information from > >> http://blogs.sun.com/raghuram/entry/niu_hybrid_i_o. which states, > >> specirfically: > >> > >> "NIU has support for 2 ports. Both T5120 and T5220 have two slots > >> for NIU, that is one slot for each port. The XAUI adapters > need to be > >> installed in order gain access to the NIU ports." > >> > >> Additionally, if I configure the e1000g interface to support the > >> hybrid mode anyway, it simply does not have an impact on > performance. > >> > >> I guess what I am asking is if my assumptions are correct. > >> Is it generally accepted that network performance within a LDom > >> system will be poor? To correct this, is it the case that I would > >> need to install a NIU-capable NIC? > >> > >> -- > >> Take care > >> Rick Miller > >> > >> > >> > --------------------------------------------------------------------- > >> --- > >> > >> _______________________________________________ > >> ldoms-discuss mailing list > >> [email protected] > >> http://mail.opensolaris.org/mailman/listinfo/ldoms-discuss > >> > > > -- > ===================================================== > Joe Balenzano > ISV Engineering [email protected] > Oracle Phone +1 203 462 9548 > Skype: jbalenzano > > _______________________________________________ ldoms-discuss mailing list [email protected] http://mail.opensolaris.org/mailman/listinfo/ldoms-discuss
