Re: Linux and VM SAN advice
On Mon, Dec 21, 2009 at 17:51, Martha McConaghy u...@vm.marist.edu wrote: Subject: Linux and VM SAN advice I've got several test Oracle servers running on SLES 10 and z/VM 5.4.0 on our z9 using SAN disk space. I set up the SAN connection using a FICON adapter and the EDEV support. That is working OK, but I'm concerned about where to go from there. snip Is it worth moving to NPIV before hooking in our production server? I know that it can provide better security for the connections. Does it also help with performance? Another reason to use NPIV now on your z9 is looking ahead to a future move to a z10: On z9 if you don't use EDEV or NPIV to control access to LUNs you'd use IBM's LUN Access Control utility, but when you get to a z10 you'll find z10 doesn't support LUN Access Control utility so you'd do without LUN Access Control or have to migrate the guests to NPIV or EDEV. This e-mail, including any attachments, may be confidential, privileged or otherwise legally protected. It is intended only for the addressee. If you received this e-mail in error or from someone who was not authorized to send it to you, do not disseminate, copy or otherwise use this e-mail or its attachments. Please notify the sender immediately by reply e-mail and delete the e-mail from your system.
Linux and VM SAN advice
I've got several test Oracle servers running on SLES 10 and z/VM 5.4.0 on our z9 using SAN disk space. I set up the SAN connection using a FICON adapter and the EDEV support. That is working OK, but I'm concerned about where to go from there. I have to now move our production Oracle database server over to SAN in the next few months (its using CKD volumes tied together with LVM right now). This database is for the college's ERP system, so performance is going to be critical. I'm waying the options for moving it to the SAN and could use some advice from those of you who are already running SAN in production. We haven't done multipathing yet, but are looking into it now. How many physical and virtual paths do you normally use for a high-visbility server? Would 2 be enough, or should we go for 3 or more? Does multipathing introduce a lot of extra workload for z/VM? Do you normally use dedicated connections or do you share them with other servers? (I'm not sure how you monitor the workload for these devices since they are dedicated devices and VM isn't handling their I/O.) Is it worth moving to NPIV before hooking in our production server? I know that it can provide better security for the connections. Does it also help with performance? Any comments would be helpful. Martha
Re: Linux and VM SAN advice
On Mon, Dec 21, 2009 at 17:50, Martha McConaghy u...@vm.marist.edu wrote: ... I have to now move our production Oracle database server over to SAN in the next few months (its using CKD volumes tied together with LVM right now). You will probably continue to use LVM. I recommend that. One reason to continue concatenating physical volumes with LVM, even in SAN land, is that there is probably a sweet spot LUN size (your SAN people would know, and if you are the SAN people then ask your vendor). The ideal LUN size will probably be smaller than the logical volume size you want for your Oracle filesystems. General rule: use LVM and avoid partitioning * This database is for the college's ERP system, so performance is going to be critical. I'm waying the options for moving it to the SAN and could use some advice from those of you who are already running SAN in production. If performance is critical, consider direct access (that is, ditch EDEV). I highly recommend EDEV for managing content, for letting VM handle (and slice) the LUNs. But there is overhead. (I truly don't know how much.) It may be that you don't even really gain much with direct Linux access to the SAN fabric, but it cuts out one more finger pointing opportunity which at this time may cause pain. This is difficult for me to say because direct Linux access to the fabric can be a sysadmin nightmare. (Imagine hundreds of OSAs and no VSwitch, but bigger MAC addrs.) We haven't done multipathing yet, but are looking into it now. How many ... Umm... DO. (Surprised you're not doing multipathing with EDEV.) ... physical and virtual paths do you normally use for a high-visbility server? We use two paths. A pair of paths is easier to manage than (say) four. But obviously four paths provides more I/O opportunities. Would 2 be enough, or should we go for 3 or more? Does multipathing introduce a lot of extra workload for z/VM? Do you normally use dedicated connections ... I don't have numbers comparing multipath to single path. For us, single path was never an option. ... or do you share them with other servers? (I'm not sure how you monitor the workload for these devices since they are dedicated devices and VM isn't handling their I/O.) Right. The only indication you get is how busy the FCP channel is. :-( Without EDEV, I cannot imagine that multiple FCP paths adds much overhead at the VM level. And what overhead it introduces in Linux seems to be trivial. (theory, not measured) Is it worth moving to NPIV before hooking in our production server? I know that it can provide better security for the connections. Does it also help with performance? NPIV places requirements on your SAN and on the z hardware. Linux and VM are both ready to rock-n-roll with NPIV, no problem. I see no performance problem with it. But why do you need NPIV? We use NPIV to isolate LUNs, letting a subchannel look (to the fabric) like a unique host, where normally only each FCP looks like a host. Being a financial shop, you can imagine this is a big plus with the security and audit people. Any Linux guest can only hit those LUNs which are zoned/masked to its FCP subchannel(s). There is no browsing of LUNs in the fabric. Ironically, one of the benefits of NPIV is that you can share LUNs without having to buy more physical FCP cards. A LUN can be zoned/masked to multiple Linux-side WWPNs, which (with NPIV) might be on the same card. A LUN can thus be simultaneously in-use to several hosts without giving a busy signal like you would get with multiple subchannels and no NPIV. I hope this helps. * I will clarify use LVM and avoid partitioning in a separate note. -- R;
Re: Linux and VM SAN advice
About Sir Santa's General Rule for Disk ... On Mon, Dec 21, 2009 at 22:02, Richard Troth vmcow...@gmail.com wrote: ... General rule: use LVM and avoid partitioning * I have to be careful when I say avoid partitioning. If you use CDL, which is the default low-level formatting for ECKD DASD with 'dasdfmt', YOU MUST use partitioning. In all other cases (FBA, EDEV, LDL ECKD, IDE, SCSI, SATA) partitioning is not needed and may actually be counter productive. Even CD-ROM. Thumb drives too (if you don't need Windoze). So ... if you're NOT on CDL, avoid partitioning. By the way, there is no low-level formatting for FBA, EDEV, IDE, SCSI, SATA, ... or thumb drives ... or RAM. (There is for ECKD, whether CDL or LDL, and there is for CD-ROM.) If you think you need to partition a volume, consider instead feeding it to LVM as a PV and then letting LVM manage the slicing and dicing. Need more room in a partition? Do a quick 'lvextend'. Need more total space? Add more PVs! Partitioning is, and always has been, a crude form of filesystem below the filesystems we all know and love. It actually gets in the way of content sharing. And it doesn't grow past the one physical volume. You know how we share ECKD vols between VM LPARs? You can share SAN LUNs across systems, even different types of hardware and operating systems, if you avoid partitioning. The partition table *might* be recognized by the sharing OS, but it might not, and in any case it's one more thing to manage. Don't! Make life simple. The original question Martha asked had to do with SAN connectivity. Considering partitioning is relevant there. Want to do multipath? Coalesced paths are much more easily handled if you blow off partitioning. Want to copy content? You can use 'dd' most effectively where you're only after PV content or filesystem content. (Not likely that a CDL partition table will be worth much if you 'dd' it over to a SAN LUN.) This is not to say that there are not more opportunities for wonder and amazement. Surely there are myriad more ways we can obfuscate our data. It's just that partitioning is NOT one which I find effective anymore. -- R;
