On 3/2/2012 5:30 AM, Alex Netes wrote: > On 11:22 Wed 29 Feb , Ira Weiny wrote: >> Doug, >> >> First thanks for this. Some comments below. >> >> On Wed, 29 Feb 2012 00:01:16 -0500 >> Doug Ledford <dledf...@redhat.com> wrote: >> >>> There are two things that stand in the way of opensm being run on >>> redundant fabrics easily: >>> >>> 1) The opensm init script only starts one instance of opensm and opensm >>> will only work on one fabric per instance >>> 2) Even if you start multiple instances, you have to hand modify config >>> files for each instance and then when you upgrade the opensm rpm you >>> either loose your modifications or loose getting new default settings >>> >>> I worked around both of these issues, I've attached the files I used to >>> do so. >>> >>> First, I have an opensm init script that allows starting multiple opensm >>> instances. It supports configuring this in one of two ways: >>> >>> 1) Create multiple opensm.conf files, each with a numbered suffix (so >>> opensm.conf.1, opensm.conf.2, etc.) and it will start one opensm >>> instance per config file. This allows an admin to copy the default >>> config over and edit the things they need, and on rpm upgrade there will >>> be a new default opensm.conf file so they can diff between their edited >>> version and the new default and see if there are changes they need to >>> bring back in. This also allows for complete flexibility in setting up >>> the different fabrics, for instance you could use one type of routing on >>> one and a totally different type on the others. >>> >>> 2) Edit the file /etc/sysconfig/opensm and define more than one GUID in >>> the GUIDs variable. This will cause the opensm init script to >>> automatically start one instance per GUID, passing the GUID in on the >>> command line. >> >> I know you are going for ease of use here, which is good, however, I worry >> about this file becoming a redefinition of opensm.conf. >> >>> >>> For the most part, this works well. However, openmpi in particular >>> doesn't like you to have physically separate fabrics that have the same >>> subnet_prefix, and you can't specify a subnet_prefix on the command line >>> to go along with the GUIDs. So I wrote a patch for that and made the >>> init script unilaterally increment the subnet prefix for each different >>> GUID it's attaching to. >> >> If you only allow option 1 above this takes care of itself by making the >> admin configure his subnet prefixes in each config file as appropriate. The >> only down side is the loss of new configuration options as you upgrade. >> However, that is probably better taken care of by a default config file with >> each package. I mentioned this to Sasha years back and was denied since >> "you can always generate a new one with '-c'". :-( >> >> Alex would a default config file be acceptable? It would mean more work on >> your part. >> > > What the default opensm.conf would be used for? Just as a reference to the > default values?
No, he's referring to having a default config file that is parsed, then
an override config file that is parsed where you only put options you
want to update in the override config file. That way you could have,
for instance, a default opensm.conf in the normal location and totally
unedited so that it gets updated with each update of the opensm rpm,
then you could create an opensm.conf.1 that is empty except for just a
guid setting, a subnet_prefix setting, maybe a cache dir setting, etc.
In that way, if say the default routing engine gets a new option in the
future, your override config file won't already be populated with the
old stuff. It's a means of inheritance that is functionally identical
to specifying all this stuff on the command line, but doesn't require a
huge command line or a complex init script.
--
Doug Ledford <dledf...@redhat.com>
GPG KeyID: 0E572FDD
http://people.redhat.com/dledford
Infiniband specific RPMs available at
http://people.redhat.com/dledford/Infiniband
signature.asc
Description: OpenPGP digital signature
