Hi again Yevgeny, On Thu, 2007-01-25 at 11:53, Yevgeny Kliteynik wrote: > Hi Hal. > > Hal Rosenstock wrote: > > Hi Yevgeny, > > > > On Wed, 2007-01-24 at 09:10, Yevgeny Kliteynik wrote: > >> Hi Hal, Sasha. > >> > >> Here's a description of the QoS policy file, and an > >> example of such file (with more comments inside). > > > > This makes the start of a good document on this. If you add this to > > osm/doc, I will incorporate it into the opensm man page. > > OK, I'll do that. > > >> QoS Policy file > >> -- > >> > >> The QoS policy file is divided into 4 sub sections: > >> > >> * Node Group: a set of HCAs, Routers or Switches that share the same > >> settings. > >> A node groups might be a partition defined by the partition manager > >> policy in > >> terms of GUIDs. > > > > Are these Node or Port Groups ? It looks like port groups from the > > below. > > Good point - it should be "Port Groups". > > >> Future implementations might provide support for NodeDescription > >> based definition of node groups. > >> > >> * Fabric Setup: > >> Defines how the SL2VL and VLArb tables should be setup. This policy > >> definition > >> assumes the computation of target behavior should be performed outside > >> of > >> OpenSM. > >> > >> * QoS-Levels Definition: > >> This section defines the possible sets of parameters for QoS that a > >> client might > >> be mapped to. Each set holds: SL and optionally: Max MTU, Max Rate, Path > >> Bits > >> (in case LMC > 0 is used for QoS) and TClass. > > > > How does this relate to/interact with partition configuration ? Also, > > what about preexisting QoS ? > > As I understand from the osm man or from the partition-config.txt, > partitions definition is intended to be used for IPoIB only. > [quote] > sl=<val> - specifies SL for this IPoIB MC group > (default is 0) > [/quote] > > I think that QoS policy may only "tighten" the constraints and enforce > lower-than-requested values, both in case of partition and in case of > preexisting QoS settings.
I'm not following you on this specific point. A specific SL is chosen by partition config so how can it be "tightened" ? Does it mean it might be changed to a different SL (in which case this QoS config superceeds the partition config for SL setting) ? Have you tried this to be sure ? Are multicast groups handled as part QoS definition in the XML syntax ? If not, might this be a future addition ? If it is, how are they specified ? The other half of the original question was how a QoS request is handled if the original QoS support is enabled rather than this new QoS support in terms of the SA PR and MPR code. > >> * Matching Rules: > >> A list of rules that match an incoming PathRecord request to a > >> QoS-Level. The > >> rules are processed in order such as the first match is applied. Each > >> rule is > >> built out of set of match expressions which should all match for the > >> rule to > >> apply. The matching expressions are defined for the following fields > >> - SRC and DST to lists of node groups > >> - Service-ID to a list of Service-ID or Service-ID ranges > >> - TClass to a list of TClass values or ranges > >> > >> QoS policy file example > >> -- > >> > >> <?xml version="1.0" encoding="ISO-8859-1"?> > >> <qos-policy> > >> <!-- Port Groups define sets of ports to be used later in the settings > >> --> > >> <port-groups> > >> <!-- using port GUIDs --> > >> <port-group> > >> <name>Storage</name> > >> <use>our SRP storage targets</use> > > > > Is the use clause more than commentary ? How is it "used" ? > > The 'use' clause is just a description of the port group that > can be used for logging. Other than for logging, it is just a > commentary. > > >> <port-guid>0x1000000000000001</port-guid> > >> <port-guid>0x1000000000000002</port-guid> > >> </port-group> > >> <!-- using names obtained by concatenation of first 2 words of > >> NodeDescription > >> and port number --> > >> <port-group> > >> <name>Virtual Servers</name> > >> <use>node desc and IB port #</use> > >> <port-name>vs1/HCA-1/P1</port-name> > >> <port-name>vs3/HCA-1/P1</port-name> > >> <port-name>vs3/HCA-2/P1</port-name> > > > > How are port-names used ? > > The syntax of the port name is as follows: > "hostname/CA-num/Pnum" What's it's purpose ? Is it used somewhere else in the syntax ? > >> </port-group> > >> <!-- using partitions defined in the partition policy --> > >> <port-group> > >> <name>Partition 1</name> > >> <use>default settings</use> > >> <partition>Part1</partition> > >> </port-group> > >> <!-- using node types HCA|ROUTER|SWITCH --> > > > > Is this CA rather than HCA ? (What about TCAs ?) > > Sure, it should be 'CA'. Will this be changed ? If so, when ? > >> <port-group> > >> <name>Routers</name> > >> <use>all routers</use> > >> <node-type>ROUTER</node-type> > >> </port-group> > >> </port-groups> > >> > >> <qos-setup> > >> <!-- define all types of SL2VL tables always have 16 VL entries --> > > ^^ > > Actually, it is SL > > assuming the device supports SL2VL mapping as indicate by > > IsSLMappingSupported in the PortInfo:CapabilityMask. > > Will the syntax handle single data VL devices which only implement SL > > filtering ? > > Yes, it should. > > > Will the QoS manager support this (SL2VL without VLArb > > settings) or are these required together ? > > Yes, it should support sl2vl w/o vlarb settings as well. > > >> <sl2vl-tables> > >> <!-- scope defines the exact devices and in/out ports the tables > >> apply to > >> if the same port is matching several rules the last one > >> applies --> > >> <sl2vl-scope> > >> <group>Part1</group> > >> <from>*</from> > >> <to>*</to> > >> > >> <sl2vl-table>0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,7</sl2vl-table> > >> </sl2vl-scope> > >> <!-- also the link across port 1 is probably supporting only 2 > >> VLs --> > >> <sl2vl-scope> > >> <across>Storage</across> > >> <!-- "across-from" means the port just connected to the > >> given group --> > >> <across-from>Storage2</across-from> > >> <!-- "across-to" means the port just connected *to* the > >> given group --> > >> <across-to>Storage3</across-to> > > > > I don't quite follow across-from/to. > > Right, the comments there are garbage. Here the explanation: > SL2VL table describes VL as function of from-port, to-port, and SL. > > <across-to>group_name</across-to>: > It defines sl2vl table where 'to-port's belong to group_name > <across-from>group_name</across-from>: > Same as above, only that this time 'from-port's belong to group_name > <across>group_name</across>: > sl2vl tables both for 'to-port's 'from-port's that belong to group_name I'm still not following what is going on here and how this is used. > >> <from>*</from> > >> <to>1</to> > >> <sl2vl-table>0,1,1,1,1,1,1,1,1,1,1,1,1,1,1,0</sl2vl-table> > >> </sl2vl-scope> > >> </sl2vl-tables> > >> > >> <!-- define all types of VLArb tables. The length of the tables > >> should > >> match the physically supported tables by their target ports > >> --> > >> <vlarb-tables> > >> <!-- scope defines the exact ports the VLArb tables apply to > >> --> > >> <vlarb-scope> > >> <group>Storage</group> > >> <!-- VLArb table holds VL and weight pairs --> > >> > >> <vlarb-high>0:255,1:127,2:63,3:31,4:15,5:7,6:3,7:1</vlarb-high> > >> > >> <vlarb-low>8:255,9:127,10:63,11:31,12:15,13:7,14:3</vlarb-low> > >> <vl-high-limit>10</vl-high-limit> > > > > What happens if the shape of VLArb indicated here does not match the > > device ? > > The part that sets up the QoS in SM (I'm not writing this part right now), What is the plan for this ? > should issue error message in case VLArb definition doesn't match the device > properties. Aside from the error message, is there any additional error handling for this ? > >> </vlarb-scope> > >> </vlarb-tables> > >> </qos-setup> > >> > >> <qos-levels> > >> <!-- the first one is just setting SL --> > >> <qos-level> > >> <sn>1</sn> > > > > What does sn mean ? What is it used for ? > > 'sn' is an id of this qos level definition. > It is referenced later in by QoS match rules as 'qos-level-sn' What is 'sn' short for ? > >> <use>for the lowest priority comm</use> > >> <sl>16</sl> > >> </qos-level> > >> <!-- the second sets SL and TClass --> > >> <qos-level> > >> <sn>2</sn> > >> <use>low latency best bandwidth</use> > >> <sl>0</sl> > >> <class>7</class> > > > > What is class ? I saw TClass mentioned earlier. Is this TClass or > > something else ? > > Instead of "TClass" there should be "QoS Class". > The <class> value is the PathRecord.qos_class value that should be > returned in the path record query response when a certain <qos-level> > is applied to the returned path. So these names need to change to be more consistent ? > >> </qos-level> > >> <!-- the whole set: SL, TClass, MTU-Limit, Rate-Limit, Path-Bits > >> --> > > > > If specified, do MTU limit and rate limit add extra limits to be imposed > > on what is selected (and realizable) ? > > Yes > > > Strictly speaking, couldn't packet lifetime limit also be added to this > > syntax here ? I presume it was left out as being not "interesting" as > > yet. Is that correct ? > > I can add packet lifetime limit - it's not a big deal > > > Also, how are path bits used ? > > For now I don't do anything with them - we'll discuss this issue in the > future. How are they envisioned to be used ? Why are they in the syntax now ? Seems inconsistent with PLL. Should there be a warning if they are specified now since they are not used ? > >> <qos-level> > >> <sn>3</sn> > >> <use>just an example</use> > >> <sl>0</sl> > >> <class>32</class> > >> <mtu-limit>1</mtu-limit> > >> <rate-limit>1</rate-limit> > >> </qos-level> > >> </qos-levels> > >> > >> <!-- Match rules are scanned in a first-fit manner (like firewall > >> rules table) --> > >> <qos-match-rules> > >> <!-- matching by single criteria: class (list of values and > >> ranges) --> > >> <qos-match-rule> > >> <qos-level-sn>1</qos-level-sn> <!-- defined in <sn> of > >> <qos-level> --> > >> <use>low latency by class 7-9 or 11</use> <!-- just a > >> description --> > >> <class>7-9,11</class> <!-- --> > >> <match-level>1</match-level> <!-- ID of this match rule --> > >> </qos-match-rule> > >> <!-- show matching by destination group AND service-ids --> > >> <qos-match-rule> > >> <qos-level-sn>2</qos-level-sn> > >> <use>Storage targets connection></use> > >> <destination>Storage</destination> > >> <service>22,4719</service> > > > > What is service ? What does 22.4719 mean ? > > The syntax is <service>service_id1,service_id1,...</service>, so in the > example above these are actually two service ids. So you can create arbitrary lists of service IDs. What about ranges ? Does the syntax support that ? > As for the exact meaning of this, I'm not sure - I need to think about it... Let me know. I'd really like to understand the syntax. > >> <match-level>3</match-level> > > > > What are match-levels used for ? > > Actually, they are not used - they shouldn't appear here. > Somehow it was copy-pasted here from one of the older versions > of the policy file. So can this be updated for what is current ? Thanks. -- Hal > -- Yevgeny > > > -- Hal > > > >> </qos-match-rule> > >> </qos-match-rules> > >> > >> </qos-policy> > >> > >> > >> > >> -- Yevgeny > >> > >> Yevgeny Kliteynik wrote: > >>> Hi Sasha, > >>> > >>> Sasha Khapyorsky wrote: > >>>> On 10:46 Sun 21 Jan , Yevgeny Kliteynik wrote: > >>>>> Hi Sasha. > >>>>> > >>>>> Sasha Khapyorsky wrote: > >>>>>> Hi Yevgeny, > >>>>>> > >>>>>> On 17:01 Wed 17 Jan , Yevgeny Kliteynik wrote: > >>>>>>> Hi Hal > >>>>>>> > >>>>>>> The following series of six patches implements QoS policy file parser: > >>>>>>> > >>>>>>> 1. QoS parser Lex file > >>>>>>> 2. QoS parser Lex-generated c file > >>>>>>> 3. QoS parser grammar (Yacc) file > >>>>>>> 4. QoS parser Yacc-generated grammar c and h file > >>>>>>> 5. QoS parser header file that defines parse tree data structures > >>>>>>> 6. Changes in makefiles and configure.in file for compiling QoS > >>>>>>> parser files > >>>>>> Is there any description of proposed format and functionality? > >>>>> The parser is based on QoS RFC sent by Eitan in May 2006, with a few > >>>>> minor modifications. You can find the RFC here: > >>>>> http://openib.org/pipermail/openib-general/2006-May/022336.html > >>>> This was RFC and couple of issues were discussed then. Now you are about > >>>> implementation phase and exact format description would be desired. For > >>>> example what "few minor modifications" are? > >>> I'll prepare an example file with explanations. > >>> > >>> -- Yevgeny > >>> > >>>>>> Also what about using human readable formats? > >>>>> To me the xml-like format in the RFC looks pretty readable. > >>>>> It has very limited number of keywords (tags), so it's easy > >>>>> to follow and/or to modify. > >>>> It is your opinion, not everybody will agree with it (AFAIR this was > >>>> discussed too during RFC). > >>>> > >>>> I would not be care, but I don't know any example of really successful > >>>> XML using for configuration purposes (especially where advanced graphical > >>>> config editors/viewers were not used). Do you know? > >>>> > >>>> Sasha > >>>> > >>> _______________________________________________ > >>> openib-general mailing list > >>> openib-general@openib.org > >>> http://openib.org/mailman/listinfo/openib-general > >>> > >>> To unsubscribe, please visit > >>> http://openib.org/mailman/listinfo/openib-general > >>> > > _______________________________________________ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general