Re: [openib-general] [RFC] OpenSM Interactive Console
On Thu, Oct 27, 2005 at 09:29:57AM -0500, Troy Benjegerdes wrote: > I guess the point of all this is find a end-user use-case for the SM > MIB, and work back from there to decide if haveing a MIB actually helps > solve the problem. The end-use case is likely to be something like "an enterprise which insists on managing as much as possible through HP OpenView." Which isn't anyone in HPC, hence the current lack of interest. Now the things you'd actually want to monitor for a cluster, it's not really the normal stuff that's in MIBs. I'd want to know if a cable was unexpectedly unplugged, or if a node was up but its IB connection wasn't. I'd like to know if a link had an unusual error rate. -- greg ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] [RFC] OpenSM Interactive Console
Title: RE: [openib-general] [RFC] OpenSM Interactive Console Hi Hal, I still think that a "server" like behavior is much preferable to having the SM sit there and wait for console inputs. The SM is a service and thus should run like a daemon. MIB is just a standard way to avoid the need to define our own protocol to do that. In your implementation the SM should be put in console mode from the first invocation and thus will need a dedicated terminal. Even with osmsh one could implement (using standard Tcl sockets) a simple server that could just wait for remote commands (I can provide the code as I have done zillions of such servers). The MIB is nicer and I think it is not very complicated to implement. At least not the trivial groups of setting SM parameters. The more I think about it the more I get convinced we need to do it. Eitan Zahavi Design Technology Director Mellanox Technologies LTD Tel:+972-4-9097208 Fax:+972-4-9593245 P.O. Box 586 Yokneam 20692 ISRAEL > -Original Message- > From: Hal Rosenstock [mailto:[EMAIL PROTECTED]] > Sent: Thursday, October 27, 2005 1:45 PM > To: Eitan Zahavi > Cc: Troy Benjegerdes; openib-general@openib.org > Subject: RE: [openib-general] [RFC] OpenSM Interactive Console > > There have been requests for this CLI functionality from at least the labs. It has been > discussed on the list. > > Also, there was the following comment in OpenSM::main.c: > > /* > Sit here forever > In the future, some sort of console interactivity could > be implemented in this loop. > */ > > -- Hal > > > > From: Eitan Zahavi [mailto:[EMAIL PROTECTED]] > Sent: Thu 10/27/2005 2:03 AM > To: Hal Rosenstock; Eitan Zahavi > Cc: Troy Benjegerdes; openib-general@openib.org > Subject: RE: [openib-general] [RFC] OpenSM Interactive Console > > > > Yes this MIB needs some cleanup. > I would love to hear from the community some feedback regarding SM MIB > usefulness. > > In the past we did not get any push for interactive SM or online configurable SM so I > did not see any reason to work on it. > > I do not think it is a huge task to make SM MIB work with OpenSM. At least not the > 90% of it that I glanced through. > > > Eitan Zahavi > Design Technology Director > Mellanox Technologies LTD > Tel:+972-4-9097208 > Fax:+972-4-9593245 > P.O. Box 586 Yokneam 20692 ISRAEL > > > > -Original Message- > > From: Hal Rosenstock [mailto:[EMAIL PROTECTED]] > > Sent: Wednesday, October 26, 2005 7:44 PM > > To: Eitan Zahavi > > Cc: Troy Benjegerdes; openib-general@openib.org > > Subject: RE: [openib-general] [RFC] OpenSM Interactive Console > > > > Hi Eitan, > > > > I sit corrected. There are R/W parameters in the SM MIB as you indicate. I was > > thinking of all the other IPoIB MIBs. It's been a while since I looked at the SM MIB. > > > > Also, the SM MIB (draft-ietf-ipoib-subnet-manager-mib-00) expired a while ago. At > a > > minimum, it needs to be dusted off. That would include updating it for IBA 1.2. > > > > -- Hal > > > > > > > > From: Eitan Zahavi [mailto:[EMAIL PROTECTED]] > > Sent: Tue 10/25/2005 5:19 AM > > To: Hal Rosenstock > > Cc: Troy Benjegerdes; openib-general@openib.org > > Subject: Re: [openib-general] [RFC] OpenSM Interactive Console > > > > > > > > Hal Rosenstock wrote: > > > On Mon, 2005-10-24 at 14:38, Eitan Zahavi wrote: > > > > > >>Hal Rosenstock wrote: > > >> > > >>>On Mon, 2005-10-24 at 03:08, Eitan Zahavi wrote: > > >>> > > >>> > > >>>>I would suggest to use SNMP for the tasks below. IETF IPoIB group > > > > > > has > > > > > >>>>defined an SNMP MIB that can support the required functionality > > > > > > below. > > > > > >>> > > >>>The IETF SNMP MIBs are one way of presenting the information to the > > >>>outside world. There are other possible management interfaces. The > > > > > > SNMP > > > > > >>>MIB instrumentation would need to use lower layer APIs to get this > > >>>information out of the SM. > > >> > > >>Yes but the IETF SM MIB is the only one that is close to a standard > > > > > > way. > > > > > >>It does not require low level interface if it will integrate into the > > > > > > OpenSM code. > > > > > >>One way to do it is buy extending OpenSM with an AgentX interface. >
Re: [openib-general] [RFC] OpenSM Interactive Console
For me, the only purpose for an SNMP MIB would be to get the information into a network management system. In my case, I'll be using something that's open-source or has a plugin architecture like Nagios, and I'd really rather just have the network management system communicate with the subnet manager or SMA packets directly rather than introducting an extra translation to SNMP. SNMP is only usefull to me because it is (in theory) an interoperable cross-vendor standard. In the infiniband case, we already have a cross-vendor standard implementation (OpenIB), and adding SNMP is another dependency and layer of complexity that can break and be difficult to set up. If I knew of an open-source tool that was actually able to use SNMP to query a random ethernet vendor's switch and be able to tell me what port a particular MAC address was plugged into, I might be more positive. But as far as I know, each vendor's SNMP implementation is broken in subtly different ways, so that this gets to be a nightmare to actually implement. I guess the point of all this is find a end-user use-case for the SM MIB, and work back from there to decide if haveing a MIB actually helps solve the problem. On Thu, Oct 27, 2005 at 08:03:57AM +0200, Eitan Zahavi wrote: > Yes this MIB needs some cleanup. > I would love to hear from the community some feedback regarding SM MIB > usefulness. > > In the past we did not get any push for interactive SM or online > configurable SM so I did not see any reason to work on it. > > I do not think it is a huge task to make SM MIB work with OpenSM. At least > not the 90% of it that I glanced through. > > > Eitan Zahavi > Design Technology Director > Mellanox Technologies LTD > Tel:+972-4-9097208 > Fax:+972-4-9593245 > P.O. Box 586 Yokneam 20692 ISRAEL > > > > -Original Message- > > From: Hal Rosenstock [mailto:[EMAIL PROTECTED] > > Sent: Wednesday, October 26, 2005 7:44 PM > > To: Eitan Zahavi > > Cc: Troy Benjegerdes; openib-general@openib.org > > Subject: RE: [openib-general] [RFC] OpenSM Interactive Console > > > > Hi Eitan, > > > > I sit corrected. There are R/W parameters in the SM MIB as you indicate. I > was > > thinking of all the other IPoIB MIBs. It's been a while since I looked at > the SM MIB. > > > > Also, the SM MIB (draft-ietf-ipoib-subnet-manager-mib-00) expired a while > ago. At a > > minimum, it needs to be dusted off. That would include updating it for IBA > 1.2. > > > > -- Hal > > > > ____________________ > > > > From: Eitan Zahavi [mailto:[EMAIL PROTECTED] > > Sent: Tue 10/25/2005 5:19 AM > > To: Hal Rosenstock > > Cc: Troy Benjegerdes; openib-general@openib.org > > Subject: Re: [openib-general] [RFC] OpenSM Interactive Console > > > > > > > > Hal Rosenstock wrote: > > > On Mon, 2005-10-24 at 14:38, Eitan Zahavi wrote: > > > > > >>Hal Rosenstock wrote: > > >> > > >>>On Mon, 2005-10-24 at 03:08, Eitan Zahavi wrote: > > >>> > > >>> > > >>>>I would suggest to use SNMP for the tasks below. IETF IPoIB group > > > > > > has > > > > > >>>>defined an SNMP MIB that can support the required functionality > > > > > > below. > > > > > >>> > > >>>The IETF SNMP MIBs are one way of presenting the information to the > > >>>outside world. There are other possible management interfaces. The > > > > > > SNMP > > > > > >>>MIB instrumentation would need to use lower layer APIs to get this > > >>>information out of the SM. > > >> > > >>Yes but the IETF SM MIB is the only one that is close to a standard > > > > > > way. > > > > > >>It does not require low level interface if it will integrate into the > > > > > > OpenSM code. > > > > > >>One way to do it is buy extending OpenSM with an AgentX interface. > > >> > > >>IMO one clear advantage of using SNMP for SM integration is that the > > > > > > code will work with any SM that is IETF compliant. > > > > > >>Also if you want to write a "client server" type of application on top > > > > > > of an SM you > > > > > >>can either stick to sending MADs which translate into SA client based > > > > > > application or > > > > > >>you better stay with some known protocol for management (like SNMP) > > > > > > and not develop
RE: [openib-general] [RFC] OpenSM Interactive Console
There have been requests for this CLI functionality from at least the labs. It
has been discussed on the list.
Also, there was the following comment in OpenSM::main.c:
/*
Sit here forever
In the future, some sort of console interactivity could
be implemented in this loop.
*/
-- Hal
From: Eitan Zahavi [mailto:[EMAIL PROTECTED]
Sent: Thu 10/27/2005 2:03 AM
To: Hal Rosenstock; Eitan Zahavi
Cc: Troy Benjegerdes; openib-general@openib.org
Subject: RE: [openib-general] [RFC] OpenSM Interactive Console
Yes this MIB needs some cleanup.
I would love to hear from the community some feedback regarding SM MIB
usefulness.
In the past we did not get any push for interactive SM or online configurable
SM so I did not see any reason to work on it.
I do not think it is a huge task to make SM MIB work with OpenSM. At least not
the 90% of it that I glanced through.
Eitan Zahavi
Design Technology Director
Mellanox Technologies LTD
Tel:+972-4-9097208
Fax:+972-4-9593245
P.O. Box 586 Yokneam 20692 ISRAEL
> -Original Message-
> From: Hal Rosenstock [mailto:[EMAIL PROTECTED]
> Sent: Wednesday, October 26, 2005 7:44 PM
> To: Eitan Zahavi
> Cc: Troy Benjegerdes; openib-general@openib.org
> Subject: RE: [openib-general] [RFC] OpenSM Interactive Console
>
> Hi Eitan,
>
> I sit corrected. There are R/W parameters in the SM MIB as you indicate. I
> was
> thinking of all the other IPoIB MIBs. It's been a while since I looked at the
> SM MIB.
>
> Also, the SM MIB (draft-ietf-ipoib-subnet-manager-mib-00) expired a while
> ago. At a
> minimum, it needs to be dusted off. That would include updating it for IBA
> 1.2.
>
> -- Hal
>
>
>
> From: Eitan Zahavi [mailto:[EMAIL PROTECTED]
> Sent: Tue 10/25/2005 5:19 AM
> To: Hal Rosenstock
> Cc: Troy Benjegerdes; openib-general@openib.org
> Subject: Re: [openib-general] [RFC] OpenSM Interactive Console
>
>
>
> Hal Rosenstock wrote:
> > On Mon, 2005-10-24 at 14:38, Eitan Zahavi wrote:
> >
> >>Hal Rosenstock wrote:
> >>
> >>>On Mon, 2005-10-24 at 03:08, Eitan Zahavi wrote:
> >>>
> >>>
> >>>>I would suggest to use SNMP for the tasks below. IETF IPoIB group
> >
> > has
> >
> >>>>defined an SNMP MIB that can support the required functionality
> >
> > below.
> >
> >>>
> >>>The IETF SNMP MIBs are one way of presenting the information to the
> >>>outside world. There are other possible management interfaces. The
> >
> > SNMP
> >
> >>>MIB instrumentation would need to use lower layer APIs to get this
> >>>information out of the SM.
> >>
> >>Yes but the IETF SM MIB is the only one that is close to a standard
> >
> > way.
> >
> >>It does not require low level interface if it will integrate into the
> >
> > OpenSM code.
> >
> >>One way to do it is buy extending OpenSM with an AgentX interface.
> >>
> >>IMO one clear advantage of using SNMP for SM integration is that the
> >
> > code will work with any SM that is IETF compliant.
> >
> >>Also if you want to write a "client server" type of application on top
> >
> > of an SM you
> >
> >>can either stick to sending MADs which translate into SA client based
> >
> > application or
> >
> >>you better stay with some known protocol for management (like SNMP)
> >
> > and not develop yet another protocol for
> >
> >>doing exactly the same things as SNMP already supports.
> >
> >
> > There are limitations in the SNMP MIBs. One is that they are RO so they
> > are more for monitoring. Also, many environments do not use SNMP. It is
> > unclear how much of a requirement it is to manage any SM or how many
> > other SMs support the SM MIB. (There are other IB associated MIBs too).
>
> SNMP MIBs are certainly not just RO a simple example from the SM MIB:
>ibSmPortInfoLMC OBJECT-TYPE
>SYNTAX Unsigned32(0..7)
>MAX-ACCESS read-write
>STATUS current
>DESCRIPTION
> "LID mask for multipath support. User should take extra caution
> when setting this value, since any change will effect packet
> routing."
>::= { ibSmPortInfoEntry 19 }
>
>
> I agree that it is possible that currently no SM is supporting the SM MIB.
> But it does make sense to have ALL of the them suppor
RE: [openib-general] [RFC] OpenSM Interactive Console
Title: RE: [openib-general] [RFC] OpenSM Interactive Console
Yes this MIB needs some cleanup.
I would love to hear from the community some feedback regarding SM MIB usefulness.
In the past we did not get any push for interactive SM or online configurable SM so I did not see any reason to work on it.
I do not think it is a huge task to make SM MIB work with OpenSM. At least not the 90% of it that I glanced through.
Eitan Zahavi
Design Technology Director
Mellanox Technologies LTD
Tel:+972-4-9097208
Fax:+972-4-9593245
P.O. Box 586 Yokneam 20692 ISRAEL
> -Original Message-
> From: Hal Rosenstock [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, October 26, 2005 7:44 PM
> To: Eitan Zahavi
> Cc: Troy Benjegerdes; openib-general@openib.org
> Subject: RE: [openib-general] [RFC] OpenSM Interactive Console
>
> Hi Eitan,
>
> I sit corrected. There are R/W parameters in the SM MIB as you indicate. I was
> thinking of all the other IPoIB MIBs. It's been a while since I looked at the SM MIB.
>
> Also, the SM MIB (draft-ietf-ipoib-subnet-manager-mib-00) expired a while ago. At a
> minimum, it needs to be dusted off. That would include updating it for IBA 1.2.
>
> -- Hal
>
>
>
> From: Eitan Zahavi [mailto:[EMAIL PROTECTED]]
> Sent: Tue 10/25/2005 5:19 AM
> To: Hal Rosenstock
> Cc: Troy Benjegerdes; openib-general@openib.org
> Subject: Re: [openib-general] [RFC] OpenSM Interactive Console
>
>
>
> Hal Rosenstock wrote:
> > On Mon, 2005-10-24 at 14:38, Eitan Zahavi wrote:
> >
> >>Hal Rosenstock wrote:
> >>
> >>>On Mon, 2005-10-24 at 03:08, Eitan Zahavi wrote:
> >>>
> >>>
> >>>>I would suggest to use SNMP for the tasks below. IETF IPoIB group
> >
> > has
> >
> >>>>defined an SNMP MIB that can support the required functionality
> >
> > below.
> >
> >>>
> >>>The IETF SNMP MIBs are one way of presenting the information to the
> >>>outside world. There are other possible management interfaces. The
> >
> > SNMP
> >
> >>>MIB instrumentation would need to use lower layer APIs to get this
> >>>information out of the SM.
> >>
> >>Yes but the IETF SM MIB is the only one that is close to a standard
> >
> > way.
> >
> >>It does not require low level interface if it will integrate into the
> >
> > OpenSM code.
> >
> >>One way to do it is buy extending OpenSM with an AgentX interface.
> >>
> >>IMO one clear advantage of using SNMP for SM integration is that the
> >
> > code will work with any SM that is IETF compliant.
> >
> >>Also if you want to write a "client server" type of application on top
> >
> > of an SM you
> >
> >>can either stick to sending MADs which translate into SA client based
> >
> > application or
> >
> >>you better stay with some known protocol for management (like SNMP)
> >
> > and not develop yet another protocol for
> >
> >>doing exactly the same things as SNMP already supports.
> >
> >
> > There are limitations in the SNMP MIBs. One is that they are RO so they
> > are more for monitoring. Also, many environments do not use SNMP. It is
> > unclear how much of a requirement it is to manage any SM or how many
> > other SMs support the SM MIB. (There are other IB associated MIBs too).
>
> SNMP MIBs are certainly not just RO a simple example from the SM MIB:
> ibSmPortInfoLMC OBJECT-TYPE
> SYNTAX Unsigned32(0..7)
> MAX-ACCESS read-write
> STATUS current
> DESCRIPTION
> "LID mask for multipath support. User should take extra caution
> when setting this value, since any change will effect packet
> routing."
> ::= { ibSmPortInfoEntry 19 }
>
>
> I agree that it is possible that currently no SM is supporting the SM MIB.
> But it does make sense to have ALL of the them support it. Such that they can
> be activated/deactivated and configured in the manner.
>
> Most unix distributions and windows box have standard SNMP agent and client
> included in them
> So it does not take more then simple bash or C code to interact with the SM if it
> supports SNMP.
>
> >
> >
> >>>>Everything but the dynamic partitioning (OpenSM does not have
> >>>>partition manager to this moment)
> >>>
> >>>
> >>>What Troy meant by partitioning is not necessarily IB partitioning.
> >
RE: [openib-general] [RFC] OpenSM Interactive Console
Hi Eitan,
I sit corrected. There are R/W parameters in the SM MIB as you indicate. I was
thinking of all the other IPoIB MIBs. It's been a while since I looked at the
SM MIB.
Also, the SM MIB (draft-ietf-ipoib-subnet-manager-mib-00) expired a while ago.
At a minimum, it needs to be dusted off. That would include updating it for IBA
1.2.
-- Hal
From: Eitan Zahavi [mailto:[EMAIL PROTECTED]
Sent: Tue 10/25/2005 5:19 AM
To: Hal Rosenstock
Cc: Troy Benjegerdes; openib-general@openib.org
Subject: Re: [openib-general] [RFC] OpenSM Interactive Console
Hal Rosenstock wrote:
> On Mon, 2005-10-24 at 14:38, Eitan Zahavi wrote:
>
>>Hal Rosenstock wrote:
>>
>>>On Mon, 2005-10-24 at 03:08, Eitan Zahavi wrote:
>>>
>>>
>>>>I would suggest to use SNMP for the tasks below. IETF IPoIB group
>
> has
>
>>>>defined an SNMP MIB that can support the required functionality
>
> below.
>
>>>
>>>The IETF SNMP MIBs are one way of presenting the information to the
>>>outside world. There are other possible management interfaces. The
>
> SNMP
>
>>>MIB instrumentation would need to use lower layer APIs to get this
>>>information out of the SM.
>>
>>Yes but the IETF SM MIB is the only one that is close to a standard
>
> way.
>
>>It does not require low level interface if it will integrate into the
>
> OpenSM code.
>
>>One way to do it is buy extending OpenSM with an AgentX interface.
>>
>>IMO one clear advantage of using SNMP for SM integration is that the
>
> code will work with any SM that is IETF compliant.
>
>>Also if you want to write a "client server" type of application on top
>
> of an SM you
>
>>can either stick to sending MADs which translate into SA client based
>
> application or
>
>>you better stay with some known protocol for management (like SNMP)
>
> and not develop yet another protocol for
>
>>doing exactly the same things as SNMP already supports.
>
>
> There are limitations in the SNMP MIBs. One is that they are RO so they
> are more for monitoring. Also, many environments do not use SNMP. It is
> unclear how much of a requirement it is to manage any SM or how many
> other SMs support the SM MIB. (There are other IB associated MIBs too).
SNMP MIBs are certainly not just RO a simple example from the SM MIB:
ibSmPortInfoLMC OBJECT-TYPE
SYNTAX Unsigned32(0..7)
MAX-ACCESS read-write
STATUS current
DESCRIPTION
"LID mask for multipath support. User should take extra caution
when setting this value, since any change will effect packet
routing."
::= { ibSmPortInfoEntry 19 }
I agree that it is possible that currently no SM is supporting the SM MIB.
But it does make sense to have ALL of the them support it. Such that they can
be activated/deactivated and configured in the manner.
Most unix distributions and windows box have standard SNMP agent and client
included in them
So it does not take more then simple bash or C code to interact with the SM if
it supports SNMP.
>
>
>>>>Everything but the dynamic partitioning (OpenSM does not have
>>>>partition manager to this moment)
>>>
>>>
>>>What Troy meant by partitioning is not necessarily IB partitioning.
>>
>>How are you sure about that? Troy - please comment.
>
>
> I think you missed an email on this.
>
>
>>>>and forwarding of Performance
>>>>Monitoring traps (which are generated by the PM) can be done through
>>>>osmsh or through SA client today.
>>>
>>>
>>>What PerfMgr are you referring to ?
>>
>>No specific one. But the specification does not require the SM too.
>
>
> Huh ? What spec ? An SM is required in a subnet. There is no subnet
> without this. There is a subnet without a PerfMgr.
Yes its a typo I meant PM. SM is a requirement. You know I did not mean that.
>
>
>>For various reasons (like load) it might make more sense to have the
>
> PM distributed.
>
> Sure. Also, the PerfMgr need not be colocated with the SM anyhow.
>
>
>>Anyway, my point is that the SM is not the owner of PM trap reporting.
>
> It is the PM that
>
>>should support Reporting (I.e InformInfo registration and Trap
>
> forwarding) for PM traps.
>
>>But the spec does not define such traps anyway.
>
>
> My point was that the PerfMgr is beyond the IBA spec. It is only the PMA
> that is defined and has no traps so these will all need synthesis by the
> PerfMgr.
Agree.
>
> -- Hal
>
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [RFC] OpenSM Interactive Console
Hal Rosenstock wrote:
On Mon, 2005-10-24 at 14:38, Eitan Zahavi wrote:
Hal Rosenstock wrote:
On Mon, 2005-10-24 at 03:08, Eitan Zahavi wrote:
I would suggest to use SNMP for the tasks below. IETF IPoIB group
has
defined an SNMP MIB that can support the required functionality
below.
The IETF SNMP MIBs are one way of presenting the information to the
outside world. There are other possible management interfaces. The
SNMP
MIB instrumentation would need to use lower layer APIs to get this
information out of the SM.
Yes but the IETF SM MIB is the only one that is close to a standard
way.
It does not require low level interface if it will integrate into the
OpenSM code.
One way to do it is buy extending OpenSM with an AgentX interface.
IMO one clear advantage of using SNMP for SM integration is that the
code will work with any SM that is IETF compliant.
Also if you want to write a "client server" type of application on top
of an SM you
can either stick to sending MADs which translate into SA client based
application or
you better stay with some known protocol for management (like SNMP)
and not develop yet another protocol for
doing exactly the same things as SNMP already supports.
There are limitations in the SNMP MIBs. One is that they are RO so they
are more for monitoring. Also, many environments do not use SNMP. It is
unclear how much of a requirement it is to manage any SM or how many
other SMs support the SM MIB. (There are other IB associated MIBs too).
SNMP MIBs are certainly not just RO a simple example from the SM MIB:
ibSmPortInfoLMC OBJECT-TYPE
SYNTAX Unsigned32(0..7)
MAX-ACCESS read-write
STATUS current
DESCRIPTION
"LID mask for multipath support. User should take extra caution
when setting this value, since any change will effect packet
routing."
::= { ibSmPortInfoEntry 19 }
I agree that it is possible that currently no SM is supporting the SM MIB.
But it does make sense to have ALL of the them support it. Such that they can
be activated/deactivated and configured in the manner.
Most unix distributions and windows box have standard SNMP agent and client
included in them
So it does not take more then simple bash or C code to interact with the SM if
it supports SNMP.
Everything but the dynamic partitioning (OpenSM does not have
partition manager to this moment)
What Troy meant by partitioning is not necessarily IB partitioning.
How are you sure about that? Troy - please comment.
I think you missed an email on this.
and forwarding of Performance
Monitoring traps (which are generated by the PM) can be done through
osmsh or through SA client today.
What PerfMgr are you referring to ?
No specific one. But the specification does not require the SM too.
Huh ? What spec ? An SM is required in a subnet. There is no subnet
without this. There is a subnet without a PerfMgr.
Yes its a typo I meant PM. SM is a requirement. You know I did not mean that.
For various reasons (like load) it might make more sense to have the
PM distributed.
Sure. Also, the PerfMgr need not be colocated with the SM anyhow.
Anyway, my point is that the SM is not the owner of PM trap reporting.
It is the PM that
should support Reporting (I.e InformInfo registration and Trap
forwarding) for PM traps.
But the spec does not define such traps anyway.
My point was that the PerfMgr is beyond the IBA spec. It is only the PMA
that is defined and has no traps so these will all need synthesis by the
PerfMgr.
Agree.
-- Hal
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [RFC] OpenSM Interactive Console
On Mon, 2005-10-24 at 14:38, Eitan Zahavi wrote: > Hal Rosenstock wrote: > > On Mon, 2005-10-24 at 03:08, Eitan Zahavi wrote: > > > >>I would suggest to use SNMP for the tasks below. IETF IPoIB group has > >>defined an SNMP MIB that can support the required functionality below. > > > > > > The IETF SNMP MIBs are one way of presenting the information to the > > outside world. There are other possible management interfaces. The SNMP > > MIB instrumentation would need to use lower layer APIs to get this > > information out of the SM. > Yes but the IETF SM MIB is the only one that is close to a standard way. > It does not require low level interface if it will integrate into the OpenSM > code. > One way to do it is buy extending OpenSM with an AgentX interface. > > IMO one clear advantage of using SNMP for SM integration is that the code > will work with any SM that is IETF compliant. > Also if you want to write a "client server" type of application on top of an > SM you > can either stick to sending MADs which translate into SA client based > application or > you better stay with some known protocol for management (like SNMP) and not > develop yet another protocol for > doing exactly the same things as SNMP already supports. There are limitations in the SNMP MIBs. One is that they are RO so they are more for monitoring. Also, many environments do not use SNMP. It is unclear how much of a requirement it is to manage any SM or how many other SMs support the SM MIB. (There are other IB associated MIBs too). > >>Everything but the dynamic partitioning (OpenSM does not have > >>partition manager to this moment) > > > > > > What Troy meant by partitioning is not necessarily IB partitioning. > How are you sure about that? Troy - please comment. I think you missed an email on this. > >>and forwarding of Performance > >>Monitoring traps (which are generated by the PM) can be done through > >>osmsh or through SA client today. > > > > > > What PerfMgr are you referring to ? > No specific one. But the specification does not require the SM too. Huh ? What spec ? An SM is required in a subnet. There is no subnet without this. There is a subnet without a PerfMgr. > For various reasons (like load) it might make more sense to have the PM > distributed. Sure. Also, the PerfMgr need not be colocated with the SM anyhow. > Anyway, my point is that the SM is not the owner of PM trap reporting. It is > the PM that > should support Reporting (I.e InformInfo registration and Trap forwarding) > for PM traps. > But the spec does not define such traps anyway. My point was that the PerfMgr is beyond the IBA spec. It is only the PMA that is defined and has no traps so these will all need synthesis by the PerfMgr. -- Hal ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [RFC] OpenSM Interactive Console
Hal Rosenstock wrote: On Mon, 2005-10-24 at 03:08, Eitan Zahavi wrote: I would suggest to use SNMP for the tasks below. IETF IPoIB group has defined an SNMP MIB that can support the required functionality below. The IETF SNMP MIBs are one way of presenting the information to the outside world. There are other possible management interfaces. The SNMP MIB instrumentation would need to use lower layer APIs to get this information out of the SM. Yes but the IETF SM MIB is the only one that is close to a standard way. It does not require low level interface if it will integrate into the OpenSM code. One way to do it is buy extending OpenSM with an AgentX interface. IMO one clear advantage of using SNMP for SM integration is that the code will work with any SM that is IETF compliant. Also if you want to write a "client server" type of application on top of an SM you can either stick to sending MADs which translate into SA client based application or you better stay with some known protocol for management (like SNMP) and not develop yet another protocol for doing exactly the same things as SNMP already supports. Everything but the dynamic partitioning (OpenSM does not have partition manager to this moment) What Troy meant by partitioning is not necessarily IB partitioning. How are you sure about that? Troy - please comment. and forwarding of Performance Monitoring traps (which are generated by the PM) can be done through osmsh or through SA client today. What PerfMgr are you referring to ? No specific one. But the specification does not require the SM too. For various reasons (like load) it might make more sense to have the PM distributed. Anyway, my point is that the SM is not the owner of PM trap reporting. It is the PM that should support Reporting (I.e InformInfo registration and Trap forwarding) for PM traps. But the spec does not define such traps anyway. -- Hal EZ Eitan Zahavi Design Technology Director Mellanox Technologies LTD Tel:+972-4-9097208 Fax:+972-4-9593245 P.O. Box 586 Yokneam 20692 ISRAEL -Original Message- From: Troy Benjegerdes [mailto:[EMAIL PROTECTED] Sent: Thursday, October 20, 2005 3:23 AM To: Hal Rosenstock Cc: openib-general@openib.org Subject: Re: [openib-general] [RFC] OpenSM Interactive Console On Tue, Oct 18, 2005 at 03:10:31PM -0400, Hal Rosenstock wrote: Currently, OpenSM does not support an interactive console. There has been a desire to introduce the ability to change certain parameters (as well as display things) once OpenSM has started. This patch introduces the first most basic commands: help and loglevel. I am investgating adding smpriority to this. The console is invoked by specifying -console as an option on the opensm command line. If you have a request for a command you would like in the console, I would like to compile a list of these. Comments ? As well as a console, I'd like an API for some way for external programs (say a cluster queue manager) to be able to query the SM (or the sm + some helper library) for the following things: * Topology * guid/lid/IPoIB address/switch port mappings * link state Future neat things to do: * An interface to dynamically partition the fabric * Register for notifications for certain events (excessive traffic queueing, or error counts) ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] [RFC] OpenSM Interactive Console
On Mon, 2005-10-24 at 03:08, Eitan Zahavi wrote: > I would suggest to use SNMP for the tasks below. IETF IPoIB group has > defined an SNMP MIB that can support the required functionality below. The IETF SNMP MIBs are one way of presenting the information to the outside world. There are other possible management interfaces. The SNMP MIB instrumentation would need to use lower layer APIs to get this information out of the SM. > Everything but the dynamic partitioning (OpenSM does not have > partition manager to this moment) What Troy meant by partitioning is not necessarily IB partitioning. > and forwarding of Performance > Monitoring traps (which are generated by the PM) can be done through > osmsh or through SA client today. What PerfMgr are you referring to ? -- Hal > EZ > > Eitan Zahavi > Design Technology Director > Mellanox Technologies LTD > Tel:+972-4-9097208 > Fax:+972-4-9593245 > P.O. Box 586 Yokneam 20692 ISRAEL > > > > -Original Message- > > From: Troy Benjegerdes [mailto:[EMAIL PROTECTED] > > Sent: Thursday, October 20, 2005 3:23 AM > > To: Hal Rosenstock > > Cc: openib-general@openib.org > > Subject: Re: [openib-general] [RFC] OpenSM Interactive Console > > > > On Tue, Oct 18, 2005 at 03:10:31PM -0400, Hal Rosenstock wrote: > > > Currently, OpenSM does not support an interactive console. There > has > > > been a desire to introduce the ability to change certain > parameters (as > > > well as display things) once OpenSM has started. This patch > introduces > > > the first most basic commands: help and loglevel. I am > investgating > > > adding smpriority to this. The console is invoked by specifying > -console > > > as an option on the opensm command line. > > > > > > If you have a request for a command you would like in the console, > I > > > would like to compile a list of these. > > > > > > Comments ? > > > > As well as a console, I'd like an API for some way for external > programs > > (say a cluster queue manager) to be able to query the SM (or the sm > + some > > helper library) for the following things: > > > > * Topology > > * guid/lid/IPoIB address/switch port mappings > > * link state > > > > Future neat things to do: > > > > * An interface to dynamically partition the fabric > > * Register for notifications for certain events (excessive traffic > > queueing, or error counts) > > ___ > > openib-general mailing list > > openib-general@openib.org > > http://openib.org/mailman/listinfo/openib-general > > > > To unsubscribe, please visit > http://openib.org/mailman/listinfo/openib-general > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] [RFC] OpenSM Interactive Console
Title: RE: [openib-general] [RFC] OpenSM Interactive Console I would suggest to use SNMP for the tasks below. IETF IPoIB group has defined an SNMP MIB that can support the required functionality below. Everything but the dynamic partitioning (OpenSM does not have partition manager to this moment) and forwarding of Performance Monitoring traps (which are generated by the PM) can be done through osmsh or through SA client today. EZ Eitan Zahavi Design Technology Director Mellanox Technologies LTD Tel:+972-4-9097208 Fax:+972-4-9593245 P.O. Box 586 Yokneam 20692 ISRAEL > -Original Message- > From: Troy Benjegerdes [mailto:[EMAIL PROTECTED]] > Sent: Thursday, October 20, 2005 3:23 AM > To: Hal Rosenstock > Cc: openib-general@openib.org > Subject: Re: [openib-general] [RFC] OpenSM Interactive Console > > On Tue, Oct 18, 2005 at 03:10:31PM -0400, Hal Rosenstock wrote: > > Currently, OpenSM does not support an interactive console. There has > > been a desire to introduce the ability to change certain parameters (as > > well as display things) once OpenSM has started. This patch introduces > > the first most basic commands: help and loglevel. I am investgating > > adding smpriority to this. The console is invoked by specifying -console > > as an option on the opensm command line. > > > > If you have a request for a command you would like in the console, I > > would like to compile a list of these. > > > > Comments ? > > As well as a console, I'd like an API for some way for external programs > (say a cluster queue manager) to be able to query the SM (or the sm + some > helper library) for the following things: > > * Topology > * guid/lid/IPoIB address/switch port mappings > * link state > > Future neat things to do: > > * An interface to dynamically partition the fabric > * Register for notifications for certain events (excessive traffic > queueing, or error counts) > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [RFC] OpenSM Interactive Console
On Thu, 2005-10-20 at 09:53, Troy Benjegerdes wrote: > > > * Topology > > > > This can be done via SA queries currently. > > > > > * guid/lid/IPoIB address/switch port mappings > > > > The SM does not know (see) IPoIB addresses. The only thing it sees is > > the part of the subnet address. > > > > The rest can be done via SA queries currently. > > > > > * link state > > > > This can be done via SA query currently. > > > > This argues for a higher layer API to make these queries easy. > > > > > Future neat things to do: > > > > > > * An interface to dynamically partition the fabric > > > > Is this referring to IB partitioning ? > > I think so, but IB partitioning may not actually map to what I'm > interested in. From the high-level (application) point of view, I want to > ensure that communication traffic for one cluster job minimally affects > another job. Do the set of end nodes overlap for jobs ? This might be via using different SLs rather than different (IB) partitions depending on the requirement. In any case, there is more work here than just this API. > > > * Register for notifications for certain events (excessive traffic > > > queueing, or error counts) > > > > Not sure what you mean by excessive traffic queuing. > > I guess I'd like to know whenever utilization on a single link exceeds > 90%, or whatever % you would want to be notified about (with sampling/polling at some interval (assuming there is no IB defined event for these). > or the queuing delay exceeds XXX nanoseconds. I think you are talking more in the abstract here. I need to think about this one some more as to if/how to determine something like this for IB. > > It is the event set which is of interest to me. Are there others ? > > > > There are a set of events which can be subscribed to currently. The ones > > along these lines are local link integrity threshold reached on a port, > > excessive buffer overrun threshold reached on a port, flow control and > > update watchdog timer expired on a switch port. > > > > If you are referring to the PortCounters, these would need to be polled > > (at some periodicity) and then an event created as there is no event for > > this defined in IBA. > > > > Higher layer APIs could help with this area too. > > Some of this stuff may not necessarily belong in the OpenSM process either.. > Stuff like getting IPoIB address from GUID's would be usefull in a > library, but isn't the SM's responsibility. There are a couple of approaches I can imagine for obtaining the mappings of GUID to IPoIB address(es). 1. Vendor specific MADs could be implemented for this but this is ugly. Interaction would be required to register and unregister each IPoIB address with the vendor specific agent for this. 2. OpenSM node needs to be on either all IPoIB subnets or those of "interest". It could then do the equivalent of a broadcast ping on each IPoIB subnet and match the ARP/neighbor entries with the GUID requested. Note that the same GUID can have multiple IP addresses on the same or different subnets. A RARP based approach won't work as the QPN is also part of the IPoIB hardware address. -- Hal ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [RFC] OpenSM Interactive Console
> > * Topology > > This can be done via SA queries currently. > > > * guid/lid/IPoIB address/switch port mappings > > The SM does not know (see) IPoIB addresses. The only thing it sees is > the part of the subnet address. > > The rest can be done via SA queries currently. > > > * link state > > This can be done via SA query currently. > > This argues for a higher layer API to make these queries easy. > > > Future neat things to do: > > > > * An interface to dynamically partition the fabric > > Is this referring to IB partitioning ? I think so, but IB partitioning may not actually map to what I'm interested in. From the high-level (application) point of view, I want to ensure that communication traffic for one cluster job minimally affects another job. > > * Register for notifications for certain events (excessive traffic > > queueing, or error counts) > > Not sure what you mean by excessive traffic queuing. I guess I'd like to know whenever utilization on a single link exceeds 90%, or the queuing delay exceeds XXX nanoseconds. > It is the event set which is of interest to me. Are there others ? > > There are a set of events which can be subscribed to currently. The ones > along these lines are local link integrity threshold reached on a port, > excessive buffer overrun threshold reached on a port, flow control and > update watchdog timer expired on a switch port. > > If you are referring to the PortCounters, these would need to be polled > (at some periodicity) and then an event created as there is no event for > this defined in IBA. > > Higher layer APIs could help with this area too. Some of this stuff may not necessarily belong in the OpenSM process either.. Stuff like getting IPoIB address from GUID's would be usefull in a library, but isn't the SM's responsibility. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [RFC] OpenSM Interactive Console
On Wed, 2005-10-19 at 21:23, Troy Benjegerdes wrote: > On Tue, Oct 18, 2005 at 03:10:31PM -0400, Hal Rosenstock wrote: > > Currently, OpenSM does not support an interactive console. There has > > been a desire to introduce the ability to change certain parameters (as > > well as display things) once OpenSM has started. This patch introduces > > the first most basic commands: help and loglevel. I am investgating > > adding smpriority to this. The console is invoked by specifying -console > > as an option on the opensm command line. > > > > If you have a request for a command you would like in the console, I > > would like to compile a list of these. > > > > Comments ? > > As well as a console, I'd like an API for some way for external programs > (say a cluster queue manager) to be able to query the SM (or the sm + some > helper library) for the following things: > > * Topology This can be done via SA queries currently. > * guid/lid/IPoIB address/switch port mappings The SM does not know (see) IPoIB addresses. The only thing it sees is the part of the subnet address. The rest can be done via SA queries currently. > * link state This can be done via SA query currently. This argues for a higher layer API to make these queries easy. > Future neat things to do: > > * An interface to dynamically partition the fabric Is this referring to IB partitioning ? > * Register for notifications for certain events (excessive traffic > queueing, or error counts) Not sure what you mean by excessive traffic queuing. It is the event set which is of interest to me. Are there others ? There are a set of events which can be subscribed to currently. The ones along these lines are local link integrity threshold reached on a port, excessive buffer overrun threshold reached on a port, flow control and update watchdog timer expired on a switch port. If you are referring to the PortCounters, these would need to be polled (at some periodicity) and then an event created as there is no event for this defined in IBA. Higher layer APIs could help with this area too. Thanks for the input. -- Hal ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [RFC] OpenSM Interactive Console
On Tue, Oct 18, 2005 at 03:10:31PM -0400, Hal Rosenstock wrote: > Currently, OpenSM does not support an interactive console. There has > been a desire to introduce the ability to change certain parameters (as > well as display things) once OpenSM has started. This patch introduces > the first most basic commands: help and loglevel. I am investgating > adding smpriority to this. The console is invoked by specifying -console > as an option on the opensm command line. > > If you have a request for a command you would like in the console, I > would like to compile a list of these. > > Comments ? As well as a console, I'd like an API for some way for external programs (say a cluster queue manager) to be able to query the SM (or the sm + some helper library) for the following things: * Topology * guid/lid/IPoIB address/switch port mappings * link state Future neat things to do: * An interface to dynamically partition the fabric * Register for notifications for certain events (excessive traffic queueing, or error counts) ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] [RFC] OpenSM Interactive Console
Hi Fab, On Tue, 2005-10-18 at 15:32, Fab Tillier wrote: > > From: Hal Rosenstock [mailto:[EMAIL PROTECTED] > > Sent: Tuesday, October 18, 2005 12:11 PM > > > > If you have a request for a command you would like in the console, I > > would like to compile a list of these. > > I think it would be great to have console commands to dump information from > the > SM - like linear and multicast forwarding tables, service registrations, LID > assignment, etc. Maybe there's a way already to do this interactively, but > I'm > not aware of one. If there is, please ignore me. Yes, that's one area I was contemplating covering: both SM and SA queries. This can currently be done with some IBA defined limitations with IBIS. One can also obtain SM attributes via smpquery from the nodes in the network. -- Hal ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
RE: [openib-general] [RFC] OpenSM Interactive Console
> From: Hal Rosenstock [mailto:[EMAIL PROTECTED] > Sent: Tuesday, October 18, 2005 12:11 PM > > If you have a request for a command you would like in the console, I > would like to compile a list of these. I think it would be great to have console commands to dump information from the SM - like linear and multicast forwarding tables, service registrations, LID assignment, etc. Maybe there's a way already to do this interactively, but I'm not aware of one. If there is, please ignore me. Thanks, - Fab ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
