On Fri, 2009-09-04 at 13:53 -0700, Peter Memishian wrote:
> > Another belated opinion for PSARC review. Please review by 09/11/2009.
>
> Is there supposed to be some content?
Yes, I included the wrong file (blush). The correct file is now
attached.
-Seb
-------------- next part --------------
sun
microsystems Systems Architecture Committee
_________________________________________________________________
Subject: Project Clearview: IPMP Rearchitecture
Submitted by: Peter Memishian
File: PSARC/2007/272/opinion.ms
Date: November 19th, 2008
Committee: Kais Belgaied, James Carlson, Glenn Skinner.
Abstain: Gary Winiger. (opinion written by
Sebastien Roy)
Product Approval Committee:
Solaris PAC
solaris-pac-opinion at sun.com
1. Summary
This is one of a series of projects under the PSARC/2005/132
umbrella case, "Clearview: Network Interface Coherence".
This project rearchitects the existing Solaris IP Multipath-
ing (IPMP) technology so that it can work transparently with
all IP-based applications, allowing it to be deployed much
more widely by customers. As a side effect, it also enables
core technologies such as DHCP to work seamlessly with IPMP.
It also provides an improved administrative model and new
diagnostic facilities which should greatly ease troub-
leshooting in IPMP environments -- and reduce our support
costs. Finally, it massively simplifies the IPMP kernel
implementation, shrinking it by over 35% (3000 lines less
code) and isolating the majority of the remaining code beh-
ing a well-defined internal API.
2. Decision & Precedence Information
The project is approved as specified in references [1-6].
The project may be delivered in a Minor release of Solaris
as part of the ON consolidation.
3. Interfaces
PSARC/2007/272 Copyright 2007 Sun Microsystems
- 2 -
The project exports the following interfaces.
____________________________________________________________________________
| Interfaces Exported |
|__________________________________|__________________|____________________|
|Interface | Classification | Comments |
|__________________________________|__________________|____________________|
|IPMP IP Interface | Committed | [1] section 3.1 |
|outbount load spreading behavior | Volatile | [1] section 3.2 |
|source address selection behavior | Volatile | [1] section 3.3 |
|ifconfig "ipmp" subcommand | Committed | [1] section 4.1.5 |
|ifconfig "ipmp" subcommand | Committed | [1] section 4.1.5 |
|"ipmp" in /etc/hostname[6].* | Committed | [1] section 4.3 |
|networking commands on IPMP and | Committed | [1] sections 4.1, |
|underlying IP interfaces | | 4.4, 4.5, 4.8-4.10|
|IPMP packet monitoring semantics | Committed | [1] section 4.11 |
|IPMP packet filtering semantics | Committed | [1] section 4.12 |
|DHCP for IPMP data/test addresses | Committed | [1] section 4.13 |
|kstats for IPMP IP interfaces | Committed[4] | [1] section 4.8 |
|MIBII stats for IPMP IP interfaces| Uncommitted | [1] section 5.18 |
|IPv6 link-local IPMP interaction | Committed | [1] section 4.6 |
|IPMP bring-up at boot | Project Private | [1] section 4.3 |
| | | |
|/usr/sbin/ipmpstat | Committed | [1] section 4.2 |
|/sbin/ipmpstat alternate location | Volatile | [1] section 4.3.4 |
|/sbin/in.mpathd alternate location| Volatile | [1] section 4.3.4 |
|ipmpstat output modes | Committed | [1] section 4.2 |
|(-a, -g, -i, -p, -t) | | |
|ipmpstat normal output format | Not-an-Interface| [1] section 4.2 |
|ipmpstat -P output format | Committed | [1] section 4.2 |
| | | |
|SIOCG[L]IFCONF and SIOCG[L]IFNUM | Committed | [1] section 5.1 |
|IPMP interaction | | |
|LIFC_UNDER_IPMP | Committed | [1] section 5.1 |
|SIOC[GS]LIFFLAGS IPMP interaction | Committed | [1] section 5.2 |
|IFF_IPMP | Committed | [1] section 5.3 |
|IFF_RUNNING on IPMP IP interfaces | Committed | [1] section 5.3 |
|Visibility of IPMP and underlying | Committed | [1] section 5.4 |
|IP interfaces via PF_ROUTE | | |
|SOL_ROUTE socket level | Committed | [1] section 5.4.2 |
|RT_AWARE / RTAW_* socket option | Committed | [1] section 5.4.2 |
|Assorted "set" SIOC* and PF_ROUTE | Volatile | [1] section 5.* |
|ops on underlying IP interfaces | | |
|SIOC*ARP IPMP behavior | Committed | [1] section 5.11 |
|SIOC*ND IPMP behavior | Committed | [1] section 5.12 |
|SIOC[GS]LIFZONE/DLDIOC_SETZID | Cons. Private | [1] section 5.13 |
|SIOC*LIF*SRC* IPMP interaction | Volatile | [1] section 5.14 |
|IPMP data address multicast API | Committed | [1] section 5.15 |
|behavior and semantics | | |
|SIOCGLIFBINDING | Project Private | [1] section 5.17 |
|SIOCGLIFGROUPINFO | Project Private | [1] section 5.17 |
|lifr_binding lifreq member | Project Private | [1] section 5.17 |
|struct lifgroupinfo | Project Private | [1] section 5.17 |
|EXPER_IP_AND_TESTHIDDEN MIB level | Cons. Private | [1] section 5.18 |
|__________________________________|__________________|____________________|
PSARC/2007/272 Copyright 2007 Sun Microsystems
- 3 -
____________________________________________________________________________
| Interfaces Exported |
|__________________________________|__________________|____________________|
|Interface | Classification | Comments |
|__________________________________|__________________|____________________|
|dlpistub kernel module | Cons. Private | [1] section 5.20 |
|/dev/ipmpstub | Project Private | [1] section 5.20 |
| | | |
|libipmp APIs | Contracted Cons.| [1] section 5.21 |
|in.mpathd/libipmp communication | Project Private | [1] section 5.21 |
|/usr/include/ipmp_admin.h | Contracted Cons.| [1] section 5.21 |
|IPMP async events | Contracted Cons.| [1] section 5.22 |
|if_indextoname() IPMP behavior | Committed | [1] section 5.23.1|
|if_nametoindex() IPMP behavior | Committed | [1] section 5.23.1|
|if_nameindex() IPMP behavior | Committed | [1] section 5.23.2|
|ifaddrlist() enhancements | Cons. Private | [1] section 5.23.3|
|ifaddrlistx(), ifaddrlistx_free() | Cons. Private | [1] section 5.23.4|
|sockaddrcmp() | Cons. Private | [1] section 5.23.5|
|__________________________________|__________________|____________________|
The project imports the following interfaces.
___________________________________________________________________
| Interfaces Imported |
|__________________________|_________________|____________________|
|Interface | Classification | Comments |
|__________________________|_________________|____________________|
|vni device driver | Cons. Private | [1] section 5.20 |
|NIC kstats | Committed | [1] section 4.8 |
|sysevent API | Committed | For PSARC/2002/137|
|IP module IPSQ framework | Project Private| [2] question 14 |
|ARP/IP message-passing API| Cons. Private | [2] question 14 |
|modhash kernel API | Cons. Private | Impl. artifact |
|/etc/hostname[6].* | Committed | [1] section 4.3 |
|__________________________|_________________|____________________|
4. Opinion
4.1. IPMP is a Network Layer Mechanism
During the case review, concern was raised that an IPMP
group represented by a virtual IP interface could be confus-
ing to users. Specifically, IP interfaces that have no
one-to-on relationship with an underlying link require spe-
cial handling by observability tools, and thus require addi-
tional architecture such as PSARC/2006/475.
The discussion did not result in any technical change nor
advise. IPMP is by definition a network layer mechanism,
and thus cannot be represented at the link layer.
PSARC/2007/272 Copyright 2007 Sun Microsystems
- 4 -
4.2. Reverse ARP Autoconfiguration and IPMP
Section 4.1.11 of [1] states that the ifconfig auto-revarp
subcommand will fail on an IPMP interface. The group inter-
face does not exist as a DLPI device, and DLPI is needed for
ifconfig to send and receive ARP packets.
One PSARC member noted that while it is true that the group
interface does not exist as a DLPI device, implementing
auto-revarp should still be possible by interacting directly
with one of the underlying interface's DLPI device. All
agreed that implementing such a mechanism is likely not
worth the effort.
4.3. Incompatible Change to Networking MIB
Section 5.18 of [1] describes that data associated with test
addresses on underlying interfaces will be invisible to con-
sumers of the Networking MIB unless the new
EXPER_IP_AND_TESTHIDDEN MIB II level is specified in the
request, under the assumption that only in.mpathd should be
concerned with those details.
The committee noted that this constituted a slight regres-
sion in functionality, as SNMP applications will no longer
have observability into IPMP configuration. This resulted
in advice to management described in section 6.
5. Minority Opinion(s)
None
6. Advisory Information
This project's management should fund a project to add SNMP
observability into IPMP configuration, perhaps using a new
private MIB. See section 4.3 for details.
7. Appendices
7.1. Appendix A: Technical Changes Required
None
7.2. Appendix B: Technical Changes Advised
None
7.3. Appendix C: Reference Material
Unless stated otherwise, path names are relative to the case
directory PSARC/2007/272/final.materials/.
1. High-Level Design Document
PSARC/2007/272 Copyright 2007 Sun Microsystems
- 5 -
File: ipmp-highlevel-design.pdf
2. PSARC 20 Questions
File: ipmp-20q.txt
3. Specification for ifaddrlistx() and
ifaddrlistx_free()
File: ifaddrlistx.txt
4. IPMP Async Events Specification
File: ipmp-async.txt
5. libipmp Specification
File: libipmp.txt
6. Draft Manual Pages
File: manpages/index.html
PSARC/2007/272 Copyright 2007 Sun Microsystems
