I botched the e-mail address for networking-discuss in sac_next case.
I'm resending so they get it. Sorry for the duplicate for those of you
who also get PSARC mail.
-- Garrett
Garrett D'Amore - sun microsystems wrote:
Template Version: @(#)sac_nextcase 1.68 02/23/09 SMI
This information is Copyright 2009 Sun Microsystems
1. Introduction
1.1. Project/Component Working Name:
Driver Private Minor Numbers for GLDv3
1.2. Name of Document Author/Supplier:
Author: Garrett D'Amore
1.3 Date of This Document:
24 July, 2009
4. Technical Description
Driver Private Minor Numbers for GLDv3
--------------------------------------
Problem:
If a device driver that is a GLDv3 mac provider desires to export
additional minor nodes (such as for use with the techniques
described in PSARC 2009/380), then the driver must be able to
allocate minor numbers in a way that does not conflict with
GLDv3's use of them.
GLDv3 currently allocates minor numbers as follows: 0 is used for
the special DLPI style 2 node, numbers 1-1000 are used for PPAs
(minor = PPA + 1), and minor numbers from 1001 through MAXMIN32
are used for dynamic stream allocations. (Numbers above MAXMIN32
are effectively useless since minor numbers must be accessible to
32-bit applications.) Note that this latter range is shared
across all device drivers using a global id_space_t.
We are currently in the process of trying to engineer a driver
that is both an InfiniBand HCA, and a GLDv3 ethernet provider. So
the problem of minor number allocation is not merely a theoretical
curiosity.
Solution:
We propose to constrain the range of minor numbers used by GLDv3
to just half that that it currently uses, leaving half of the
minor numbers available for driver private use. We will also
export the value that represents the first minor number available
for driver private as the macro MAC_PRIVATE_MINOR.
This will take the value 0x20000. Note that MAXMIN32 takes the
value 0x3ffff. The construction of MAC_PRIVATE_MINOR is
calculated from MAXMIN32:
#define MAC_PRIVATE_MINOR ((MAXMIN32 + 1) / 2)
This macro will be available in <sys/mac_provider.h>
Imported Interfaces:
+---------------------------------------------------------------------+
| Interface Commitment Level Comments |
+---------------------------------------------------------------------+
| GLDv3 Consolidation Private Specifically the mac |
| provider interfaces. |
| MAXMIN32 Committed <sys/mkdev.h> |
+---------------------------------------------------------------------+
Exported Interfaces:
+---------------------------------------------------------------------+
| Interface Commitment Level Comments |
+---------------------------------------------------------------------+
| MAC_PRIVATE_MINOR Consolidation Private Start of driver private |
| minor numbers. |
+---------------------------------------------------------------------+
Limitations:
This approach allows drivers like our hybrid ethernet/IB driver to
be constructed. However, it does have one drawback, which is that
there will only be about 130,000 possible concurrent streams that
can be open against all GLDv3 devices at a time. This is roughly
half the current number allowed. We don't think this is a
limitation. If it becomes one, there is an alternate approach
that can be used; see below.
The other limitation is that this assumes that mac providers which
need to have separate minor number management are not also
participating in other frameworks which have the same minor number
allocation issue. At the moment we are not aware of any such
cases, but if the situation arises there is an alternate approach
that can be used; see below.
Ramifications for Mac Providers:
Drivers that make use of this feature will have to do some
additional work as well. Most obviously, they will need to
provide an alternate implementation of getinfo(9e). They will
also need to provide alternate implementations of open(9e) and
possibly other functions as well, in the cb_ops and streamtab
structures. The logical place for them to do this will be after
calling mac_init_ops(). mac_init_ops() currently allocates
storage on the heap for these structures, private to the driver,
so such modifications can be made in drivers harmlessly. This
behavior will have to become part of the interface specification
for GLDv3 mac providers.
Of course, current mac providers that don't need this
functionality will not require any modification.
Alternate Approach:
An alternate way of solving this problem would be to give the
ability to Mac providers to manage their own minor number spaces.
This could be triggered by a capability. This heavier weight
solution would be ultimately flexible, but would require more
significant changes to the GLDv3. After discussion with several
other engineers, we decided against using this approach at this
time, because we didn't feel the complexity was needed, or is
likely to be needed for the foreseeable future. One benefit of
this atlernate approach, is that it could be used to provide
per-driver minor number spaces, which could ease pressure on the
minor number spaces, if such pressure is ever found to be a
concern. However, we have a difficult time imagining a system
where more than 100,000 concurrent mac clients would be in use.
6. Resources and Schedule
6.4. Steering Committee requested information
6.4.1. Consolidation C-team Name:
ON
6.5. ARC review type: FastTrack
6.6. ARC Exposure: open
_______________________________________________
networking-discuss mailing list
[email protected]
