SYSTEM ARCHITECTURE COUNCIL
Platform Software ARC
---------------------------------
PSARC Regular Meeting time: Wednesdays 10:00-1:00pm in MPK17-3507.
07-22-2009 MEETING MINUTES
============================================================================
Send CORRECTIONS, additions, deletions to psarc-coord at sun.com.
Co-Chair(s):
Garrett D'Amore: Yes
Tim Marsland: no
ATTENDEES - Members: (6 active members)
Kais Belgaied: Yes
Mark Carlson: no
Richard Matthews: Yes
Darren Moffat: no (on sabbatical)
Sebastien Roy: Yes
Glenn Skinner: Yes
Bill Sommerfeld: no (on sabbatical)
Gary Winiger: Yes (on sabbatical)
STAFF -
Asa Romberger (PM): Yes
ATTENDEES - Interns:
Frank Che no
Charles Debardeleben: no
James Falkner: no (on sabbatical)
Daniel Hain: Yes
Michael Haines: no
Alan Hargreaves: no
Phil Harman: no
Cecilia Hu: no
Wyllys Ingersoll: no
Alec Muffett: no (on sabbatical)
Darren Reed: no
Dean Roehrich no
Ienup Sung: no
Phi Tran Yes
Brian Utterback: no
James Walker Yes
Mark Martin Yes (external)
Don Cragun Yes (external)
Guests:
-- GUESTS --
Colin Zou Yes
Menno Lageman Yes
Jordan Brown Yes
Govinda Tatti Yes
Alan Perry Yes
Scott Carter Yes
Wesley Shao Yes
Not all names are captured. Please send email to Asa.Romberger at Sun.com, if
you attended the meeting and your name is missing from the list.
---------------------------------------------------------------------------
MEETING SUMMARY:
================
AGENDA
10:00-10:10 Open ARC Business (use open dial in above)
10:10-10:55 Open Commitment: Solaris Hotplug Framework
(2008/181)
Submitter: Colin Zou
Owner: Garrett D'Amore
Intern: Phi Tran
Exposure: open
10:55-11:40 Open Inception: Clearview IP Tunneling (2009/373)
Submitter: Sebastien Roy
Owner: Sebastien Roy
Exposure: open
11:45-11:55 Closed ARC Business (use closed dial in above)
===========================================================================
Fast Tracks:
============
Case (Timeout) Exposure Title
2009/374 (07/08/09) open libxmlsec
approved
2009/388 (07/29/09) open VRRP Update
let run
2009/392 (07/20/09) open prctl usage output
approved
2009/394 (07/21/09) open SATA Framework Port Multiplier Support
approved
2009/395 (07/22/09) open Add gnuplot command to Solaris WOS
approved
2009/397 (07/23/09) open GnuPG and friends
approved
2009/398 (07/23/09) open IDMU Support for idmap
approved
2009/401 (07/27/09) open Fibre Channel port link reinitialize
support
approved
2009/406 (07/28/09) open fakeroot
approved
===========================================================================
Name: Solaris Hotplug Framework
Submitter: Colin Zou
Owner: Garrett D'Amore
Intern: Phi Tran
Exposure: open
SUMMARY
=======
The main goal of this project is to provide a generic common
Solaris
hotplug framework, a foundation which can support hotplug
functionality
for any hotpluggable bus. The key features include:
o A state machine based, bus independent hotplug framework
which will
interact with other frameworks (such as PM, FMA and Devfs).
It will
manage the hotplug slot states, interact with bus specific
hotplug
controller modules and configuators (including kernel driven
auto-configuration), deliver hotplug events through the Device
Contract, LDI and Sysevent frameworks, and handle all user
initiated
hotplug operations.
o Well defined, common interfaces to the physical and virtual
hotplug
controller drivers. This will simplify writing a new driver with
hotplug controller functionality, because drivers can
implement their
hotplug requirements by calling into the new framework.
o DDI hotplug interfaces for leaf device drivers to support
features
such as surprise removal, hot replacement, dynamic resource
re-balance etc., and framework notification of any asynchronous
events such as
errors and brute force user operations.
At the same time, the new hotplug framework will be
compatible with
existing DDI compliant leaf device drivers.
o A new resource allocator, which will manage bus resources through
device tree and provides support for dynamic resource re-balance
operations.
o Generic hotplug management interfaces, so that various types of
management applications, including GUIs, can be written to these
interfaces. The project will deliver a new GUI based hotplug tool
and may also modify cfgadm or deliver a new command line
(CLI) tool.
o A hotplug daemon to deliver hotplug events to all hotplug aware
applications and userland modules through Device Contract and
Sysevent frameworks.
o Support for PCI (SHPC based) and PCI Express hotplug
functionality
under the new Solaris Hotplug Framework. In addition, a set
of new
features will be supported such as surprise removal, hot
replacement,
power fault, hotplug FMA, and hotplug operation on an individual
device or function in addition to slot or attachment point.
Please note that the old or proprietary hotplug functionality
will
not be migrated to new hotplug framework.
o Support hotplug under virtual environment like IOV, Ldoms,
Xen etc.
Supporting hotplug for virtualized devices requires a virtual
hotplug
controller driver. These virtual HPC drivers may use different
interfaces or methods for accessing physical/IOV compliant
devices
(v/s para-virtualized (PV) devices).
The project will be implemented and delivered in multiple
phases due to
the priority of some key features & time constraints. For
clarification
of the big picture, all phases are presented here. However, the
scope of ARC review requested for this case is limited to the
first phase. The
full definition of other phases including resources and
schedules are
not known at this time. Subsequent ARC cases will be submitted when
subsequent project phases commence.
o Phase-I:
Support existing standard PCI, Native and ACPI based PCI Express
hotplug functionality under the new Solaris Hotplug Framework
along
with several new key features such as:
- New hotplug DDI interfaces for leaf drivers
- New resource allocator interfaces
- Kernel initiated auto-configuration
- Hotplug events through device contract, LDI and sysevent
frameworks
- Dynamic resource re-balance
- Surprise removal, hot replacement, hotplug FMA
- Modified cfgadm or new userland admin tool (CLI)
o Phase-II:
- Support hotplug under virtual environments such as IOV,
Ldoms, Xen
- Support for a GUI admin tool
Please note that the virtual hotplug support may get delivered as
part of phase-I if required (based on its priority and resource
availability).
o Phase-III:
- Support for Express Card
ISSUES
=======
gw-99 Sorry for the late and parochial issues, but I'm still officially
on sabbatical. And I still seem to be the first in the fine.
gw-1 How does this play with devkit, device allocation, SunRay?
gw-2 This project seems to require administrative audit, yet there
is no mention. See the 20 Questions (and Solaris Audit Policy).
gw-3 What is the method context for the service?
Are there any properties? Is this enabled by the profile?
Is there any reason to ever disable it manually?
gw-4 Root is not a privilege (nor is uid 0 special) why isn't
this authorization driven? Just how are you doing access
control? Access control decisions also require audit.
gw-5 I'm dubious about the Patch binding. Verify with the Solaris
Evaluations and Trusted Extensions project teams that this is
appropriate for a Patch. The manager is Craig Payne.
gw-6 What would be the meaning of "remote clients". INET sockets
should be avoided as they add an attack vector. I'm still
dubious about doors being complex.
gw-7 What is the compelling reason that RBAC must be postponed?
How is the project complete without meeting the Solaris
Policies? Is a PAC waver going to be requested?
gw-99a userland: RBAC is not authentication, in this case it is
authorization based access control.
gw-1 How does this play with devkit, device allocation, SunRay?
All the above are investigated. They are out of scope of this project and
there are no conflicts. Details see following:
- devkit:
It is being ported to Solaris and is similar as HAL regarding as
functionality and interface dependencies. It is a userland utility which is
going to depend on libdevinfo and libsysevent. It gets device information
from libdevinfo and listens to sysevents for device add/remove events. This
project does not break the existing interfaces which devkit will depend on.
I've talked to the devkit team and confirmed this.
- device allocation:
It "manages the ownership of devices". It depends on devfs interfaces. This
project implements hotplug controller drivers and framework which are in
the kernel device driver level which is under devfs. So it does not impacts
device allocation. The userland administer tool introduced by this project
is used to issue commands to kernel drivers to hot add/remove devices
to/from the "system". It is not related with user ownerships. And the tool
will conform to RBAC, so only a user with proper authorizations can issue
commands to hotplug devices to/from the system. This prevents the tool from
conflicting/breaking device allocation.
- SunRay: Sun Ray client is not related. Sun Ray server is a userland
application which can run on Solaris and Linux. It manages remote devices
on Sun Ray clients including hotplug operations. However, Sun Ray server is
an independent userland application which does not interact with Solaris
hotplug framework or device drivers. For example, if a usb keyboard is
plugged to a Sun Ray client, the Sun Ray server gets the event from network
and starts a userland process to deal with the hotplug event. And then a
Sun Ray specific userland usb device driver works for the device. Sun Ray
support is out of scope of this Solaris project.
gw-2 This project seems to require administrative audit, yet there
is no mention. See the 20 Questions (and Solaris Audit Policy).
In scope. See section 3.1.7 and 3.3.8 of "Userland Components of Solaris
Hotplug" document (file: shp-userland.txt).
gw-3 What is the method context for the service?
Are there any properties? Is this enabled by the profile?
Is there any reason to ever disable it manually?
In scope. See section 3.3 of "Userland Components of Solaris Hotplug"
document (file: shp-userland.txt).
gw-4 Root is not a privilege (nor is uid 0 special) why isn't
this authorization driven? Just how are you doing access
control? Access control decisions also require audit.
In scope. See section 3.1.5, 3.2.1, 3.3.1 and 3.3.7 of document
"Userland Components of Solaris Hotplug" (file: shp-userland.txt).
gw-5 I'm dubious about the Patch binding. Verify with the Solaris
Evaluations and Trusted Extensions project teams that this is
appropriate for a Patch. The manager is Craig Payne.
Hotpug project team got comments from Craig's team for this issue:
"The TX implementation of device allocation for hotlug doesn't rely on
cfgadm directly. Instead it relies on devfsadm which relies on HAL. I
can't find any reference in the project materials to devfsadm. That
is the interface we care about since that is where entries get added
to the device allocation files. Do devices managed via the new
hotplug framework still go through devfsadm processing? If so, then
this project seems to be compatible with existing functionality. But
it should be required to verify that by testing with Trusted
Extensions enabled. "
The project team's answer to the above question is:
"Yes, they still go through devfsadm processing."
So it is understood that this project is OK to co-exist with TX and have
a Patch binding.
And the project team will do the tests with Trusted Extensions enabled.
For reference, the following are more investigation results from
hotplug project team on this issue:
There are mainly two parts of this project, one is to implement hotplug
controller drivers and framework in the kernel device driver level;
another is the userland administer tool to issue commands to the kernel
drivers to initiate the operation of hot add/remove device to/from the
system. All these jobs are not directly related with "Solaris Evaluations
and Trusted Extensions project". Our userland administer tool will
conform to RBAC, so that only users with proper authorizations can
perform the device hotplug operations for a system. Given these, we
didn't see any risks that our project would break "Solaris Evaluations
and Trusted Extensions project". And, for a reference of the existing
stuff, cfgadm(1M) is the existing framework which does the similar
functionality. It co-exists with TX for many years.
gw-6 What would be the meaning of "remote clients". INET sockets
should be avoided as they add an attack vector. I'm still
dubious about doors being complex.
The project team turns to door in commitment documents. See section 3.3.2
of "Userland Components of Solaris Hotplug" document (file:
shp-userland.txt).
gw-7 What is the compelling reason that RBAC must be postponed?
How is the project complete without meeting the Solaris
Policies? Is a PAC waver going to be requested?
RBAC will be supported. See section 3.1.5, 3.2.1, 3.3.1 and 3.3.7 of
document
"Userland Components of Solaris Hotplug" (file: shp-userland.txt).
pt-1 Why does the hotplug daemon need to run with all privileges?
The existing software stack for DR and hotplugging is based upon cfgadm(1M),
libcfgadm plugins, and calls to the RCM framework. The diversity of client
scripts and plugins for the RCM framework complicates the issue of trying
to determine the exact privileges needed. Different plugins or scripts may
need different privileges to function properly. And the RCM scripting API
is a public interface. We could not really know exactly what privileges
are required by customer scripts in the field. It would be a long term
maintenance nightmare to try and specify a minimized set of privileges
needed by the RCM daemon to cope with the needs of its client plugins and
scripts. (Especially since any customer could write a new RCM script that
needs a new privilege we didn't think we needed.)
This is the problem as I understand it. It's possible that my understanding
of how privileges and authorizations are inherited by fork()'ed processes is
incorrect. I am not totally certain about it. Because, you see, there was
an effort to investigate and possibly solve this issue, but it was abandoned
before all the investigations were completed. An agreement was reached by
upper management to defund further enhancements of this nature to the
existing
software stack. See the 1-pager about the original investigation, and then
see section 2.3.2 of the agreement that was eventually reached:
http://esp.west/espsw/msw/rm-dr/ProjectStatus/BugCourt/DRCE/docs/lp-1pager.txt
http://esp.west/espsw/msw/rm-dr/ProjectStatus/BugCourt/DRCE/docs/DRtransitionplan.pdf
I don't think our situation is hopeless here because our architecture is
different. The problem with cfgadm as I understand it is that everything
depends upon the user who runs a cfgadm command, and the privileges that
are then inherited from there by all other parts of the stack. In our
architecture we have separate address spaces and inserted a controlled door
interface between those address spaces, so that we don't have the same kind
of problem about inherited privileges.
For example, if a user runs cfgadm(1M), it links a libcfgadm plugin into
that same cfgadm(1M) address space. That libcfgadm plugin then calls the
librcm interface to communicate with RCM. The librcm library will fork()
a new process to run the RCM daemon on demand for each operation. First,
librcm checks that the user is running as 'root' before it tries to fork().
Second, if any privileges were dropped before the fork(), I think it would
limit what the RCM daemon could then do when it runs.
In our architecture, the hotplug(1M) CLI uses libhotplug.so to communicate
through a private door interface to the hotplugd(1M) daemon. No privileges
are required, except certain RBAC authorizations are required. The CLI and
the library perform the RBAC checks before making a door call to the daemon.
And the daemon makes the same RBAC checks using the credentials of each
incoming door call. The only way to make the hotplugd(1M) daemon do
anything
is through that controlled door interface, which is protected by RBAC. And
then the hotplugd(1M) will only do very specific things in response to those
door calls. In some cases, the hotplugd(1M) responds to the door call by
doing certain operations that require RCM interactions. But it has full
privileges so that the RCM daemon will inherit them and function properly
in those cases.
So that's why I tried to say we're keeping with the spirit of the principle
of least privilege in our architecture. The old model required you to run
as root with full privileges in order to do a cfgadm state change operation.
Our new model let's you grant a specific RBAC authorization to an ordinary
user permitting them to perform hotplug operations, without giving them the
full privileges that cfgadm used to require.
Commitment issues 07/22/2009
gw-8 What's the requirement for the read authorization?
What data is being protected from viewing?
TCR: Delete read authorization or justify
gw-9 I find the description of where (non-smf) authorizations are being
checked confusing: authorizations are to be checked at the point
of access control enforcement. Presumably this is in the hotplug
service (hotplugd).
Where and how is access control being enforced?
AI: Review doc, include clear description of audit and
authorization
checks (includes gw-10)
gw-10 I find the description of where (non-smf) Solaris Audit take place
confusing: Audit is to take place (with the calling user's Audit
context) at the policy enforcement point. Presumably this is
in the
hotplug service (hotplugd). Audit records are to conform with the
Solaris Audit Policy through the use of PSARC/2000/517 Thread-safe
audit API and with a conforming contract as noted in PSARC/2003/397
Contracted audit interfaces for open source.
Where and how is Audit being done?
gw-11 Nit: solaris.smf.manage.hotplug probably should follow the
example found in the SMF policy for the general/framework
property.
Policy section 3.3.1
gw-12 All the interfaces seem to be Consolidation Private. How
does this permit management of the authorizations? I'd
expect some of the interfaces to be Committed, some to be
Project Private and maybe some Consolidation Private.
Review interface table
gw-13 hotplug(1m) Evolving -> Committed?
Some mention should be made of the Rights Profile (i.e., interface)
required to successfully manage hotpluging.
Why are there --long options?
see cfgadm as example
ged-01 In shp-userland.txt, why are you delivering the libhotplug headers.
Private headers need not be delivered to end-users. Especially I
would not deliver the _impl.h, since its internal use only
within ON.
Okay, not delivering the header files sounds right. At least not
delivering the libhotplug_impl.h. The libhotplug interfaces are
all "Consolidation Private" in phase 1, at least. I think we might
increase the interface stability in phase 2. So I'm not entirely
sure yet about libhotplug.h. Let's defer a final decision until the
meeting, so no updates to the materials yet.
ged-02 It seems like changestate is a bit awkward/counterintuitive to admin
users (at least from my read).
I'd recommend new commands instead:
"enable", "disable", "power_off", "power_on" for connectors
for ports, "attach" and "detach" seem enough... is there a need for
administrators to enter the initialize state explicitly? (That
doesn't make a lot of sense to me.)
ged-03 In shp-overview.pdf, it seems like you could predefine some more
values
for DDI_HP_CN_TYPE... I'd like to see types for USB, PCMCIA,
CARDBUS,
FIREWIRE, SCSI, FC, IB, and SDCARD. These are the types of
devices in
Solaris that have some form of hotplug support via cfgadm.
(They might
not be implemented, but reserving the name space still seems like a
good idea to me.)
The team's response is that we do not yet have enough information on
which buses will be supported by the new hotplug framework, or how
exactly that support would be implemented. It might be premature to
reserve such namespaces. Also, there is a concern we might reserve
namespaces incorrectly if we try doing it now.
One example cited within the team was with USB. Would we want to
reserve a single value for USB right now? What if we implement the
USB support and instead realize we want separate values for each
version of USB?
seb-01 What stability level are the options that the -o argument to
the hotplug command operates on? They're not documented in
the man page. How are users expected to know their semantics?
VOTE
====
Approve -
Deny -
Abstain -
Not Participating (NP) -
THE NEXT STEP
=============
Spec and AI resolution needed for vote, work with sponsor
===========================================================================
Name: Clearview IP Tunneling
Submitter: Sebastien Roy
Owner: Sebastien Roy
Exposure: open
SUMMARY
=======
This project is a redesign and reimplementation of the IP
tunneling functionality in Solaris. It delivers a GLDv3 driver
(iptun) that implements IP tunnel links atop which IP interfaces
can be plumbed. Tunnel links are managed using new dladm(1M)
subroutines (implemented using new libdladm functions), and IP
interfaces above tunnel links are created using ifconfig(1M) as
with any other type of IP interface.
It replaces the old STREAMS-based IP tunneling implementation
which depended on the ifconfig(1M) command to plumb tunneling
modules (tun, atun, and 6to4tun) atop of /dev/ip or /dev/ip6.
With this project, tunnel links gain functionality common to
other links, including link vanity naming (introduced by
Clearview in PSARC 2006/499), link-layer observability using
traditional observability tools such as Wireshark and snoop(1M),
and the assignment of tunnel links to exclusive stack non-global
zones.
ISSUES
======
kb-00 It seems to me that we've got two distinct cases here:
(1) datalinks control from non global zones
(2) Clearview IP Tunneling
case (1) brings sufficiently complex architectural changes and is
independant from (2) to warrant a separate case.
Keeping these together also increases the risk of making (1)'s
choices taylored to (2)'s needs.
split them
kb-01 How does information float from the lower IP to the upper one across
the tun mac provider
- support of LSO
- MTU
I'm thinking of the cases where MTU for example changes because
the outer header for a tunnel get re-routed through an interface
with a different MTU
Spec update
kb-02 For IPsec tunnels created from the non global zone. I am not
clear on
how the global zone sees the keys for encrypting and/or
authenticating
the payload.
THE NEXT STEP
=============
commitment review
