I am sponsoring the following fast track for Bill Gorrell. The release
binding
is minor.
Template Version: @(#)onepager.txt 1.35 07/11/07 SMI
Copyright 2007 Sun Microsystems
1. Introduction
1.1. Project/Component Working Name:
SAM-QFS Large Host Table
1.2. Name of Document Author/Supplier:
William Gorrell
1.3. Date of This Document:
02/29/2008
1.3.1. Date this project was conceived:
05/10/2007
1.4. Name of Major Document Customer(s)/Consumer(s):
1.4.1. The PAC or CPT you expect to review your project:
NWS PAC
1.4.2. The ARC(s) you expect to review your project:
PSARC
1.4.3. The Director/VP who is "Sponsoring" this project:
Karen.Jourdenais at sun.com
1.4.4. The name of your business unit:
SDASH
1.5. Email Aliases:
1.5.1. Responsible Manager:
Ted.Pogue at sun.com
1.5.2. Responsible Engineer:
William.Gorrell at sun.com
1.5.3. Marketing Manager:
Margaret.Hamburger at sun.com
1.5.4. Interest List:
samqfs-psarc-interest at sun.com
2. Project Summary
2.1. Project Description:
See supplied functional specification document
"largehosttable.6555811.txt" section "FEATURE DESCRIPTION".
2.2. Risks and Assumptions:
None.
3. Business Summary
3.1. Problem Area:
The SAM-QFS HPC initiative requires that a larger number of clients
be supported.
3.2. Market/Requester:
The SAM-QFS HPC initiative requires that a larger number of clients
be supported.
3.3. Business Justification:
The SAM-QFS HPC initiative requires that a larger number of clients
be supported.
3.4. Competitive Analysis:
The SAM-QFS HPC initiative requires that a larger number of clients
be supported.
3.5. Opportunity Window/Exposure:
FY2009
3.6. How will you know when you are done?:
When the HPC target number of SAM-QFS clients can be configured.
4. Technical Description:
4.1. Details:
See supplied functional specification document
"largehosttable.6555811.txt" section "FUNCTIONAL REQUIREMENTS".
4.2. Bug/RFE Number(s):
6555811: RFE: Increase the size of the client host table.
This RFE is currently targeted for the next release of SAM-QFS
(FY09).
All documentation references to release "x.x" should be interpreted
as that release.
4.3. In Scope:
See supplied functional specification document
"largehosttable.6555811.txt" section "SCOPE OF THE PROJECT".
4.4. Out of Scope:
See supplied functional specification document
"largehosttable.6555811.txt" section "SCOPE OF THE PROJECT".
4.5. Interfaces:
See supplied functional specification document
"largehosttable.6555811.txt" section "FUNCTIONAL REQUIREMENTS".
4.6. Doc Impact:
Man pages do not exist for the affected library interfaces.
The SAM-QFS administration manual will be updated to describe the
extended host table size, and limitations of mounting file systems
with a large host table on hosts running a legacy release of
SAM-QFS.
4.7. Admin/Config Impact:
The administrator will experience the ability to configure a larger
number of SAM-QFS clients on a SAM-QFS file system.
4.8. HA Impact:
A larger number of supported clients supports improved cluster
availability because of the larger number of available failover
host candidates.
4.9. I18N/L10N Impact:
The error messages that describe a client host table size
incompatibility condition will require investigation.
4.10. Packaging & Delivery:
Providing a larger client host table will not specifically impact
package install/upgrade procedures.
Associated packages are: SUNWsamfsu, SUNWsamfsr, SUNWqfsu, SUNWqfsr
4.11. Security Impact:
None. All client host table operations require administrative
authority.
4.12. Dependencies:
This feature requires that SAM-QFS voluntary and involuntary
failover be reliable prior to release.
5. Reference Documents:
See supplied functional specification document.
Also reference the full SAM-QFS 5.0 PSARC case 2007/588 at:
http://sac.eng/arc/PSARC/2007/588
6. Resources and Schedule:
6.1. Projected Availability:
Available with the first release of the SAM-QFS HPC product.
6.2. Cost of Effort:
Unspecified.
6.3. Cost of Capital Resources:
Unspecified.
6.4. Product Approval Committee requested information:
6.4.1. Consolidation or Component Name:
Available with the first release of the SAM-QFS HPC product.
6.4.3. Type of CPT Review and Approval expected:
FastTrack
6.4.4. Project Boundary Conditions:
Unknown.
6.4.5. Is this a necessary project for OEM agreements:
No.
6.4.6. Notes:
Intentionally left blank.
6.4.7. Target RTI Date/Release:
RTI in time for the first release of the SAM-QFS HPC
product.
6.4.8. Target Code Design Review Date:
Review in time for the first release of the SAM-QFS HPC
product.
6.4.9. Update approval addition:
No.
6.5. ARC review type:
FastTrack
6.6. ARC Exposure:
open
6.6.1. Rationale:
N/A
7. Prototype Availability:
7.1. Prototype Availability:
Q4 2008
7.2. Prototype Cost:
Unspecified.
Functional Spec
===============
SAM-QFS x.x Planning - Functional Specification Document
========================================================
Updated: 03/27/2008
FEATURE DESCRIPTION:
-------------------
The current limit of 128 clients in a shared QFS cluster needs to be
increased. Independent of the other issues related to a large
number of clients, the current infrastructure for storing the client
host table needs to be modified to accomodate a much larger number of
clients (the short-term goal is 1024 clients). The current allocated
space for the on-disk client hosts table is 16KB. This feature
provides the ability to add new client hosts to a shared QFS cluster
until the new 2MB of allocated space is exhausted.
NOTE: Each client host table entry can be of arbitrary size. The
primary influence on the size of each entry is it's host name.
From an administrative standpoint, if we assume that the largest
practical host name consumes 1/4 KB, then the maximum number of
entries in a 2MB host table is on the order of 8K entries.
Small client host tables will continue to be supported for existing
file systems. sammkfs will allocate a larger (2MB) client host table
on-disk for new SAM-QFS file systems. For a SAM-QFS file system that
contains a 16KB client host table, that client host table will
automatically be relocated into a new 2MB client host table when more
than 16KB is necessary to hold the supplied client host table data.
A file system with a 2MB client host table will not be mountable
on a system running a legacy release of SAM-QFS, except under
special circumstances (see section "FUNCTIONAL REQUIREMENTS").
6555811: RFE: Increase the size of the client host table.
SCOPE OF THE PROJECT:
--------------------
- The sammkfs command and the libfscmd library will be modified to allow the
creation of large host tables.
- The new on disk host file will be a 2M direct map file created at sammkfs
time.
- The behaviour of existing small host tables will not change. A
mechanism to
convert existing small host tables to large ones will be provided.
- In general, configuration and observation of large numbers of SAM-QFS
clients via the affected interfaces.
All other issues related to large numbers of SAM-QFS clients are outside
the scope of this feature.
FUNCTIONAL REQUIREMENTS:
-----------------------
The existing library interfaces (see section 4.5 Interfaces) that
are used to retrieve and update the client host table already include
a buffer size embedded within their parameters. This allows SAM-QFS
infrastructure to determine whether or not the supplied size is
sufficient for retrieving or updating the client host table.
However, the behavior of the existing library interfaces will be
updated to recognize a buffer size incompatibility, and to return the
error code EMSGSIZE ("Message size too long") when that condition is
detected. In case of a size incompatibility error, the correct client
host table size is embedded within the returned parameters.
The parameters and format of existing library interfaces will remain
unchanged. An error return code, EMSGSIZE ("Message size too long"),
will be added to the set of possible return codes for those library
interfaces. Affected library interfaces are:
int sam_gethost(const char *fs, int len, char *tab)
int sam_sethost(const char *fs, int newserver, int len, const char *tab)
The above library interfaces are included in the SAM-QFS library
libsam. Those interfaces are unpublished/internal/private.
Certain SAM-QFS software components account for the error condition
where a corrupted SAM-QFS file system can't be mounted. To retrieve
and/or update the client host table those software components open
and read/write the client host table directly from/to the appropriate
metadata device. Those components will be updated to recognize, read
and write both the small and large client host table formats. Affected
components are:
sammkfs
samfsck
samfsconfig
sharefsd
Sun Cluster Agent
The small client host table of an existing file system will be
automatically converted (i.e. moved) to a newly-allocated large
client host table only when the client host table requires more
than 16KB of storage space. Available space limitations may require
that the new large client host table must be allocated on a
non-ordinal-0 metadata device. The SAM-QFS host table management
infrastructure, which currently assumes that the client host table
always resides on the ordinal 0 metadata device, will be updated
so that it can manage a client host table that resides on a
metadata device of any available file system ordinal.
Once a SAM-QFS file system contains a large client host table (2MB),
the ability to mount that file system on a host that is running a
legacy release of SAM-QFS becomes limited. Legacy releases of SAM-QFS
can only manage a small (16KB) client host table. Therefore, a file
system that contains a client host table with more than 16KB of data
cannot be mounted on a host running a legacy release of SAM-QFS.
However, if a filesystem that supports a 2MB client host table, but
contains only 16KB (or less) of client host table data, then it can
be mounted on a host running a legacy release of SAM-QFS.
This feature will toggle an option bit in the SAM-QFS file system
superblock to indicate whether or not the client host table contains
more than 16KB of data. That superblock option field controls
the ability to mount newer file system versions on hosts running
a legacy release of SAM-QFS.
DELIVERABLES:
------------
- Restructure the in core host table.
- Allocate the on disk host table as a 2M direct map file at sammkfs time.
- The existing sam_gethost and sam_sethost interfaces will continue to
work for existing host tables and new host tables that fit within the
current space limitation of 16k.
- The behaviour of the existing interface will change when reading or
writing
a new large host table. If the caller of sam_gethost or sam_sethost
provides
a buffer that is too small then EMSGSIZE will be returned but the
needed size
will be returned in the size element of the argument structure.
FUNCTIONAL DEPENDENCIES:
-----------------------
This requires that voluntary and involuntary failover be reliable prior to
integration of this feature.
