I'm sponsoring the following Fact-Track, time out Jan 27th 2010.
-------------------------------------------------------------------------------
The link-editors provide a number of means for selecting alternative
implementations at runtime. Reserved tokens can be used with runpaths
(ie. $PLATFORM), and capabilities can be recorded within dynamic objects
(ie. ELF{32|64}_Cap - $HWCAP). On SPARC, the model for using these
techniques is through numerous "psr" libraries, provided by a network of
symlinked libraries under /platform.
This "model" is rather complex, hard to maintain (at least from a packaging
point of view), and can, when the optimized functions are not heavily used,
actually lead to a runtime overhead because of searching for "psr" candidates.
The compiler engineers have also asked for a means of selecting between a
family of functions contained within the same dynamic object. Each family
member is compiled differently to use various capabilities, where these
capabilities can be provided by different systems.
To reduce "psr" overhead, and address the compiler requirements, the
capabilities that can be recorded within dynamic objects have been extended,
and an ELF infrastructure has been created that associates individual symbols
with their required capabilities. With this infrastructure, a family of
functions, each with different capability requirements, can be contained
within a single dynamic object. The runtime linker can use this infrastructure
to analyze a capabilities family at runtime, and select the best function to
bind to for the underlying system.
In other words, we can take the contents of all our libc_psr libraries and
put them in one libc.so.1. Then we can then throw away the /platform
symlinked psr libraries.
The change in the link-editors is being tracked under:
6918143 symbol capabilities
-------------------------------------------------------------------------------
Presently, the capabilities that can be defined within an object are derived
from the HWCAP aux vector flags (see AV_SPARC in sys/auxv_SPARC.h and AV_386
in sys/auxv_386.h). However, these capabilities are insufficient to represent
the present "psr" incantations. The capability tags have been expanded to
provide:
i. a platform name (one or more can be applied to a capabilities group)
ii. a machine hardware name ( " " " " " ")
iii. a capabilities identifier name
The latter identifier (iii.) does not add to the capabilities definitions, but
provides a means of symbolically naming a capabilities group (this comes in
to play with the new ld(1) -z symbolcap option ... more later).
With these identifiers, we can now, for example, identify a capabilities
group that would be applicable for an "opl" system:
Symbol Capabilities:
index tag value
[4] CA_SUNW_ID sun4u-opl
[5] CA_SUNW_PLAT SUNW,SPARC-Enterprise
or, even more fun, one of the existing hwcap variants (this is "psr" variant
that gets loop-back mounted on a "psr" mount point):
Symbol Capabilities:
index tag value
[7] CA_SUNW_ID sun4u-us3-hwcap1
[8] CA_SUNW_PLAT SUNW,A70
[9] CA_SUNW_PLAT SUNW,Netra-210
[10] CA_SUNW_PLAT SUNW,Netra-240
[11] CA_SUNW_PLAT SUNW,Netra-440
[12] CA_SUNW_PLAT SUNW,Netra-CP3010
[13] CA_SUNW_PLAT SUNW,Netra-T12
[14] CA_SUNW_PLAT SUNW,Netra-T4
[15] CA_SUNW_PLAT SUNW,Sun-Blade-1000
[16] CA_SUNW_PLAT SUNW,Sun-Blade-1500
[17] CA_SUNW_PLAT SUNW,Sun-Blade-2500
[18] CA_SUNW_PLAT SUNW,Sun-Fire-15000
[19] CA_SUNW_PLAT SUNW,Sun-Fire-280R
[20] CA_SUNW_PLAT SUNW,Sun-Fire-480R
[21] CA_SUNW_PLAT SUNW,Sun-Fire-880
[22] CA_SUNW_PLAT SUNW,Sun-Fire-V210
[23] CA_SUNW_PLAT SUNW,Sun-Fire-V215
[24] CA_SUNW_PLAT SUNW,Sun-Fire-V240
[25] CA_SUNW_PLAT SUNW,Sun-Fire-V245
[26] CA_SUNW_PLAT SUNW,Sun-Fire-V250
[27] CA_SUNW_PLAT SUNW,Sun-Fire-V440
[28] CA_SUNW_PLAT SUNW,Sun-Fire-V445
[29] CA_SUNW_PLAT SUNW,Sun-Fire-V490
[30] CA_SUNW_PLAT SUNW,Sun-Fire-V890
[31] CA_SUNW_PLAT SUNW,Sun-Fire
[32] CA_SUNW_MACH sun4u
The present Elf{32|64}_Cap data structure is captured in a SHT_SUNW_CAP
section, .SUNW_cap. This data structure consists of one or more CA_SUNW_*
tags, and a terminating CA_SUNW_NULL (0) tag. This data structure, which
starts at .SUNW_cap[0] defines the capabilities of the object. This
"convention" will remain unchanged.
This project allows there to be multiple CA_SUNW_NULL terminated capabilities
groups. Again, .SUNW_cap[0] defines the capabilities of the object, but any
follow on groups allow for the definition of symbol capabilities.
A new SHT_SUNW_capinfo section is provided. This section is an array that
parallels the symbol table, and allows individual symbols to be associated
with capability groups. elfdump(1)'s existing -H option displays capabilities
information, and when symbol capabilities are detected, will display the
capabilities and the associate symbols:
% elfdump -H /lib/libc.so.1
Capabilities Section: .SUNW_cap
Symbol Capabilities:
index tag value
[1] CA_SUNW_ID sun4u
[2] CA_SUNW_MACH sun4u
Symbols:
index value size type bind oth ver shndx name
[1] 0x000f05e0 0x000000bc FUNC LOCL D 0 .text memmove%sun4u
[7] 0x000f1aac 0x00001b28 FUNC LOCL D 0 .text memcmp%sun4u
[11] 0x000f069c 0x00001280 FUNC LOCL D 0 .text memcpy%sun4u
[17] 0x000f1920 0x0000018c FUNC LOCL D 0 .text memset%sun4u
Symbol Capabilities:
index tag value
[4] CA_SUNW_ID sun4u-opl
[5] CA_SUNW_PLAT SUNW,SPARC-Enterprise
Symbols:
index value size type bind oth ver shndx name
[2] 0x000f3600 0x00000310 FUNC LOCL D 0 .text
memmove%sun4u-opl
[8] 0x000f424c 0x00000120 FUNC LOCL D 0 .text memcmp%sun4u-opl
[12] 0x000f3940 0x0000076c FUNC LOCL D 0 .text memcpy%sun4u-opl
[18] 0x000f40c0 0x0000018c FUNC LOCL D 0 .text memset%sun4u-opl
....
Each capabilities symbol is local. This prevents direct access to these symbols
from any external reference. External references are directed to a global lead
symbol, i.e., memmove(), memcpy(), etc. These lead symbols provide the generic
instance of the function, one that is not compiled to require any capabilities.
A new SHT_SUNW_capchain section is provided within dynamic objects. This
structure contains one of more collections of capabilities families, each
lead by the global lead symbol.
When the runtime linker binds to a capabilities lead symbol, i.e., memcpy(),
the fact that this symbol is the lead of a capabilities family is detected
from the SHT_SUNW_capinfo section information. From this information the
associated SHT_SUNW_capchain family is determined, and from this information
ld.so.1 can look at each family member to determine which instance is the
best candidate for the present system. elfdump(1) -H also displays these
capability families:
....
Capabilities Chain Section: .SUNW_capchain
Capabilities family: memset
chainndx symndx name
23 [565] memset
24 [17] memset%sun4u
25 [18] memset%sun4u-opl
26 [19] memset%sun4u-us3-hwcap1
27 [20] memset%sun4u-us3-hwcap2
28 [21] memset%sun4v-hwcap1
Capabilities family: _memcpy
chainndx symndx name
15 [1771] memcpy
16 [11] memcpy%sun4u
17 [12] memcpy%sun4u-opl
18 [13] memcpy%sun4u-us3-hwcap1
19 [14] memcpy%sun4u-us3-hwcap2
20 [15] memcpy%sun4v-hwcap1
21 [16] memcpy%sun4v-hwcap2
This SHT_SUNW_capchain array can be cached within ld.so.1, and rewritten
once a family member is chosen. Hence, subsequent lookups of the same symbol
will be directed to the best family instance without having to reinspect
all the family members.
It is hoped that later versions of our compilers can create the SHT_SUNW_cap
and SHT_SUNW_capinfo infrastructure as outlined above. Multiple versions of
the same function can then be created within relocatable objects, and the
link-editor will combine these, create the necessary SHT_SUNW_capchain entries,
and provide for a dynamic symbol capabilities runtime environment.
In the mean time, the link-editor has been updated to provide a means of
conventing an object capabilities relocatable object into a symbol capabilities
relocatable object. This is the means by which the above examples have been
produced, and how an OSNet environment has been rebuilt to convent libc,
libmd and libsoftcrypto to use symbol capabilities.
ld(1) has a new -z symbolcap option. It is already possible for ld(1) to
take a relocatable object, and add object capabilities via a mapfile. ld(1)
can now take this object capabilities relocatable object, and convert it to
a symbol capabilities object. For example, the steps used to create a
symbol capabilities version of memcpy() might be:
% cc -c <options to trigger opl specific code> memcpy.s
% cat mapfile
idcap = sun4u-opl;
platcap = SUNW,SPARC-Enterprise
% ld -r -o objcap.o -Mmapfile -Breduce memcpy.o
% elfdump -H objcap.o
Capabilities Section: .SUNW_cap
Object Capabilities:
index tag value
[0] CA_SUNW_ID sun4u-opl
[1] CA_SUNW_PLAT SUNW,SPARC-Enterprise
% ld -r -o symcap.o -z symbolcap objcap.o
% elfdump -H symcap.o
Capabilities Section: .SUNW_cap
Symbol Capabilities:
index tag value
[1] CA_SUNW_ID sun4u-opl
[2] CA_SUNW_PLAT SUNW,SPARC-Enterprise
Symbols:
index value size type bind oth ver shndx name
[13] 0x00000000 0x00000310 FUNC LOCL D 0 .text memcpy%sun4u-opl
% elfdump -s symcap.o | fgrep memcpy
[15] 0x00000340 0x0000076c FUNC LOCL D 0 .text memcpy%sun4u-opl
[19] 0x00000000 0x00000000 FUNC GLOB D 0 UNDEF memcpy
In a nutshell, ld(1) has:
i. converted the SHT_SUNW_cap object capabilities definition to a
symbol capabilities definition,
ii. created a LOCAL symbol to associate with memcpy(), using the
capabilities identifier supplied with the mapfile,
iii. created an UNDEF (reference) symbol to associate with the original
memcpy() definition,
iv. created a SHT_SUNW_capinfo section to capture the association of
memcpy with its capabilities group.
Using the link-editor, together with mapfiles and -z symbolcap, instances of
"psr" functions have been created and added to libc, libmd and libsoftcrypto.
After the integration of this link-editor project (6918143), a follow on project
will deliver the OSNet changes that use symbol capabilities.
-------------------------------------------------------------------------------
The materials directory contains the changes for this project that have been
applied to the Linker and Libraries Manual. Note, the LLM.pdf file only
contains the updated pages that are relevant to this project, this isn't the
whole book. Changes are diff-marked.
-------------------------------------------------------------------------------
The following updates are provided.
Release Binding: Patch/Micro
All ELF updates that follow: Committed
<sys/elf.h>
New section types:
#define SHT_SUNW_capchain 0x6fffffef
#define SHT_SUNW_capinfo 0x6ffffff0
New section definitions:
typedef Elf32_Word Elf32_Capinfo;
typedef Elf32_Word Elf32_Capchain;
typedef Elf64_Xword Elf64_Capinfo;
typedef Elf64_Word Elf64_Capchain;
Capability section version numbers and reserved group tag:
#define CAPINFO_NONE 0
#define CAPINFO_CURRENT 1
#define CAPINFO_NUM 2
#define CAPCHAIN_NONE 0
#define CAPCHAIN_CURRENT 1
#define CAPCHAIN_NUM 2
#define CAPINFO_SUNW_GLOB 0xff
New Capabilities tags:
#define CA_SUNW_HW_2 3 /* second hardware capabilities entry */
#define CA_SUNW_PLAT 4 /* platform capability entry */
#define CA_SUNW_MACH 5 /* machine capability entry */
#define CA_SUNW_ID 6 /* capability identifier */
<sys/machelf.h>
New section aliases:
typedef Elf64_Capinfo Capinfo;
typedef Elf64_Capchain Capchain;
typedef Elf32_Capinfo Capinfo;
typedef Elf32_Capchain Capchain;
<sys/link.h>
New .dynamic tags:
#define DT_SUNW_CAPINFO 0x60000018 /* capabilities symbols */
#define DT_SUNW_CAPCHAIN 0x6000001a /* capabilities chain info */
#define DT_SUNW_CAPCHAINENT 0x6000001d /* capabilities chain entry */
#define DT_SUNW_CAPCHAINSZ 0x6000001f /* capabilities chain size */
New .Syminfo flag:
#define SYMINFO_FLG_CAP 0x0100 /* symbol is capabilities */
/* specific */
ld(1)
-z symbolcap
Convert a relocatable object that defines object capa-
bilities into a relocatable object that defines symbol
capabilities. See Converting Object Capabilities to Sym-
bol Capabilities in Linker and Libraries Guide.
New mapfile tokens:
platcap = name;
machcap = name;
idcap = name;
elfdump()
No option changes, but the -H recognizes symbol capabilities and prints the
capabilities and the associated symbols.
ld.so.1
Recognizes the new symbol capabilities infrastructure, and provides new
environment variables to control experimentation:
LD_PLATCAP={name}
LD_MACHCAP={name}
LD_HWCAP=[+-]{token | number},...
LD_SFCAP=[+-]{token | number},...
LD_CAP_FILES=file,...
--
Rod.
