Hi all,
My ::whatis putback from last week:
6875845 ::whatis should be able to dump the bufctl/vmem seg
got me thinking about revisiting some long-standing issues with ::whatis:
1. addr::kgrep | ::whatis can take a huge amount of time, since
each line of ::whatis output walks the entire set of possible
locations. It would be much more efficient to do one pass,
sorting the addresses first.
2. There are several things which call for a more generic whatis
framework:
2.1: the sun4u "whatis" implementation which reports on
traptrace records, then returns DCMD_NEXT to call
the genunix version.
2.2: the minimal libumem "whatis" implementation, which
doesn't report on thread stacks, etc.
So over a few evenings last week, I rewrote ::whatis and designed a
plugin architecture for it. The ::whatis dcmd itself moves into the
builtin MDB module, and it handles tracking the requested addresses,
which ones have been found, etc. Unlike the current ::whatis, it
notices when it is invoked with input from a pipe. In that case, it
slurps up the and sorts the pipeline input, and does a single pass over
the plugins, which reports each pointer as it is discovered. At the
end, it prints out the remaining "unknown" pointers.
If the sorting isn't desired, it can be turned off (::whatis -k), which
will do individual passes.
When run without an address, the new ::whatis prints the active plugins.
With my wad, the output will be: (***s mark new functionality)
% echo ::whatis | mdb -k
NAME MODULE
modules genunix
threads genunix
pages genunix
kmem_touch genunix
kmem_notouch genunix
vmem genunix
mappings mdb ***
% echo ::whatis | mdb -p $$
NAME MODULE
threads libc.so.1 ***
umem_touch libumem.so.1
umem_notouch libumem.so.1
vmem libumem.so.1
mappings mdb ***
>
Note that the genunix ::whatis searches have been split up into 6 seperate
callbacks, and a new generic "mappings" walker has been added, which reports
which element of "::mappings" the symbol is in. (for the kernel, this is
the kas segment the address is in. For userspace, it's the segment the
address is in.)
I've also added a thread walker to libc, which reports ulwp_t pointers
and thread stacks.
Doing the single pass helps speed up the whatis processing (best of 4 times
on a machine with ample memory):
% echo ffffff0c8ce4a3a0::kgrep | mdb 126 > /tmp/pointers.out
old:
% echo "::cat /tmp/pointers.out | ::whatis" | ptime mdb 126 >/dev/null
real 3.984753993
user 3.083241102
sys 0.561740893
new:
% echo "::cat /tmp/pointers.out | ::whatis" |
ptime /path/to/mdb -R/path/to/workspace/proto/root_i386 126 >/dev/null
real 3.550363619
user 2.506509793
sys 0.550597086
(note the reduced user and real time)
Here's some sample output:
old:
> kmem_flags::whatis -a
fffffffffbf784f8 is kmem_flags, in genunix's data segment
> kmem_flags::whatis -av
Searching modules...
fffffffffbf784f8 is kmem_flags, in genunix's data segment
Searching threads...
Searching page structures...
Searching kmem_magazine_1's slabs...
...
Searching rpc_gss_data_cache's slabs...
Searching crypto_session_cache's slabs...
Searching kmem_va_4096's slabs...
...
Searching Hex0xffffff0151399588_minor_1's slabs...
Searching genunix`vmem...
Searching vmem arena vmem_seg...
Searching vmem arena vmem_seg for free virtual...
...
Searching vmem arena namefs_inodes for free virtual...
> *kmem_bufctl_cache::whatis -aq
ffffff0146e23018 is vmem_seg ffffff0146e0c660 allocated from kmem_cache
vmem arena
ffffff0146e23018 is ffffff0146e23000+18, vmem_seg ffffff0146e0e570
allocated from kmem_metadata vmem arena
ffffff0146e23018 is ffffff0146e21000+2018, vmem_seg ffffff0146e0b270
allocated from heap vmem arena
>
new:
> kmem_flags::whatis -a
fffffffffbf784f8 is kmem_flags, in genunix's data segment
fffffffffbf784f8 is kmem_flags, in ktextseg
> kmem_flags::whatis -av
Searching genunix`modules...
fffffffffbf784f8 is kmem_flags, in genunix's data segment
Searching genunix`threads...
Searching genunix`pages...
Searching genunix`kmem_touch...
Searching kmem_magazine_1's slabs...
...
Searching rpc_gss_data_cache's slabs...
Searching crypto_session_cache's slabs...
Searching genunix`kmem_notouch...
Searching kmem_va_4096's slabs...
...
Searching Hex0xffffff0151399588_minor_1's slabs...
Searching genunix`vmem...
Searching vmem arena vmem_seg...
Searching vmem arena vmem_seg for free virtual...
...
Searching vmem arena namefs_inodes for free virtual...
Searching mdb`mappings...
fffffffffbf784f8 is kmem_flags, in ktextseg
> *kmem_bufctl_cache::whatis -aq
ffffff0146e23018 is vmem_seg ffffff0146e0c660 allocated from kmem_cache
vmem arena
ffffff0146e23018 is ffffff0146e23000+18, vmem_seg ffffff0146e0e570
allocated from kmem_metadata vmem arena
ffffff0146e23018 is ffffff0146e21000+2018, vmem_seg ffffff0146e0b270
allocated from heap vmem arena
ffffff0146e23018 is in kvseg
>
In userspace:
old:
> foo::whatis
8061068 is unknown
> *foo::whatis -qa
807df88 is 807df80+8, bufctl 807f080 allocated from umem_alloc_112
807df88 is 807d000+f88, vmem_seg 807c438 allocated from umem_default
vmem arena
807df88 is 807d000+f88, vmem_seg 807c618 allocated from sbrk_heap
vmem arena
807df88 is 807d000+f88, vmem_seg 807c780 allocated from sbrk_top
vmem arena
> <esp::whatis
80470d8 is unknown
new:
> foo::whatis
8061068 is tmpc`foo, in ./tmpc
> *foo::whatis -qa
807df88 is 807df80+8, bufctl 807f080 allocated from umem_alloc_112
807df88 is 807d000+f88, vmem_seg 807c438 allocated from umem_default
vmem arena
807df88 is 807d000+f88, vmem_seg 807c618 allocated from sbrk_heap
vmem arena
807df88 is 807d000+f88, vmem_seg 807c780 allocated from sbrk_top
vmem arena
807df88 is in [ heap ]
> <esp::whatis
80470d8 is in [ stack tid=1 ]
To register a callback, modules call:
typedef int mdb_whatis_cb_t(mdb_whatis_t *, void *);
void mdb_whatis_register(uint_t prio, const char *name, mdb_whatis_cb_t *func,
void *arg, uint_t flags);
during their _mdb_init() processing. The arguments are:
prio: WHATIS_PRIO_EARLY, WHATIS_PRIO_ALLOCATOR, or WHATIS_PRIO_LATE.
Determines when the function will be called. Functions from
the same module with the same priority will be run in order
of registration.
name An identifier following the usual MDB semantics, which is
descriptive of the information searched. Displayed when
'::whatis" is invoked by itself.
func The function to invoke
arg A void * for the function
flags Can be:
0 Always invoke
WHATIS_REG_ID_ONLY Only invoke with "-i"
WHATIS_REG_NO_ID Only invoke without "-i"
The basic structure of a ::whatis callback is:
int
whatis_run_foo(mdb_whatis_t *w, void *arg)
{
foreach (address range of interest) {
const uintptr_t *cur = NULL;
while (mdb_whatis_match(w, base, size, &cur))
mdb_whatis_report_pointer(w, *cur, base,
"allocated from ...");
}
return (0 on success, 1 if there was an error);
}
mdb_whatis_t is an opaque type. The "pages" callback is the simplest
one in the wad; it looks like:
/*ARGSUSED*/
static int
whatis_walk_page(uintptr_t addr, const void *ignored, mdb_whatis_t *w)
{
static int machsize = 0;
const uintptr_t *cur = NULL;
if (machsize == 0) {
... /* initialize machsize to be sizeof (unix`page_t) */
}
while (mdb_whatis_match(w, addr, machsize, &cur))
mdb_whatis_report_pointer(w, *cur, addr,
"allocated as a page structure\n");
return (WHATIS_WALKRET(w));
}
/*ARGSUSED*/
static int
whatis_run_pages(mdb_whatis_t *w, void *ignored)
{
if (mdb_walk("page", (mdb_walk_cb_t)whatis_walk_page, w) == -1) {
mdb_warn("couldn't find page walker");
return (1);
}
return (0);
}
...
mdb_whatis_register(
WHATIS_PRIO_EARLY, "pages", whatis_run_pages, NULL,
WHATIS_REG_NO_ID);
...
The interfaces available to the callbacks are:
int mdb_whatis_overlaps(mdb_whatis_t *w, uintptr_t base, size_t size)
Returns non-zero if there are any buffers of interest in the range
[base, base + size).
int mdb_whatis_match(mdb_whatis_t *w, uintptr_t base, size_t size,
const uinptr_t **out)
Iteratively gives all pointers of interest in the range
[base, base+size). On the first call, *out must be NULL. If
the mdb_whatis_match() returns a non-zero value, *out is filled
with a pointer to the next pointer of interest. Otherwise,
*out is filled with a NULL pointer, and the search is complete.
void mdb_whatis_report_pointer(mdb_whatis_t *w, uintptr_t addr, uintptr_t base,
const char *format, ...)
Reports addr (an address returned from mdb_whatis_match()) as being
in an object starting at "base". If there is no such base, base should
be set to "addr". format, ... are an mdb_printf() format string and
any arguments.
The caller is responsible for printing a newline at the end of
their output.
addr is format... (base == size)
addr is base+offset, format... (base != size)
void mdb_whatis_report_symbol(mdb_whatis_t *w, uintptr_t addr,
const char *format, ...)
Similar to mdb_whatis_report_pointer(), but how addr is printed
depends upon whether it is in a symbol or not. The output will
look like:
addr is format... (no symbol matching addr)
addr is sym, format... (addr matches start of sym)
addr is sym+offset, format... (addr is inside sym)
uint_t mdb_whatis_flags(mdb_whatis_t *w)
Returns a flag word, with the following possible flags:
WHATIS_BUFCTL -b Caller requests buffer control
pointers
WHATIS_IDSPACE -i Caller requests a search of
ID spaces only.
WHATIS_QUIET -q Caller requests single-line
results only.
WHATIS_VERBOSE -v Caller requests information
about the progress of the
search.
uint_t mdb_whatis_done(mdb_whatis_t *w)
Returns non-zero if the search is complete.
#define WHATIS_WALKRET(w) (mdb_whatis_done(w) ? WALK_DONE : WALK_NEXT)
Shortcut macro for walker callbacks.
At the moment, I'm planning on these being project-private interfaces;
eventually, they could be added to the module API.
Does this sound good to everyone? I'm looking for approval of the design and
codereviewers. The webrev is here:
http://cr.opensolaris.org/~jwadams/onnv-whatis-2/
If you're in SWAN and want to play with it, I can e-mail you commands to
use to invoke it.
Thanks,
- jonathan
