Hi Steven, Masami,
Open question 2 in the v7 cover letter was whether the map generation
should be part of the ABI. I think it has to be, and I would rather
settle the record format with you before sending a v8.
The problem is that reset clears the map and resets the element cursor to
zero, so a new stack takes slot 42 immediately afterwards. A stack_id 42
recorded before the reset is still sitting in the ring buffer, and
resolving it now yields the new occupant of that slot. Userspace cannot
tell that apart from a correct lookup - it gets a plausible stack, not an
error.
I first tried to fix it in the table, storing the generation per slot and
comparing it with the current one on lookup. That does not work: the
comparison tells you which generation the slot now holds, not which
generation the caller's id came from, and the case that matters is
exactly when those differ and the slot has already been refilled. The
identity has to travel with the record.
So for v8:
struct stack_id_entry {
struct trace_entry ent;
u64 stack_cookie; /* gen << 32 | id */
};
A single opaque 64-bit cookie, high half generation, low half slot. An id
is still only valid within its own generation; the cookie makes crossing
that boundary detectable.
stack_map_bin reports the generation of the dump in the header word that
is currently reserved (ABI version 1 -> 2), so a dump and a record can be
paired. Text output splits the cookie rather than printing it raw:
<stack_id 42 gen 3>
Cost, against the workload measured in the v7 cover letter:
full stack, ~20 frames trace_entry + size + 8*20 = 172 B
v7, 4-byte id trace_entry + 4 = 12 B
v8, 8-byte cookie trace_entry + 8 = 16 B
Retention on that workload drops from about 2.17x to about 2.0x. I will
re-measure rather than quote that from arithmetic.
Two alternatives I rejected, in case you would have picked one:
Packing the generation into the existing 32-bit id keeps the record at 4
bytes, but with FTRACE_STACKMAP_BITS_MAX at 18 the slot index needs 19
bits, leaving 12 bits of generation in a signed field. 4096 resets wrap,
and a wrapped generation passes the check - the same failure, only
rarer. It also ties the id layout to the capacity limit.
Two separate u32 fields come to the same 16 bytes and would make the
format file self-describing, so an event filter could match the slot
directly. I went with the cookie because an opaque value says the whole
thing is the identifier - two fields invite userspace to read the slot
number on its own, which is exactly what is not meaningful without its
generation. If the filtering matters more, that is an easy change.
One consequence I had not expected. If a trace spans several resets,
dumping stack_map_bin before each one keeps the whole timeline resolvable
afterwards: every record says which generation it belongs to and every
dump header says which generation it is. Without the tag, saving several
dumps is pointless, since nothing says which dump a given record should
be read against.
Does that format look right?
v7 split the series into 10 focused patches following Masami's review of
v6, and the design is otherwise unchanged.
Thanks,
Pengfei
