On Mon, Jun 8, 2026 at 6:00 AM Tanushree Shah <[email protected]> wrote: > > This RFC patch series introduces support for converting perf.data files > containing tracepoint events into trace.dat format, enabling seamless > visualization and analysis using KerneShark.
Thanks for doing this, this is a useful feature! nit: typo KernelShark > > ====================== > Background and Motivation > ====================== > > Currently, perf and trace-cmd operate as separate tracing ecosystems with > incompatible data formats. Users who collect tracepoint data with > 'perf record' cannot easily visualize it in KernelShark's graphical > timeline view or leverage trace-cmd's analysis capabilities. > > This creates workflow friction when users need to: > > - Visualize perf tracepoint data in KernelShark's interactive graphical > timeline > - Share trace data between perf and trace-cmd workflows and toolchains > - Perform architecture-independent conversion and analysis of traces > > This conversion bridge eliminates these barriers by enabling seamless > data exchange between perf and trace-cmd ecosystems, allowing users to > choose the best tool for each analysis phase. > > ====================== > Implementation Overview > ====================== > > The series implements the trace.dat file format specification (version 7) > within perf's data conversion framework. > > **Patch 1/4: Core trace.dat Export Infrastructure** > Introduces util/trace-dat.c and util/trace-dat.h implementing: > - Per-CPU raw event buffer management (init, collect, free) > - Ftrace ring buffer page construction > - trace.dat section writers (strings, options, flyrecord sections) > > **Patch 2/4: Metadata Integration** > Extends util/trace-event-read.c to write trace.dat metadata during > perf.data > parsing: > - Initial format header (magic, version, endian, page size, compression) > - Section 16: HEADER INFO (header_page + header_event) > - Section 17: FTRACE EVENT FORMATS > - Section 18: EVENT FORMATS (per system/event format files) > - Section 19: KALLSYMS > - Section 21: CMDLINES > - Section 15: STRINGS (written last after all sections) > > **Patch 3/4: Conversion Backend** > Implements util/data-convert-trace.c with trace_convert__perf2dat() > function: > - Processes PERF_TYPE_TRACEPOINT samples via process_sample_event() > - Collects raw event data per-CPU using trace_dat__collect_cpu_event() > - Writes OPTIONS sections (CPUCOUNT, TRACECLOCK, metadata offsets) > - Writes FLYRECORD section with per-CPU ring buffer pages > > **Patch 4/4: User Interface** > Extends tools/perf/builtin-data.c with --to-trace-dat option: > - Adds command-line option for trace.dat output > - Mutually exclusive with --to-ctf and --to-json > - Calls trace_convert__perf2dat() to perform conversion > > ====================== > Current Implementation Details > ====================== > > **trace.dat Format Version:** > The implementation currently targets trace.dat format version 7, which > is the stable version supported by current trace-cmd releases (v3.x). > This version is hardcoded to ensure compatibility with existing > trace-cmd and KernelShark installations. Future enhancements could add > version negotiation or support for newer format versions as they become > standardized. > > **Compression Strategy:** > Compression is explicitly disabled (set to NONE) in the generated > trace.dat files. > This design choice: > - Simplifies the initial implementation and testing > - Ensures maximum compatibility across trace-cmd versions > - Avoids external compression library dependencies > > Future work could add support for various compression algorithms (zlib, > zstd, lz4) with runtime selection via command-line options, significantly > reducing file sizes for large traces. > > ====================== > Usage Example > ====================== > > ```bash > *Record tracepoint events with perf* > perf record -e sched:sched_switch -e sched:sched_wakeup -a sleep 10 > > *Convert to trace.dat format* > perf data convert --to-trace-dat=output.dat > > *Verify trace.dat structure* > trace-cmd dump --summary output.dat > > *Analyze with trace-cmd* > trace-cmd report output.dat > > *Visualize in KernelShark* > kernelshark output.dat > ``` > > **Conversion Output:** > ``` > [ perf data convert: Converted 'perf.data' into trace.dat format > 'output.dat' ] > [ perf data convert: Converted 2684 events ] > ``` > **trace-cmd dump --summary Output:** > ``` > Tracing meta data in file output.dat: > [Initial format] > 7 [Version] > 0 [Little endian] > 8 [Bytes in a long] > 65536 [Page size, bytes] > none [Compression algorithm] > [Compression version] > [buffer "", "local" clock, 65536 page size, 16 cpus, 1048576 bytes > flyrecord data] > [10 options] > [Saved command lines, 0 bytes] > [Kallsyms, 0 bytes] > [Ftrace format, 0 events] > [Header page, 206 bytes] > [Header event, 205 bytes] > [Events format, 1 systems] > [9 sections] > ``` > ====================== > Testing and Verification > ====================== > > The series has been extensively tested with: > - Various tracepoint events (sched, irq, syscalls, block I/O) > - Mixed recordings containing both tracepoint and non-tracepoint events > only tracepoints converted) > - Verification with trace-cmd report and KernelShark visualization > - Memory leak testing with Valgrind (0 bytes leaked) > - Cross-architecture testing (x86_64, ppc64le) It seems that some of this could be a test to give coverage of the feature. We have similar tests for other convertors: https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/tests/shell/test_perf_data_converter_ctf.sh?h=perf-tools-next https://web.git.kernel.org/pub/scm/linux/kernel/git/perf/perf-tools-next.git/tree/tools/perf/tests/shell/test_perf_data_converter_json.sh?h=perf-tools-next I think Sashiko has caught some coding issues, so I'll hold off on a full review until the churn from Sashiko subsides. Thanks! Ian > All generated trace.dat files successfully open in: > - trace-cmd report (v3.1+) > - KernelShark (v2.0+) > > ====================== > Next Steps > ====================== > > We would highly appreciate reviews, comments, and feedback on: > - The overall architectural approach and integration points > - Compatibility considerations with trace-cmd ecosystem > - Performance characteristics for large-scale traces > - Additional use cases or workflow scenarios > - Future enhancement priorities > > Tanushree Shah (4): > perf/trace-dat: Add trace.dat export infrastructure > perf/trace-event: Write trace.dat metadata sections during parsing > perf data-convert: Add perf.data to trace.dat conversion backend > perf data: Add --to-trace-dat option for converting perf.data > tracepoint events into trace.dat format > > tools/perf/builtin-data.c | 38 +- > tools/perf/util/Build | 2 + > tools/perf/util/data-convert-trace.c | 152 ++++++ > tools/perf/util/data-convert.h | 4 + > tools/perf/util/trace-dat.c | 705 +++++++++++++++++++++++++++ > tools/perf/util/trace-dat.h | 79 +++ > tools/perf/util/trace-event-read.c | 259 +++++++++- > 7 files changed, 1230 insertions(+), 9 deletions(-) > create mode 100644 tools/perf/util/data-convert-trace.c > create mode 100644 tools/perf/util/trace-dat.c > create mode 100644 tools/perf/util/trace-dat.h > > -- > 2.53.0 >
