Hi Jiri, On Fri, Nov 27, 2020 at 2:00 AM Jiri Olsa <jo...@kernel.org> wrote: > > hi, > adding the support to have buildid stored in mmap2 event, > so we can bypass the final perf record hunt on build ids. > > This patchset allows perf to record build ID in mmap2 event, > and adds perf tooling to store/download binaries to .debug > cache based on these build IDs. > > Note that the build id retrieval code is stolen from bpf > code, where it's been used (together with file offsets) > to replace IPs in user space stack traces. It's now added > under lib directory.
For the series: Acked-by: Namhyung Kim <namhy...@kernel.org> Thanks, Namhyung > > v4 changes: > - fixed typo in changelog [Namhyung] > - removed force_download bool from struct dso_store_data, > because it's not used [Namhyung] > > v3 changes: > - added acks > - removed forgotten debug code [Arnaldo] > - fixed readlink termination [Ian] > - fixed doc for --debuginfod=URLs [Ian] > - adopted kernel's memchr_inv function and used > it in build_id__is_defined function [Arnaldo] > > On recording server: > > - on the recording server we can run record with --buildid-mmap > option to store build ids in mmap2 events: > > # perf record --buildid-mmap > ^C[ perf record: Woken up 2 times to write data ] > [ perf record: Captured and wrote 0.836 MB perf.data ] > > - it stores nothing to ~/.debug cache: > > # find ~/.debug > find: ‘/root/.debug’: No such file or directory > > - and still reports properly: > > # perf report --stdio > ... > 99.82% swapper [kernel.kallsyms] [k] native_safe_halt > 0.03% swapper [kernel.kallsyms] [k] finish_task_switch > 0.02% swapper [kernel.kallsyms] [k] __softirqentry_text_start > 0.01% kcompactd0 [kernel.kallsyms] [k] > _raw_spin_unlock_irqrestore > 0.01% ksoftirqd/6 [kernel.kallsyms] [k] slab_free_freelist_hook > 0.01% kworker/17:1H-x [kernel.kallsyms] [k] slab_free_freelist_hook > > - display used/hit build ids: > > # perf buildid-list | head -5 > 5dcec522abf136fcfd3128f47e131f2365834dd7 /proc/kcore > 589e403a34f55486bcac848a45e00bcdeedd1ca8 /usr/lib64/libcrypto.so.1.1.1g > 94569566d4eac7e9c87ba029d43d4e2158f9527e /usr/lib64/libpthread-2.30.so > 559b9702bebe31c6d132c8dc5cc887673d65d5b5 /usr/lib64/libc-2.30.so > 40da7abe89f631f60538a17686a7d65c6a02ed31 /usr/lib64/ld-2.30.so > > - store build id binaries into build id cache: > > # perf buildid-cache -a perf.data > OK 5dcec522abf136fcfd3128f47e131f2365834dd7 /proc/kcore > OK 589e403a34f55486bcac848a45e00bcdeedd1ca8 > /usr/lib64/libcrypto.so.1.1.1g > OK 94569566d4eac7e9c87ba029d43d4e2158f9527e > /usr/lib64/libpthread-2.30.so > OK 559b9702bebe31c6d132c8dc5cc887673d65d5b5 /usr/lib64/libc-2.30.so > OK 40da7abe89f631f60538a17686a7d65c6a02ed31 /usr/lib64/ld-2.30.so > OK a674f7a47c78e35a088104647b9640710277b489 /usr/sbin/sshd > OK e5cb4ca25f46485bdbc691c3a92e7e111dac3ef2 /usr/bin/bash > OK 9bc8589108223c944b452f0819298a0c3cba6215 /usr/bin/find > > # find ~/.debug | head -5 > /root/.debug > /root/.debug/proc > /root/.debug/proc/kcore > /root/.debug/proc/kcore/5dcec522abf136fcfd3128f47e131f2365834dd7 > /root/.debug/proc/kcore/5dcec522abf136fcfd3128f47e131f2365834dd7/kallsyms > > - run debuginfod daemon to provide binaries to another server (below) > (the initialization could take some time) > > # debuginfod -F / > > > On another server: > > - copy perf.data from 'record' server and run: > > $ find ~/.debug/ > find: ‘/home/jolsa/.debug/’: No such file or directory > > $ perf buildid-list | head -5 > No kallsyms or vmlinux with build-id > 5dcec522abf136fcfd3128f47e131f2365834dd7 was found > 5dcec522abf136fcfd3128f47e131f2365834dd7 [kernel.kallsyms] > 5784f813b727a50cfd3363234aef9fcbab685cc4 > /lib/modules/5.10.0-rc2speed+/kernel/fs/xfs/xfs.ko > 589e403a34f55486bcac848a45e00bcdeedd1ca8 /usr/lib64/libcrypto.so.1.1.1g > 94569566d4eac7e9c87ba029d43d4e2158f9527e /usr/lib64/libpthread-2.30.so > 559b9702bebe31c6d132c8dc5cc887673d65d5b5 /usr/lib64/libc-2.30.so > > - report does not show anything (kernel build id does not match): > > $ perf report --stdio > ... > 76.73% swapper [kernel.kallsyms] [k] 0xffffffff81aa8ebe > 1.89% find [kernel.kallsyms] [k] 0xffffffff810f2167 > 0.93% sshd [kernel.kallsyms] [k] 0xffffffff8153380c > 0.83% swapper [kernel.kallsyms] [k] 0xffffffff81104b0b > 0.71% kworker/u40:2-e [kernel.kallsyms] [k] 0xffffffff810f3850 > 0.70% kworker/u40:0-e [kernel.kallsyms] [k] 0xffffffff810f3850 > 0.64% find [kernel.kallsyms] [k] 0xffffffff81a9ba0a > 0.63% find [kernel.kallsyms] [k] 0xffffffff81aa93b0 > > - add build ids does not work, because existing binaries (on another server) > have different build ids: > > $ perf buildid-cache -a perf.data > No kallsyms or vmlinux with build-id > 5dcec522abf136fcfd3128f47e131f2365834dd7 was found > FAIL 5dcec522abf136fcfd3128f47e131f2365834dd7 [kernel.kallsyms] > FAIL 5784f813b727a50cfd3363234aef9fcbab685cc4 > /lib/modules/5.10.0-rc2speed+/kernel/fs/xfs/xfs.ko > FAIL 589e403a34f55486bcac848a45e00bcdeedd1ca8 > /usr/lib64/libcrypto.so.1.1.1g > FAIL 94569566d4eac7e9c87ba029d43d4e2158f9527e > /usr/lib64/libpthread-2.30.so > FAIL 559b9702bebe31c6d132c8dc5cc887673d65d5b5 /usr/lib64/libc-2.30.so > FAIL 40da7abe89f631f60538a17686a7d65c6a02ed31 /usr/lib64/ld-2.30.so > FAIL a674f7a47c78e35a088104647b9640710277b489 /usr/sbin/sshd > FAIL e5cb4ca25f46485bdbc691c3a92e7e111dac3ef2 /usr/bin/bash > FAIL 9bc8589108223c944b452f0819298a0c3cba6215 /usr/bin/find > > - add build ids with debuginfod setup pointing to record server: > > $ perf buildid-cache -a perf.data --debuginfod http://192.168.122.174:8002 > No kallsyms or vmlinux with build-id > 5dcec522abf136fcfd3128f47e131f2365834dd7 was found > OK 5dcec522abf136fcfd3128f47e131f2365834dd7 [kernel.kallsyms] > OK 5784f813b727a50cfd3363234aef9fcbab685cc4 > /lib/modules/5.10.0-rc2speed+/kernel/fs/xfs/xfs.ko > OK 589e403a34f55486bcac848a45e00bcdeedd1ca8 > /usr/lib64/libcrypto.so.1.1.1g > OK 94569566d4eac7e9c87ba029d43d4e2158f9527e > /usr/lib64/libpthread-2.30.so > OK 559b9702bebe31c6d132c8dc5cc887673d65d5b5 /usr/lib64/libc-2.30.so > OK 40da7abe89f631f60538a17686a7d65c6a02ed31 /usr/lib64/ld-2.30.so > OK a674f7a47c78e35a088104647b9640710277b489 /usr/sbin/sshd > OK e5cb4ca25f46485bdbc691c3a92e7e111dac3ef2 /usr/bin/bash > OK 9bc8589108223c944b452f0819298a0c3cba6215 /usr/bin/find > > - and report works: > > $ perf report --stdio > ... > 76.73% swapper [kernel.kallsyms] [k] native_safe_halt > 1.91% find [kernel.kallsyms] [k] queue_work_on > 0.93% sshd [kernel.kallsyms] [k] iowrite16 > 0.83% swapper [kernel.kallsyms] [k] finish_task_switch > 0.72% kworker/u40:2-e [kernel.kallsyms] [k] process_one_work > 0.70% kworker/u40:0-e [kernel.kallsyms] [k] process_one_work > 0.64% find [kernel.kallsyms] [k] > syscall_enter_from_user_mode > 0.63% find [kernel.kallsyms] [k] > _raw_spin_unlock_irqrestore > > - because we have the data in build id cache: > > $ find ~/.debug | head -10 > .../.debug > .../.debug/home > .../.debug/home/jolsa > .../.debug/home/jolsa/.cache > .../.debug/home/jolsa/.cache/debuginfod_client > > .../.debug/home/jolsa/.cache/debuginfod_client/5dcec522abf136fcfd3128f47e131f2365834dd7 > > .../.debug/home/jolsa/.cache/debuginfod_client/5dcec522abf136fcfd3128f47e131f2365834dd7/executable > > .../.debug/home/jolsa/.cache/debuginfod_client/5dcec522abf136fcfd3128f47e131f2365834dd7/executable/5dcec522abf136fcfd3128f47e131f2365834dd7 > > .../.debug/home/jolsa/.cache/debuginfod_client/5dcec522abf136fcfd3128f47e131f2365834dd7/executable/5dcec522abf136fcfd3128f47e131f2365834dd7/elf > > .../.debug/home/jolsa/.cache/debuginfod_client/5dcec522abf136fcfd3128f47e131f2365834dd7/executable/5dcec522abf136fcfd3128f47e131f2365834dd7/debug > > > Available also in: > git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git > perf/build_id > > thanks, > jirka > > > Cc: Frank Ch. Eigler <f...@redhat.com> > Cc: Mark Wielaard <m...@redhat.com> > --- > Jiri Olsa (25): > bpf: Move stack_map_get_build_id into lib > bpf: Add size arg to build_id_parse function > perf: Add build id data in mmap2 event > tools headers uapi: Sync tools/include/uapi/linux/perf_event.h > tools lib: Adopt memchr_inv() from kernel > perf tools: Do not swap mmap2 fields in case it contains build id > perf tools: Add build_id__is_defined function > perf tools: Add filename__decompress function > perf tools: Add support to read build id from compressed elf > perf tools: Add check for existing link in buildid dir > perf tools: Use struct extra_kernel_map in > machine__process_kernel_mmap_event > perf tools: Try to load vmlinux from buildid database > perf tools: Store build id from mmap2 events > perf tools: Allow mmap2 event to synthesize kernel image > perf tools: Allow mmap2 event to synthesize modules > perf tools: Synthesize build id for kernel/modules/tasks > perf tools: Add support to display build id for mmap2 events > perf tools: Use machine__for_each_dso in perf_session__cache_build_ids > perf tools: Add __perf_session__cache_build_ids function > perf tools: Add is_perf_data function > perf tools: Add build_id_cache__add function > perf buildid-cache: Add support to add build ids from perf data > perf buildid-cache: Add --debuginfod option > perf buildid-list: Add support for mmap2's buildid events > perf record: Add --buildid-mmap option to enable mmap's build id > > include/linux/buildid.h | 12 +++++ > include/uapi/linux/perf_event.h | 42 +++++++++++++++--- > kernel/bpf/stackmap.c | 143 > ++---------------------------------------------------------- > kernel/events/core.c | 32 ++++++++++++-- > lib/Makefile | 3 +- > lib/buildid.c | 149 > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > tools/include/linux/string.h | 1 + > tools/include/uapi/linux/perf_event.h | 42 +++++++++++++++--- > tools/lib/perf/include/perf/event.h | 18 ++++++-- > tools/lib/string.c | 58 > +++++++++++++++++++++++++ > tools/perf/Documentation/perf-buildid-cache.txt | 18 +++++++- > tools/perf/Documentation/perf-config.txt | 10 ++++- > tools/perf/Documentation/perf-record.txt | 3 ++ > tools/perf/builtin-buildid-cache.c | 241 > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++--- > tools/perf/builtin-buildid-list.c | 3 ++ > tools/perf/builtin-record.c | 20 +++++++++ > tools/perf/tests/shell/trace+probe_vfs_getname.sh | 2 +- > tools/perf/util/build-id.c | 127 > ++++++++++++++++++++++++++++++++++------------------- > tools/perf/util/build-id.h | 8 ++++ > tools/perf/util/data.c | 19 ++++++++ > tools/perf/util/data.h | 1 + > tools/perf/util/dso.c | 31 ++++++++----- > tools/perf/util/dso.h | 2 + > tools/perf/util/event.c | 41 +++++++++++++----- > tools/perf/util/evsel.c | 10 +++-- > tools/perf/util/machine.c | 80 > +++++++++++++++++++++------------- > tools/perf/util/map.c | 8 +++- > tools/perf/util/map.h | 3 +- > tools/perf/util/perf_api_probe.c | 10 +++++ > tools/perf/util/perf_api_probe.h | 1 + > tools/perf/util/perf_event_attr_fprintf.c | 2 + > tools/perf/util/probe-event.c | 6 +-- > tools/perf/util/record.h | 1 + > tools/perf/util/session.c | 11 +++-- > tools/perf/util/symbol-elf.c | 37 +++++++++++++++- > tools/perf/util/symbol.c | 16 +++++++ > tools/perf/util/symbol_conf.h | 3 +- > tools/perf/util/synthetic-events.c | 121 > ++++++++++++++++++++++++++++++++++++++------------- > 38 files changed, 1024 insertions(+), 311 deletions(-) > create mode 100644 include/linux/buildid.h > create mode 100644 lib/buildid.c >
