Commit-ID: d6a2f9035bfc27d0e9d78b13635dda9fb017ac01 Gitweb: http://git.kernel.org/tip/d6a2f9035bfc27d0e9d78b13635dda9fb017ac01 Author: David Carrillo-Cisneros <davi...@google.com> AuthorDate: Wed, 17 Aug 2016 13:55:06 -0700 Committer: Ingo Molnar <mi...@kernel.org> CommitDate: Thu, 18 Aug 2016 10:53:59 +0200
perf/core: Introduce PMU_EV_CAP_READ_ACTIVE_PKG Introduce the flag PMU_EV_CAP_READ_ACTIVE_PKG, useful for uncore events, that allows a PMU to signal the generic perf code that an event is readable in the current CPU if the event is active in a CPU in the same package as the current CPU. This is an optimization that avoids a unnecessary IPI for the common case where uncore events are run and read in the same package but in different CPUs. As an example, the IPI removal speeds up perf_read() in my Haswell system as follows: - For event UNC_C_LLC_LOOKUP: From 260 us to 31 us. - For event RAPL's power/energy-cores/: From to 255 us to 27 us. For the optimization to work, all events in the group must have it (similarly to PERF_EV_CAP_SOFTWARE). Signed-off-by: David Carrillo-Cisneros <davi...@google.com> Signed-off-by: Peter Zijlstra (Intel) <pet...@infradead.org> Cc: Alexander Shishkin <alexander.shish...@linux.intel.com> Cc: Arnaldo Carvalho de Melo <a...@redhat.com> Cc: David Carrillo-Cisneros <davi...@google.com> Cc: Jiri Olsa <jo...@redhat.com> Cc: Kan Liang <kan.li...@intel.com> Cc: Linus Torvalds <torva...@linux-foundation.org> Cc: Paul Turner <p...@google.com> Cc: Peter Zijlstra <pet...@infradead.org> Cc: Stephane Eranian <eran...@google.com> Cc: Thomas Gleixner <t...@linutronix.de> Cc: Vegard Nossum <vegard.nos...@gmail.com> Cc: Vince Weaver <vincent.wea...@maine.edu> Link: http://lkml.kernel.org/r/1471467307-61171-4-git-send-email-davi...@google.com Signed-off-by: Ingo Molnar <mi...@kernel.org> --- include/linux/perf_event.h | 3 +++ kernel/events/core.c | 25 +++++++++++++++++++++++-- 2 files changed, 26 insertions(+), 2 deletions(-) diff --git a/include/linux/perf_event.h b/include/linux/perf_event.h index 6f7459f..5c53625 100644 --- a/include/linux/perf_event.h +++ b/include/linux/perf_event.h @@ -514,8 +514,11 @@ typedef void (*perf_overflow_handler_t)(struct perf_event *, * Event capabilities. For event_caps and groups caps. * * PERF_EV_CAP_SOFTWARE: Is a software event. + * PERF_EV_CAP_READ_ACTIVE_PKG: A CPU event (or cgroup event) that can be read + * from any CPU in the package where it is active. */ #define PERF_EV_CAP_SOFTWARE BIT(0) +#define PERF_EV_CAP_READ_ACTIVE_PKG BIT(1) #define SWEVENT_HLIST_BITS 8 #define SWEVENT_HLIST_SIZE (1 << SWEVENT_HLIST_BITS) diff --git a/kernel/events/core.c b/kernel/events/core.c index 8c42a5a..3f07e6c 100644 --- a/kernel/events/core.c +++ b/kernel/events/core.c @@ -3424,6 +3424,22 @@ struct perf_read_data { int ret; }; +static int find_cpu_to_read(struct perf_event *event, int local_cpu) +{ + int event_cpu = event->oncpu; + u16 local_pkg, event_pkg; + + if (event->group_caps & PERF_EV_CAP_READ_ACTIVE_PKG) { + event_pkg = topology_physical_package_id(event_cpu); + local_pkg = topology_physical_package_id(local_cpu); + + if (event_pkg == local_pkg) + return local_cpu; + } + + return event_cpu; +} + /* * Cross CPU call to read the hardware event */ @@ -3545,7 +3561,7 @@ u64 perf_event_read_local(struct perf_event *event) static int perf_event_read(struct perf_event *event, bool group) { - int ret = 0; + int ret = 0, cpu_to_read, local_cpu; /* * If event is enabled and currently active on a CPU, update the @@ -3557,7 +3573,12 @@ static int perf_event_read(struct perf_event *event, bool group) .group = group, .ret = 0, }; - ret = smp_call_function_single(event->oncpu, __perf_event_read, &data, 1); + + local_cpu = get_cpu(); + cpu_to_read = find_cpu_to_read(event, local_cpu); + put_cpu(); + + ret = smp_call_function_single(cpu_to_read, __perf_event_read, &data, 1); /* The event must have been read from an online CPU: */ WARN_ON_ONCE(ret); ret = ret ? : data.ret;
