[PATCH 2/5] tools lib traceevent: update KVM plugin

2015-10-21 Thread Arnaldo Carvalho de Melo
From: Paolo Bonzini 

The format of the role word has changed through the years and the plugin
was never updated; some VMX exit reasons were missing too.

Signed-off-by: Paolo Bonzini 
Acked-by: Steven Rostedt 
Cc: David Ahern 
Cc: Namhyung Kim 
Cc: kvm@vger.kernel.org
Link: 
http://lkml.kernel.org/r/1443695293-31127-1-git-send-email-pbonz...@redhat.com
Signed-off-by: Arnaldo Carvalho de Melo 
---
 tools/lib/traceevent/plugin_kvm.c | 25 +
 1 file changed, 17 insertions(+), 8 deletions(-)

diff --git a/tools/lib/traceevent/plugin_kvm.c 
b/tools/lib/traceevent/plugin_kvm.c
index 88fe83dff7cd..18536f756577 100644
--- a/tools/lib/traceevent/plugin_kvm.c
+++ b/tools/lib/traceevent/plugin_kvm.c
@@ -124,7 +124,10 @@ static const char *disassemble(unsigned char *insn, int 
len, uint64_t rip,
_ER(WBINVD,  54)\
_ER(XSETBV,  55)\
_ER(APIC_WRITE,  56)\
-   _ER(INVPCID, 58)
+   _ER(INVPCID, 58)\
+   _ER(PML_FULL,62)\
+   _ER(XSAVES,  63)\
+   _ER(XRSTORS, 64)
 
 #define SVM_EXIT_REASONS \
_ER(EXIT_READ_CR0,  0x000)  \
@@ -352,15 +355,18 @@ static int kvm_nested_vmexit_handler(struct trace_seq *s, 
struct pevent_record *
 union kvm_mmu_page_role {
unsigned word;
struct {
-   unsigned glevels:4;
unsigned level:4;
+   unsigned cr4_pae:1;
unsigned quadrant:2;
-   unsigned pad_for_nice_hex_output:6;
unsigned direct:1;
unsigned access:3;
unsigned invalid:1;
-   unsigned cr4_pge:1;
unsigned nxe:1;
+   unsigned cr0_wp:1;
+   unsigned smep_and_not_wp:1;
+   unsigned smap_and_not_wp:1;
+   unsigned pad_for_nice_hex_output:8;
+   unsigned smm:8;
};
 };
 
@@ -385,15 +391,18 @@ static int kvm_mmu_print_role(struct trace_seq *s, struct 
pevent_record *record,
if (pevent_is_file_bigendian(event->pevent) ==
pevent_is_host_bigendian(event->pevent)) {
 
-   trace_seq_printf(s, "%u/%u q%u%s %s%s %spge %snxe",
+   trace_seq_printf(s, "%u q%u%s %s%s %spae %snxe %swp%s%s%s",
 role.level,
-role.glevels,
 role.quadrant,
 role.direct ? " direct" : "",
 access_str[role.access],
 role.invalid ? " invalid" : "",
-role.cr4_pge ? "" : "!",
-role.nxe ? "" : "!");
+role.cr4_pae ? "" : "!",
+role.nxe ? "" : "!",
+role.cr0_wp ? "" : "!",
+role.smep_and_not_wp ? " smep" : "",
+role.smap_and_not_wp ? " smap" : "",
+role.smm ? " smm" : "");
} else
trace_seq_printf(s, "WORD: %08x", role.word);
 
-- 
2.1.0

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[GIT PULL 0/5] perf/core improvements and fixes

2015-10-21 Thread Arnaldo Carvalho de Melo
Hi Ingo,

Please consider pulling,

- Arnaldo

The following changes since commit 43e41adc9e8c36545888d78fed2ef8d102a938dc:

  perf record: Add ability to sample call branches (2015-10-20 10:30:55 +0200)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git 
tags/perf-core-for-mingo

for you to fetch changes up to e3d006ce8180a0c025ce66bdc89bbc125f85be57:

  perf annotate: Add debug message for out of bounds sample (2015-10-21 
18:12:37 -0300)


perf/core improvements and fixes:

User visible:

- Print branch filter state with verbose mode (Andi Kleen)

- Fix core dump caused by per-socket/core system-wide stat (Kan Liang)

- Update libtraceevent KVM plugin (Paolo Bonzini)

Developer stuff:

- Add fixdep to 'tools/build' .gitignore (Yunlong Song)

Signed-off-by: Arnaldo Carvalho de Melo 


Andi Kleen (1):
  perf evsel: Print branch filter state with -vv

Arnaldo Carvalho de Melo (1):
  perf annotate: Add debug message for out of bounds sample

Kan Liang (1):
  perf cpu_map: Fix core dump caused by per-socket/core system-wide stat

Paolo Bonzini (1):
  tools lib traceevent: update KVM plugin

Yunlong Song (1):
  perf build: Add fixdep to .gitignore

 tools/build/.gitignore|  1 +
 tools/lib/traceevent/plugin_kvm.c | 25 +
 tools/perf/util/annotate.c|  5 -
 tools/perf/util/cpumap.c  |  2 +-
 tools/perf/util/evsel.c   |  1 +
 5 files changed, 24 insertions(+), 10 deletions(-)
 create mode 100644 tools/build/.gitignore
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 2/4] tools lib traceevent: update KVM plugin

2015-10-20 Thread Arnaldo Carvalho de Melo
From: Paolo Bonzini 

The format of the role word has changed through the years and the plugin
was never updated; some VMX exit reasons were missing too.

Signed-off-by: Paolo Bonzini 
Acked-by: Steven Rostedt 
Cc: David Ahern 
Cc: Namhyung Kim 
Cc: kvm@vger.kernel.org
Link: 
http://lkml.kernel.org/r/1443695293-31127-1-git-send-email-pbonz...@redhat.com
Signed-off-by: Arnaldo Carvalho de Melo 
---
 tools/lib/traceevent/plugin_kvm.c | 25 +
 1 file changed, 17 insertions(+), 8 deletions(-)

diff --git a/tools/lib/traceevent/plugin_kvm.c 
b/tools/lib/traceevent/plugin_kvm.c
index 88fe83dff7cd..18536f756577 100644
--- a/tools/lib/traceevent/plugin_kvm.c
+++ b/tools/lib/traceevent/plugin_kvm.c
@@ -124,7 +124,10 @@ static const char *disassemble(unsigned char *insn, int 
len, uint64_t rip,
_ER(WBINVD,  54)\
_ER(XSETBV,  55)\
_ER(APIC_WRITE,  56)\
-   _ER(INVPCID, 58)
+   _ER(INVPCID, 58)\
+   _ER(PML_FULL,62)\
+   _ER(XSAVES,  63)\
+   _ER(XRSTORS, 64)
 
 #define SVM_EXIT_REASONS \
_ER(EXIT_READ_CR0,  0x000)  \
@@ -352,15 +355,18 @@ static int kvm_nested_vmexit_handler(struct trace_seq *s, 
struct pevent_record *
 union kvm_mmu_page_role {
unsigned word;
struct {
-   unsigned glevels:4;
unsigned level:4;
+   unsigned cr4_pae:1;
unsigned quadrant:2;
-   unsigned pad_for_nice_hex_output:6;
unsigned direct:1;
unsigned access:3;
unsigned invalid:1;
-   unsigned cr4_pge:1;
unsigned nxe:1;
+   unsigned cr0_wp:1;
+   unsigned smep_and_not_wp:1;
+   unsigned smap_and_not_wp:1;
+   unsigned pad_for_nice_hex_output:8;
+   unsigned smm:8;
};
 };
 
@@ -385,15 +391,18 @@ static int kvm_mmu_print_role(struct trace_seq *s, struct 
pevent_record *record,
if (pevent_is_file_bigendian(event->pevent) ==
pevent_is_host_bigendian(event->pevent)) {
 
-   trace_seq_printf(s, "%u/%u q%u%s %s%s %spge %snxe",
+   trace_seq_printf(s, "%u q%u%s %s%s %spae %snxe %swp%s%s%s",
 role.level,
-role.glevels,
 role.quadrant,
 role.direct ? " direct" : "",
 access_str[role.access],
 role.invalid ? " invalid" : "",
-role.cr4_pge ? "" : "!",
-role.nxe ? "" : "!");
+role.cr4_pae ? "" : "!",
+role.nxe ? "" : "!",
+role.cr0_wp ? "" : "!",
+role.smep_and_not_wp ? " smep" : "",
+role.smap_and_not_wp ? " smap" : "",
+role.smm ? " smm" : "");
} else
trace_seq_printf(s, "WORD: %08x", role.word);
 
-- 
2.1.0

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] tools lib traceevent: update KVM plugin

2015-10-20 Thread Arnaldo Carvalho de Melo
Em Tue, Oct 20, 2015 at 05:49:22PM +0200, Paolo Bonzini escreveu:
> 
> 
> On 20/10/2015 17:48, Steven Rostedt wrote:
> > On Tue, 20 Oct 2015 17:37:43 +0200
> > Paolo Bonzini  wrote:
> > 
> >> However, it frankly seems a bit academic.  The parsing _will_ work,
> >> apart from printing a nonsensical role just like it has always done for
> >> the past four years.
> > 
> > I'm not going to be too picky about it. But things like this may seem
> > academic, but it's also what we forget about, and people still use
> > 2.6.32 kernels, and this may come back and bite us later.
> > 
> > But it's more likely to bite you than me, so if you don't think it's
> > worth while, then I'm not going to push it.
> 
> Thanks.  The role is not going to matter except in really weird cases
> and hopefully those are just not there in 2.6.32 kernels.  Thanks also
> for educating me so I won't get it wrong in the future!

Cool, Steven, can I take that as an Acked-by for this patch?

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] tools lib traceevent: update KVM plugin

2015-10-20 Thread Arnaldo Carvalho de Melo
Em Fri, Oct 09, 2015 at 10:10:13PM +0200, Paolo Bonzini escreveu:
> On 01/10/2015 12:28, Paolo Bonzini wrote:
> > The format of the role word has changed through the years and the
> > plugin was never updated; some VMX exit reasons were missing too.
> > 
> > Signed-off-by: Paolo Bonzini 
> > ---
> >  tools/lib/traceevent/plugin_kvm.c | 25 +
> >  1 file changed, 17 insertions(+), 8 deletions(-)
> > 
> > diff --git a/tools/lib/traceevent/plugin_kvm.c 
> > b/tools/lib/traceevent/plugin_kvm.c
> > index 88fe83dff7cd..18536f756577 100644
> > --- a/tools/lib/traceevent/plugin_kvm.c
> > +++ b/tools/lib/traceevent/plugin_kvm.c
> > @@ -124,7 +124,10 @@ static const char *disassemble(unsigned char *insn, 
> > int len, uint64_t rip,
> > _ER(WBINVD,  54)\
> > _ER(XSETBV,  55)\
> > _ER(APIC_WRITE,  56)\
> > -   _ER(INVPCID, 58)
> > +   _ER(INVPCID, 58)\
> > +   _ER(PML_FULL,62)\
> > +   _ER(XSAVES,  63)\
> > +   _ER(XRSTORS, 64)
> >  
> >  #define SVM_EXIT_REASONS \
> > _ER(EXIT_READ_CR0,  0x000)  \
> > @@ -352,15 +355,18 @@ static int kvm_nested_vmexit_handler(struct trace_seq 
> > *s, struct pevent_record *
> >  union kvm_mmu_page_role {
> > unsigned word;
> > struct {
> > -   unsigned glevels:4;
> > unsigned level:4;
> > +   unsigned cr4_pae:1;
> > unsigned quadrant:2;
> > -   unsigned pad_for_nice_hex_output:6;
> > unsigned direct:1;
> > unsigned access:3;
> > unsigned invalid:1;
> > -   unsigned cr4_pge:1;
> > unsigned nxe:1;
> > +   unsigned cr0_wp:1;
> > +   unsigned smep_and_not_wp:1;
> > +   unsigned smap_and_not_wp:1;
> > +   unsigned pad_for_nice_hex_output:8;
> > +   unsigned smm:8;
> > };
> >  };
> >  
> > @@ -385,15 +391,18 @@ static int kvm_mmu_print_role(struct trace_seq *s, 
> > struct pevent_record *record,
> > if (pevent_is_file_bigendian(event->pevent) ==
> > pevent_is_host_bigendian(event->pevent)) {
> >  
> > -   trace_seq_printf(s, "%u/%u q%u%s %s%s %spge %snxe",
> > +   trace_seq_printf(s, "%u q%u%s %s%s %spae %snxe %swp%s%s%s",
> >  role.level,
> > -role.glevels,
> >  role.quadrant,
> >  role.direct ? " direct" : "",
> >  access_str[role.access],
> >  role.invalid ? " invalid" : "",
> > -role.cr4_pge ? "" : "!",
> > -role.nxe ? "" : "!");
> > +role.cr4_pae ? "" : "!",
> > +role.nxe ? "" : "!",
> > +role.cr0_wp ? "" : "!",
> > +role.smep_and_not_wp ? " smep" : "",
> > +role.smap_and_not_wp ? " smap" : "",
> > +role.smm ? " smm" : "");
> > } else
> > trace_seq_printf(s, "WORD: %08x", role.word);
> >  
> > 
> 
> Ping?  Arnaldo, ok to include this patch in my tree?

Applying, I saw it before, but lost track, perhaps was waiting for
Steven Rostedt to chime in, but now I noticed he wasn't CCed, he is now.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[GIT PULL 00/33] perf/core improvements and fixes

2014-08-04 Thread Arnaldo Carvalho de Melo
Hi Ingo,

Mostly fixes plus some refactoring work, more to come as I process
Adrian Hunter big patch set and other that missed this pull request.

Please consider pulling,

- Arnaldo

The following changes since commit f9b9f812235d53f774a083e88a5a23b517a69752:

  Merge tag 'perf-core-for-mingo' of 
git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf into perf/core 
(2014-07-30 14:51:06 +0200)

are available in the git repository at:


  git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux.git 
tags/perf-core-for-mingo

for you to fetch changes up to 55b7e5fcf4782335c1ee5b4a2076697527c36407:

  perf tools: Default to python version 2 (2014-08-04 12:21:37 -0300)


perf/core improvements and fixes:

User visible:

. Show better error message in case we fail to open counters due to EBUSY error,
  for instance, when oprofile is running. (Jiri Olsa)

. Honour -w in the report tools (report, top), allowing to specify the widths
  for the histogram entries columns. (Namhyung Kim)

. Don't run workload if not told to, as happens when the user has no
  permission for profiling and even then the specified workload ends
  up running (Arnaldo Carvalho de Melo)

. Do not ignore mmap events in 'perf kmem report'. This tool was using
  the kernel mmaps in the running machine instead of processing the mmap
  records from the perf.data file. (Namhyung Kim)

. Properly show submicrosecond times in 'perf kvm stat' (Christian Borntraeger)

. Allow the user to disable time stamps in 'perf record' (Andi Kleen)

. Make sure --symfs usage includes the path separator (Arnaldo Carvalho de Melo)

Developer Stuff:

. Fix arm64 build error (Mark Salter)

. Fix make PYTHON override (Namhyung Kim)

. Rename ordered_samples to ordered_events and allow setting a queue
  size for ordering events (Jiri Olsa)

. Default to python version 2 (Thomas Ilsche)

Signed-off-by: Arnaldo Carvalho de Melo 


Andi Kleen (1):
  perf record: Allow the user to disable time stamps

Arnaldo Carvalho de Melo (2):
  perf evlist: Don't run workload if not told to
  perf symbols: Make sure --symfs usage includes the path separator

Christian Borntraeger (1):
  perf kvm stat: Properly show submicrosecond times

Jiri Olsa (18):
  perf tools: Rename ordered_samples bool to ordered_events
  perf tools: Rename ordered_samples struct to ordered_events
  perf tools: Rename ordered_events members
  perf tools: Add ordered_events__(new|delete) interface
  perf tools: Factor ordered_events__flush to be more generic
  perf tools: Limit ordered events queue size
  perf tools: Flush ordered events in case of allocation failure
  perf tools: Make perf_session__deliver_event global
  perf tools: Create ordered-events object
  perf tools: Use list_move in ordered_events_delete function
  perf tools: Add ordered_events__init function
  perf tools: Add ordered_events__free function
  perf tools: Add perf_config_u64 function
  perf tools: Add report.queue-size config file option
  perf tools: Add debug prints for ordered events queue
  perf tools: Allow out of order messages in forced flush
  perf tools: Show better error message in case we fail to open counters 
due to EBUSY error
  perf tools: Fix PERF_FLAG_FD_CLOEXEC flag probing event type open 
counters due to EBUSY error

Mark Salter (1):
  perf tools: Fix arm64 build error

Namhyung Kim (9):
  perf kmem: Do not ignore mmap events
  perf tools: Fix make PYTHON override
  perf tools: Left-align output contents
  perf tools: Make __hpp__fmt() receive an additional len argument
  perf tools: Save column length in perf_hpp_fmt
  perf report: Honor column width setting
  perf top: Add -w option for setting column width
  perf tools: Add name field into perf_hpp_fmt
  perf tools: Fix column alignment when headers aren't shown on TUI

Thomas Ilsche (1):
  perf tools: Default to python version 2

 tools/perf/Documentation/perf-report.txt  |   2 +-
 tools/perf/Documentation/perf-top.txt |   6 +
 tools/perf/Makefile.perf  |   2 +
 tools/perf/arch/arm64/include/perf_regs.h |   2 +
 tools/perf/builtin-annotate.c |   2 +-
 tools/perf/builtin-diff.c |   2 +-
 tools/perf/builtin-inject.c   |   2 +-
 tools/perf/builtin-kmem.c |   7 +-
 tools/perf/builtin-kvm.c  |  16 +-
 tools/perf/builtin-lock.c |   2 +-
 tools/perf/builtin-mem.c  |   2 +-
 tools/perf/builtin-record.c   |   1 +
 tools/perf/builtin-report.c   |  15 +-
 tools/perf/builtin-sched.c|   2 +-
 tools/perf/builtin-script.c   |   2 +-
 tools/perf/builtin-ti

[PATCH 04/33] perf kvm stat: Properly show submicrosecond times

2014-08-04 Thread Arnaldo Carvalho de Melo
From: Christian Borntraeger 

For lots of exits the min time (and sometimes max) is 0 or 1. Lets
increase the accurancy similar to what the average field alread does.

Signed-off-by: Christian Borntraeger 
Acked-by: David Ahern 
Cc: David Ahern 
Cc: Jiri Olsa 
Cc: Paolo Bonzini 
Cc: kvm@vger.kernel.org
Link: 
http://lkml.kernel.org/r/1406805231-10675-2-git-send-email-borntrae...@de.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo 
---
 tools/perf/builtin-kvm.c | 8 
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/tools/perf/builtin-kvm.c b/tools/perf/builtin-kvm.c
index 43367eb00510..fe92dfdeab46 100644
--- a/tools/perf/builtin-kvm.c
+++ b/tools/perf/builtin-kvm.c
@@ -592,8 +592,8 @@ static void print_result(struct perf_kvm_stat *kvm)
pr_info("%9s ", "Samples%");
 
pr_info("%9s ", "Time%");
-   pr_info("%10s ", "Min Time");
-   pr_info("%10s ", "Max Time");
+   pr_info("%11s ", "Min Time");
+   pr_info("%11s ", "Max Time");
pr_info("%16s ", "Avg time");
pr_info("\n\n");
 
@@ -610,8 +610,8 @@ static void print_result(struct perf_kvm_stat *kvm)
pr_info("%10llu ", (unsigned long long)ecount);
pr_info("%8.2f%% ", (double)ecount / kvm->total_count * 100);
pr_info("%8.2f%% ", (double)etime / kvm->total_time * 100);
-   pr_info("%8" PRIu64 "us ", min / 1000);
-   pr_info("%8" PRIu64 "us ", max / 1000);
+   pr_info("%9.2fus ", (double)min / 1e3);
+   pr_info("%9.2fus ", (double)max / 1e3);
pr_info("%9.2fus ( +-%7.2f%% )", (double)etime / ecount/1e3,
kvm_event_rel_stddev(vcpu, event));
pr_info("\n");
-- 
1.9.3

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH/RFC] KVM: perf/stat: Properly show submicrosecond times

2014-07-31 Thread Arnaldo Carvalho de Melo
Em Thu, Jul 31, 2014 at 08:24:03AM -0600, David Ahern escreveu:
> On 7/31/14, 5:13 AM, Christian Borntraeger wrote:
> >For lots of exits the min time (and sometimes max) is 0 or 1. Lets
> >increase the accurancy similar to what the average field alread does.
> 
> Seems reasonable to me.
> 
> Acked-by: David Ahern 

Thanks, applied.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 05/30] KVM: x86: Export svm/vmx exit code and vector code to userspace

2012-09-24 Thread Arnaldo Carvalho de Melo
From: Xiao Guangrong 

Exporting KVM exit information to userspace to be consumed by perf.

Signed-off-by: Dong Hao 
[ Dong Hao : rebase it on acme's git tree ]
Signed-off-by: Xiao Guangrong 
Acked-by: Marcelo Tosatti 
Cc: Avi Kivity 
Cc: David Ahern 
Cc: Ingo Molnar 
Cc: Marcelo Tosatti 
Cc: kvm@vger.kernel.org
Cc: Runzhen Wang 
Link: 
http://lkml.kernel.org/r/1347870675-31495-2-git-send-email-haod...@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo 
---
 arch/x86/include/asm/kvm.h  |   16 +++
 arch/x86/include/asm/kvm_host.h |   16 ---
 arch/x86/include/asm/svm.h  |  205 +--
 arch/x86/include/asm/vmx.h  |  127 
 arch/x86/kvm/trace.h|   89 -
 5 files changed, 230 insertions(+), 223 deletions(-)

diff --git a/arch/x86/include/asm/kvm.h b/arch/x86/include/asm/kvm.h
index 246617e..41e08cb 100644
--- a/arch/x86/include/asm/kvm.h
+++ b/arch/x86/include/asm/kvm.h
@@ -9,6 +9,22 @@
 #include 
 #include 
 
+#define DE_VECTOR 0
+#define DB_VECTOR 1
+#define BP_VECTOR 3
+#define OF_VECTOR 4
+#define BR_VECTOR 5
+#define UD_VECTOR 6
+#define NM_VECTOR 7
+#define DF_VECTOR 8
+#define TS_VECTOR 10
+#define NP_VECTOR 11
+#define SS_VECTOR 12
+#define GP_VECTOR 13
+#define PF_VECTOR 14
+#define MF_VECTOR 16
+#define MC_VECTOR 18
+
 /* Select x86 specific features in  */
 #define __KVM_HAVE_PIT
 #define __KVM_HAVE_IOAPIC
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 09155d6..1eaa6b0 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -75,22 +75,6 @@
 #define KVM_HPAGE_MASK(x)  (~(KVM_HPAGE_SIZE(x) - 1))
 #define KVM_PAGES_PER_HPAGE(x) (KVM_HPAGE_SIZE(x) / PAGE_SIZE)
 
-#define DE_VECTOR 0
-#define DB_VECTOR 1
-#define BP_VECTOR 3
-#define OF_VECTOR 4
-#define BR_VECTOR 5
-#define UD_VECTOR 6
-#define NM_VECTOR 7
-#define DF_VECTOR 8
-#define TS_VECTOR 10
-#define NP_VECTOR 11
-#define SS_VECTOR 12
-#define GP_VECTOR 13
-#define PF_VECTOR 14
-#define MF_VECTOR 16
-#define MC_VECTOR 18
-
 #define SELECTOR_TI_MASK (1 << 2)
 #define SELECTOR_RPL_MASK 0x03
 
diff --git a/arch/x86/include/asm/svm.h b/arch/x86/include/asm/svm.h
index f2b83bc..cdf5674 100644
--- a/arch/x86/include/asm/svm.h
+++ b/arch/x86/include/asm/svm.h
@@ -1,6 +1,135 @@
 #ifndef __SVM_H
 #define __SVM_H
 
+#define SVM_EXIT_READ_CR0  0x000
+#define SVM_EXIT_READ_CR3  0x003
+#define SVM_EXIT_READ_CR4  0x004
+#define SVM_EXIT_READ_CR8  0x008
+#define SVM_EXIT_WRITE_CR0 0x010
+#define SVM_EXIT_WRITE_CR3 0x013
+#define SVM_EXIT_WRITE_CR4 0x014
+#define SVM_EXIT_WRITE_CR8 0x018
+#define SVM_EXIT_READ_DR0  0x020
+#define SVM_EXIT_READ_DR1  0x021
+#define SVM_EXIT_READ_DR2  0x022
+#define SVM_EXIT_READ_DR3  0x023
+#define SVM_EXIT_READ_DR4  0x024
+#define SVM_EXIT_READ_DR5  0x025
+#define SVM_EXIT_READ_DR6  0x026
+#define SVM_EXIT_READ_DR7  0x027
+#define SVM_EXIT_WRITE_DR0 0x030
+#define SVM_EXIT_WRITE_DR1 0x031
+#define SVM_EXIT_WRITE_DR2 0x032
+#define SVM_EXIT_WRITE_DR3 0x033
+#define SVM_EXIT_WRITE_DR4 0x034
+#define SVM_EXIT_WRITE_DR5 0x035
+#define SVM_EXIT_WRITE_DR6 0x036
+#define SVM_EXIT_WRITE_DR7 0x037
+#define SVM_EXIT_EXCP_BASE 0x040
+#define SVM_EXIT_INTR  0x060
+#define SVM_EXIT_NMI   0x061
+#define SVM_EXIT_SMI   0x062
+#define SVM_EXIT_INIT  0x063
+#define SVM_EXIT_VINTR 0x064
+#define SVM_EXIT_CR0_SEL_WRITE 0x065
+#define SVM_EXIT_IDTR_READ 0x066
+#define SVM_EXIT_GDTR_READ 0x067
+#define SVM_EXIT_LDTR_READ 0x068
+#define SVM_EXIT_TR_READ   0x069
+#define SVM_EXIT_IDTR_WRITE0x06a
+#define SVM_EXIT_GDTR_WRITE0x06b
+#define SVM_EXIT_LDTR_WRITE0x06c
+#define SVM_EXIT_TR_WRITE  0x06d
+#define SVM_EXIT_RDTSC 0x06e
+#define SVM_EXIT_RDPMC 0x06f
+#define SVM_EXIT_PUSHF 0x070
+#define SVM_EXIT_POPF  0x071
+#define SVM_EXIT_CPUID 0x072
+#define SVM_EXIT_RSM   0x073
+#define SVM_EXIT_IRET  0x074
+#define SVM_EXIT_SWINT 0x075
+#define SVM_EXIT_INVD  0x076
+#define SVM_EXIT_PAUSE 0x077
+#define SVM_EXIT_HLT   0x078
+#define SVM_EXIT_INVLPG0x079
+#define SVM_EXIT_INVLPGA   0x07a
+#define SVM_EXIT_IOIO  0x07b
+#define SVM_EXIT_MSR   0x07c
+#define SVM_EXIT_TASK_SWITCH   0x07d
+#define SVM_EXIT_FERR_FREEZE   0x07e
+#define SVM_EXIT_SHUTDOWN  0x07f
+#define SVM_EXIT_VMRUN 0x080
+#define SVM_EXIT_VMMCALL   0x081
+#define SVM_EXIT_VMLOAD0x082
+#define SVM_EXIT_VMSAVE0x083
+#define SVM_EXIT_STGI  0x084
+#define SVM_EXIT_CLGI  0x085
+#define SVM_EXIT_SKINIT0x086
+#define SVM_EXIT_RDTSCP0x087
+#define SVM_EXIT_ICEBP 0x088
+#define SVM_EXIT_WBINVD0x089
+#define SVM_EXIT_MONITOR   0x08a
+#define SVM

[PATCH 06/30] perf kvm: Events analysis tool

2012-09-24 Thread Arnaldo Carvalho de Melo
From: Xiao Guangrong 

Add 'perf kvm stat' support to analyze kvm vmexit/mmio/ioport smartly

Usage:
- kvm stat
  run a command and gather performance counter statistics, it is the alias of
  perf stat

- trace kvm events:
  perf kvm stat record, or, if other tracepoints are interesting as well, we
  can append the events like this:
  perf kvm stat record -e timer:* -a

  If many guests are running, we can track the specified guest by using -p or
  --pid, -a is used to track events generated by all guests.

- show the result:
  perf kvm stat report

The output example is following:
13005
13059

total 2 guests are running on the host

Then, track the guest whose pid is 13059:
^C[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.253 MB perf.data.guest (~11065 samples) ]

See the vmexit events:

Analyze events for all VCPUs:

 VM-EXITSamples  Samples% Time% Avg time

 APIC_ACCESS46070.55% 0.01% 22.44us ( +-   1.75% )
 HLT 9314.26%99.98% 832077.26us ( +-  10.42% )
  EXTERNAL_INTERRUPT 64 9.82% 0.00% 35.35us ( +-  14.21% )
   PENDING_INTERRUPT 24 3.68% 0.00%  9.29us ( +-  31.39% )
   CR_ACCESS  7 1.07% 0.00%  8.12us ( +-   5.76% )
  IO_INSTRUCTION  3 0.46% 0.00% 18.00us ( +-  11.79% )
   EXCEPTION_NMI  1 0.15% 0.00%  5.83us ( +-   -nan% )

Total Samples:652, Total events handled time:77396109.80us.

See the mmio events:

Analyze events for all VCPUs:

 MMIO AccessSamples  Samples% Time% Avg time

0xfee00380:W38784.31%79.28%  8.29us ( +-   3.32% )
0xfee00300:W 24 5.23% 9.96% 16.79us ( +-   1.97% )
0xfee00300:R 24 5.23% 7.83% 13.20us ( +-   3.00% )
0xfee00310:W 24 5.23% 2.93%  4.94us ( +-   3.84% )

Total Samples:459, Total events handled time:4044.59us.

See the ioport event:

Analyze events for all VCPUs:

  IO Port AccessSamples  Samples% Time% Avg time

 0xc050:POUT  3   100.00%   100.00% 13.75us ( +-  10.83% )

Total Samples:3, Total events handled time:41.26us.

And, --vcpu is used to track the specified vcpu and --key is used to sort the
result:

Analyze events for VCPU 0:

 VM-EXITSamples  Samples% Time% Avg time

 HLT 2713.85%99.97% 405790.24us ( +-  12.70% )
  EXTERNAL_INTERRUPT 13 6.67% 0.00% 27.94us ( +-  22.26% )
 APIC_ACCESS14674.87% 0.03% 21.69us ( +-   2.91% )
  IO_INSTRUCTION  2 1.03% 0.00% 17.77us ( +-  20.56% )
   CR_ACCESS  2 1.03% 0.00%  8.55us ( +-   6.47% )
   PENDING_INTERRUPT  5 2.56% 0.00%  6.27us ( +-   3.94% )

Total Samples:195, Total events handled time:10959950.90us.

Signed-off-by: Dong Hao 
Signed-off-by: Runzhen Wang 
[ Dong Hao 
  Runzhen Wang :
 - rebase it on current acme's tree
 - fix the compiling-error on i386 ]
Signed-off-by: Xiao Guangrong 
Acked-by: David Ahern 
Cc: Avi Kivity 
Cc: David Ahern 
Cc: Ingo Molnar 
Cc: Marcelo Tosatti 
Cc: kvm@vger.kernel.org
Cc: Runzhen Wang 
Link: 
http://lkml.kernel.org/r/1347870675-31495-4-git-send-email-haod...@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo 
---
 tools/perf/Documentation/perf-kvm.txt |   30 ++-
 tools/perf/MANIFEST   |3 +
 tools/perf/builtin-kvm.c  |  840 -
 tools/perf/util/header.c  |   59 +++-
 tools/perf/util/header.h  |1 +
 tools/perf/util/thread.h  |2 +
 6 files changed, 929 insertions(+), 6 deletions(-)

diff --git a/tools/perf/Documentation/perf-kvm.txt 
b/tools/perf/Documentation/perf-kvm.txt
index dd84cb2..326f2cb 100644
--- a/tools/perf/Documentation/perf-kvm.txt
+++ b/tools/perf/Documentation/perf-kvm.txt
@@ -12,7 +12,7 @@ SYNOPSIS
[--guestkallsyms= --guestmodules= | --guestvmlinux=]]
{top|record|report|diff|buildid-list}
 'perf kvm' [--host] [--guest] [--guestkallsyms= --guestmodules=
-   | --guestvmlinux=] {top|record|report|diff|buildid-list}
+   | --guestvmlinux=] {top|record|report|diff|buildid-list|stat}
 
 DESCRIPTION
 ---
@@ -38,6 +38,18 @@ There are a couple of variants of perf kvm:
   so that other tools can be used to fetch packages with matching symbol tables
   for use by perf report.
 
+  'perf kvm stat ' to run a command and gather performance counter
+  statistics.
+  Especially, perf 'kvm stat record/report' generates a statistical analysis
+  of KVM events. Currently, vmexit, mmio and ioport events are supported.
+  'perf kvm stat record ' records kvm events and the events between
+  start and e

[GIT PULL 00/30] perf/core improvements and fixes

2012-09-24 Thread Arnaldo Carvalho de Melo
Hi Ingo,

Please consider pulling,

- Arnaldo

The following changes since commit 1e6dd8adc78d4a153db253d051fd4ef6c49c9019:

  perf: Fix off by one test in perf_reg_value() (2012-09-19 17:08:40 +0200)

are available in the git repository at:

  git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux 
tags/perf-core-for-mingo

for you to fetch changes up to b1ac754b67b5a875d63bee880f60ccb0c6bd8899:

  tools lib traceevent: Handle alloc_arg failure (2012-09-24 12:31:52 -0300)


perf/core improvements and fixes:

. Convert the trace builtins to use the growing evsel/evlist
  tracepoint infrastructure, removing several open coded constructs
  like switch like series of strcmp to dispatch events, etc.
  Basically what had already been showcased in 'perf sched'.

. Add evsel constructor for tracepoints, that uses libtraceevent
  just to parse the /format events file, use it in a new 'perf test'
  to make sure the libtraceevent format parsing regressions can
  be more readily caught.

. Some strange errors were happening in some builds, but not on the
  next, reported by several people, problem was some parser related
  files, generated during the build, didn't had proper make deps,
  fix from Eric Sandeen.

. Fix some compiling errors on 32-bit, from Feng Tang.

. Don't use sscanf extension %as, not available on bionic, reimplementation
  by Irina Tirdea.

. Fix bfd.h/libbfd detection with recent binutils, from Markus Trippelsdorf.

. Introduce struct and cache information about the environment where a
  perf.data file was captured, from Namhyung Kim.

. Fix several error paths in libtraceevent, from Namhyung Kim.

  Print event causing perf_event_open() to fail in 'perf record',
  from Stephane Eranian.

. New 'kvm' analysis tool, from Xiao Guangrong.

Signed-off-by: Arnaldo Carvalho de Melo 

------------
Arnaldo Carvalho de Melo (11):
  perf kvm: Use perf_evsel__intval
  perf kmem: Use perf_evsel__intval and 
perf_session__set_tracepoints_handlers
  perf lock: Use perf_evsel__intval and 
perf_session__set_tracepoints_handlers
  perf timechart: Use zalloc and fix a couple leaks
  tools lib traceevent: Use asprintf were applicable
  tools lib traceevent: Use calloc were applicable
  tools lib traceevent: Fix afterlife gotos
  tools lib traceevent: Remove some die() calls
  tools lib traceevent: Carve out events format parsing routine
  perf evsel: Provide a new constructor for tracepoints
  perf test: Add test for the sched tracepoint format fields

Eric Sandeen (1):
  perf tools: Fix parallel build

Feng Tang (2):
  perf tools: Fix a compiling error in trace-event-perl.c for 32 bits 
machine
  perf tools: Fix a compiling error in util/map.c

Irina Tirdea (1):
  perf tools: remove sscanf extension %as

Markus Trippelsdorf (1):
  perf tools: bfd.h/libbfd detection fails with recent binutils

Namhyung Kim (11):
  perf header: Add struct perf_session_env
  perf header: Add ->process callbacks to most of features
  perf header: Use pre-processed session env when printing
  perf header: Remove unused @feat arg from ->process callback
  perf kvm: Use perf_session_env for reading cpuid
  perf header: Remove perf_header__read_feature
  tools lib traceevent: Fix error path on process_array()
  tools lib traceevent: Make sure that arg->op.right is set properly
  tools lib traceevent: Free field if an error occurs on process_fields
  tools lib traceevent: Free field if an error occurs on 
process_flags/symbols
  tools lib traceevent: Handle alloc_arg failure

Stephane Eranian (1):
  perf record: Print event causing perf_event_open() to fail

Xiao Guangrong (2):
  KVM: x86: Export svm/vmx exit code and vector code to userspace
  perf kvm: Events analysis tool

 arch/x86/include/asm/kvm.h |   16 +
 arch/x86/include/asm/kvm_host.h|   16 -
 arch/x86/include/asm/svm.h |  205 +++--
 arch/x86/include/asm/vmx.h |  127 ++-
 arch/x86/kvm/trace.h   |   89 ---
 tools/lib/traceevent/event-parse.c |  570 +
 tools/lib/traceevent/event-parse.h |3 +
 tools/perf/Documentation/perf-kvm.txt  |   30 +-
 tools/perf/MANIFEST|3 +
 tools/perf/Makefile|6 +-
 tools/perf/builtin-kmem.c  |   90 +--
 tools/perf/builtin-kvm.c   |  836 +++-
 tools/perf/builtin-lock.c  |  233 ++
 tools/perf/builtin-record.c|6 +-
 tools/perf/builtin-test.c  |   86 ++
 tools/perf/bu

[PATCH 09/18] perf stat: Move stats related code to util/stat.c

2012-09-17 Thread Arnaldo Carvalho de Melo
From: Xiao Guangrong 

Then, the code can be shared between kvm events and perf stat.

Signed-off-by: Xiao Guangrong 
[ Dong Hao : rebase it on acme's git tree ]
Signed-off-by: Dong Hao 
Cc: Avi Kivity 
Cc: David Ahern 
Cc: Ingo Molnar 
Cc: kvm@vger.kernel.org
Cc: Marcelo Tosatti 
Cc: Runzhen Wang 
Cc: Xiao Guangrong http://lkml.kernel.org/r/1347870675-31495-3-git-send-email-haod...@linux.vnet.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo 
---
 tools/perf/Makefile   |1 +
 tools/perf/builtin-stat.c |   56 +--
 tools/perf/util/stat.c|   57 +
 tools/perf/util/stat.h|   16 
 4 files changed, 76 insertions(+), 54 deletions(-)
 create mode 100644 tools/perf/util/stat.c
 create mode 100644 tools/perf/util/stat.h

diff --git a/tools/perf/Makefile b/tools/perf/Makefile
index 209774b..5077f8e 100644
--- a/tools/perf/Makefile
+++ b/tools/perf/Makefile
@@ -406,6 +406,7 @@ LIB_OBJS += $(OUTPUT)util/target.o
 LIB_OBJS += $(OUTPUT)util/rblist.o
 LIB_OBJS += $(OUTPUT)util/intlist.o
 LIB_OBJS += $(OUTPUT)util/vdso.o
+LIB_OBJS += $(OUTPUT)util/stat.o
 
 LIB_OBJS += $(OUTPUT)ui/helpline.o
 LIB_OBJS += $(OUTPUT)ui/hist.o
diff --git a/tools/perf/builtin-stat.c b/tools/perf/builtin-stat.c
index dab347d..3c43a35 100644
--- a/tools/perf/builtin-stat.c
+++ b/tools/perf/builtin-stat.c
@@ -51,13 +51,13 @@
 #include "util/evsel.h"
 #include "util/debug.h"
 #include "util/color.h"
+#include "util/stat.h"
 #include "util/header.h"
 #include "util/cpumap.h"
 #include "util/thread.h"
 #include "util/thread_map.h"
 
 #include 
-#include 
 #include 
 
 #define DEFAULT_SEPARATOR  " "
@@ -199,11 +199,6 @@ static int output_fd;
 
 static volatile int done = 0;
 
-struct stats
-{
-   double n, mean, M2;
-};
-
 struct perf_stat {
struct stats  res_stats[3];
 };
@@ -220,50 +215,6 @@ static void perf_evsel__free_stat_priv(struct perf_evsel 
*evsel)
evsel->priv = NULL;
 }
 
-static void update_stats(struct stats *stats, u64 val)
-{
-   double delta;
-
-   stats->n++;
-   delta = val - stats->mean;
-   stats->mean += delta / stats->n;
-   stats->M2 += delta*(val - stats->mean);
-}
-
-static double avg_stats(struct stats *stats)
-{
-   return stats->mean;
-}
-
-/*
- * http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance
- *
- *   (\Sum n_i^2) - ((\Sum n_i)^2)/n
- * s^2 = ---
- *  n - 1
- *
- * http://en.wikipedia.org/wiki/Stddev
- *
- * The std dev of the mean is related to the std dev by:
- *
- * s
- * s_mean = ---
- *  sqrt(n)
- *
- */
-static double stddev_stats(struct stats *stats)
-{
-   double variance, variance_mean;
-
-   if (!stats->n)
-   return 0.0;
-
-   variance = stats->M2 / (stats->n - 1);
-   variance_mean = variance / stats->n;
-
-   return sqrt(variance_mean);
-}
-
 static struct stats runtime_nsecs_stats[MAX_NR_CPUS];
 static struct stats runtime_cycles_stats[MAX_NR_CPUS];
 static struct stats runtime_stalled_cycles_front_stats[MAX_NR_CPUS];
@@ -559,10 +510,7 @@ static int run_perf_stat(int argc __maybe_unused, const 
char **argv)
 
 static void print_noise_pct(double total, double avg)
 {
-   double pct = 0.0;
-
-   if (avg)
-   pct = 100.0*total/avg;
+   double pct = rel_stddev_stats(total, avg);
 
if (csv_output)
fprintf(output, "%s%.2f%%", csv_sep, pct);
diff --git a/tools/perf/util/stat.c b/tools/perf/util/stat.c
new file mode 100644
index 000..2374212
--- /dev/null
+++ b/tools/perf/util/stat.c
@@ -0,0 +1,57 @@
+#include 
+
+#include "stat.h"
+
+void update_stats(struct stats *stats, u64 val)
+{
+   double delta;
+
+   stats->n++;
+   delta = val - stats->mean;
+   stats->mean += delta / stats->n;
+   stats->M2 += delta*(val - stats->mean);
+}
+
+double avg_stats(struct stats *stats)
+{
+   return stats->mean;
+}
+
+/*
+ * http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance
+ *
+ *   (\Sum n_i^2) - ((\Sum n_i)^2)/n
+ * s^2 = ---
+ *  n - 1
+ *
+ * http://en.wikipedia.org/wiki/Stddev
+ *
+ * The std dev of the mean is related to the std dev by:
+ *
+ * s
+ * s_mean = ---
+ *  sqrt(n)
+ *
+ */
+double stddev_stats(struct stats *stats)
+{
+   double variance, variance_mean;
+
+   if (!stats->n)
+   return 0.0;
+
+   variance = stats->M2 / (stats->n - 1);
+   variance_mean = variance / stats->n;
+
+   return sqrt(variance_mean);
+}
+
+double rel_stddev_stats(double stddev, double avg)
+{
+   double pct = 0.0;
+
+   if (avg)
+   pct = 100.0 * 

Re: [PATCH v8 1/3] KVM: x86: export svm/vmx exit code and vector code to userspace

2012-09-17 Thread Arnaldo Carvalho de Melo
Em Mon, Sep 17, 2012 at 04:31:13PM +0800, Dong Hao escreveu:
> From: Xiao Guangrong 
> 
> Exporting KVM exit information to userspace to be consumed by perf.
> 
> [ Dong Hao : rebase it on acme's git tree ]
> Signed-off-by: Dong Hao 
> Signed-off-by: Xiao Guangrong 

Do we have acked/reviewed-by for this parth? Marcelo? Avi?

>  arch/x86/include/asm/kvm.h  |   16 +++
>  arch/x86/include/asm/kvm_host.h |   16 ---
>  arch/x86/include/asm/svm.h  |  205 
> +--
>  arch/x86/include/asm/vmx.h  |  127 
>  arch/x86/kvm/trace.h|   89 -
>  5 files changed, 230 insertions(+), 223 deletions(-)
> 
> diff --git a/arch/x86/include/asm/kvm.h b/arch/x86/include/asm/kvm.h
> index 246617e..41e08cb 100644
> --- a/arch/x86/include/asm/kvm.h
> +++ b/arch/x86/include/asm/kvm.h
> @@ -9,6 +9,22 @@
>  #include 
>  #include 
>  
> +#define DE_VECTOR 0
> +#define DB_VECTOR 1
> +#define BP_VECTOR 3
> +#define OF_VECTOR 4
> +#define BR_VECTOR 5
> +#define UD_VECTOR 6
> +#define NM_VECTOR 7
> +#define DF_VECTOR 8
> +#define TS_VECTOR 10
> +#define NP_VECTOR 11
> +#define SS_VECTOR 12
> +#define GP_VECTOR 13
> +#define PF_VECTOR 14
> +#define MF_VECTOR 16
> +#define MC_VECTOR 18
> +
>  /* Select x86 specific features in  */
>  #define __KVM_HAVE_PIT
>  #define __KVM_HAVE_IOAPIC
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index 09155d6..1eaa6b0 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -75,22 +75,6 @@
>  #define KVM_HPAGE_MASK(x)(~(KVM_HPAGE_SIZE(x) - 1))
>  #define KVM_PAGES_PER_HPAGE(x)   (KVM_HPAGE_SIZE(x) / PAGE_SIZE)
>  
> -#define DE_VECTOR 0
> -#define DB_VECTOR 1
> -#define BP_VECTOR 3
> -#define OF_VECTOR 4
> -#define BR_VECTOR 5
> -#define UD_VECTOR 6
> -#define NM_VECTOR 7
> -#define DF_VECTOR 8
> -#define TS_VECTOR 10
> -#define NP_VECTOR 11
> -#define SS_VECTOR 12
> -#define GP_VECTOR 13
> -#define PF_VECTOR 14
> -#define MF_VECTOR 16
> -#define MC_VECTOR 18
> -
>  #define SELECTOR_TI_MASK (1 << 2)
>  #define SELECTOR_RPL_MASK 0x03
>  
> diff --git a/arch/x86/include/asm/svm.h b/arch/x86/include/asm/svm.h
> index f2b83bc..cdf5674 100644
> --- a/arch/x86/include/asm/svm.h
> +++ b/arch/x86/include/asm/svm.h
> @@ -1,6 +1,135 @@
>  #ifndef __SVM_H
>  #define __SVM_H
>  
> +#define SVM_EXIT_READ_CR0  0x000
> +#define SVM_EXIT_READ_CR3  0x003
> +#define SVM_EXIT_READ_CR4  0x004
> +#define SVM_EXIT_READ_CR8  0x008
> +#define SVM_EXIT_WRITE_CR0 0x010
> +#define SVM_EXIT_WRITE_CR3 0x013
> +#define SVM_EXIT_WRITE_CR4 0x014
> +#define SVM_EXIT_WRITE_CR8 0x018
> +#define SVM_EXIT_READ_DR0  0x020
> +#define SVM_EXIT_READ_DR1  0x021
> +#define SVM_EXIT_READ_DR2  0x022
> +#define SVM_EXIT_READ_DR3  0x023
> +#define SVM_EXIT_READ_DR4  0x024
> +#define SVM_EXIT_READ_DR5  0x025
> +#define SVM_EXIT_READ_DR6  0x026
> +#define SVM_EXIT_READ_DR7  0x027
> +#define SVM_EXIT_WRITE_DR0 0x030
> +#define SVM_EXIT_WRITE_DR1 0x031
> +#define SVM_EXIT_WRITE_DR2 0x032
> +#define SVM_EXIT_WRITE_DR3 0x033
> +#define SVM_EXIT_WRITE_DR4 0x034
> +#define SVM_EXIT_WRITE_DR5 0x035
> +#define SVM_EXIT_WRITE_DR6 0x036
> +#define SVM_EXIT_WRITE_DR7 0x037
> +#define SVM_EXIT_EXCP_BASE 0x040
> +#define SVM_EXIT_INTR  0x060
> +#define SVM_EXIT_NMI   0x061
> +#define SVM_EXIT_SMI   0x062
> +#define SVM_EXIT_INIT  0x063
> +#define SVM_EXIT_VINTR 0x064
> +#define SVM_EXIT_CR0_SEL_WRITE 0x065
> +#define SVM_EXIT_IDTR_READ 0x066
> +#define SVM_EXIT_GDTR_READ 0x067
> +#define SVM_EXIT_LDTR_READ 0x068
> +#define SVM_EXIT_TR_READ   0x069
> +#define SVM_EXIT_IDTR_WRITE0x06a
> +#define SVM_EXIT_GDTR_WRITE0x06b
> +#define SVM_EXIT_LDTR_WRITE0x06c
> +#define SVM_EXIT_TR_WRITE  0x06d
> +#define SVM_EXIT_RDTSC 0x06e
> +#define SVM_EXIT_RDPMC 0x06f
> +#define SVM_EXIT_PUSHF 0x070
> +#define SVM_EXIT_POPF  0x071
> +#define SVM_EXIT_CPUID 0x072
> +#define SVM_EXIT_RSM   0x073
> +#define SVM_EXIT_IRET  0x074
> +#define SVM_EXIT_SWINT 0x075
> +#define SVM_EXIT_INVD  0x076
> +#define SVM_EXIT_PAUSE 0x077
> +#define SVM_EXIT_HLT   0x078
> +#define SVM_EXIT_INVLPG0x079
> +#define SVM_EXIT_INVLPGA   0x07a
> +#define SVM_EXIT_IOIO  0x07b
> +#define SVM_EXIT_MSR   0x07c
> +#define SVM_EXIT_TASK_SWITCH   0x07d
> +#define SVM_EXIT_FERR_FREEZE   0x07e
> +#define SVM_EXIT_SHUTDOWN  0x07f
> +#define SVM_EXIT_VMRUN 0x080
> +#define SVM_EXIT_VMMCALL   0x081
> +#define SVM_EXIT_VMLOAD0x082
> +#define SVM_EXIT_VMSAVE0x083
> +#define SVM_EXIT_STGI  0x084
> +#define SVM_EXIT_CLGI  0x085
> +#define SVM_EXIT_SKINIT0x086
> +#define SVM_EXIT_RDTSCP0x087
> +#define SVM_EXI

Re: [PATCH v7 3/3] KVM: perf: kvm events analysis tool

2012-09-13 Thread Arnaldo Carvalho de Melo
Em Thu, Sep 13, 2012 at 08:14:37AM -0600, David Ahern escreveu:
> On 9/13/12 7:45 AM, Arnaldo Carvalho de Melo wrote:
> >Em Wed, Sep 12, 2012 at 10:56:44PM -0600, David Ahern escreveu:

> >>So, I spent a fair amount of time today implementing a live mode.
> >>And after a lot of swearing at the tracepoint processing code I

> >What kind of swearing? I'm working on 'perf test' entries for
> >tracepoints to make sure we don't regress on the perf/libtraceevent
> >junction, doing that as prep work for further simplifying tracepoint
> >tools like sched, kvm, kmem, etc.
 
> Have you seen how the tracing initialization is done? ugly. record
> generates tracing data events and report uses those to do the init
> so you can access the raw_data. I ended up writing this:

And all we need is the list of fields so that we can use
perf_evsel__{int,str}val like I did in my 'perf sched' patch series (in
my perf/core branch), and even those accessors I'll tweak some more as
we don't need to check the endianness of the events, its in the same
machine, etc.

I'm trying to get by without using a 'pevent' just using 'event_format',
its doable when everything is local, as a single machine top tool is.

I want to just create the tracepoint events and process them like in
'top', using code more or less like what is in test__PERF_RECORD.

This still needs more work, so I think you can continue in your path and
eventually we'll have infrastructure to do it the way I'm describing,
optimizing the case where the "record" and "top" are in the same
machine, i.e. a short circuited 'live mode' with the top machinery
completely reused for tools, be it written in C, like 'sched', 'kvm',
'kmem', etc, or in perl or python.

- Arnaldo
 
> static int perf_kvm__tracing_init(void)
> {
> struct tracing_data *tdata;
> char temp_file[] = "/tmp/perf-";
> int fd;
> 
> fd = mkstemp(temp_file);
> if (fd < 0) {
> pr_err("mkstemp failed\n");
> return -1;
> }
> unlink(temp_file);
> 
> tdata = tracing_data_get(&kvm_events.evlist->entries, fd, false);
> if (!tdata)
> return -1;
> lseek(fd, 0, SEEK_SET);
> (void) trace_report(fd, &kvm_events.session->pevent, false);
> tracing_data_put(tdata);
> 
> return 0;
> }
> 
> 
> >
> >>finally have it working. And the format extends easily (meaning <
> >>day and the next step) to a perf-based kvm_stat replacement. Example
> >>syntax is:
> >>
> >>perf kvm stat [-p |-a|...]
> >>
> >>which defaults to an update delay of 1 second, and vmexit analysis.
> >>
> >>The guts of the processing logic come from the existing kvm-events
> >>code. The changes focus on combining the record and report paths
> >>into one. The display needs some help (Arnaldo?), but it seems to
> >>work well.
> >>
> >>I'd like to get opinions on what next? IMO, the record/report path
> >>should not get a foot hold from a backward compatibility perspective
> >>and having to maintain those options. I am willing to take the
> >>existing patches into git to maintain authorship and from there
> >>apply patches to make the live mode work - which includes a bit of
> >>refactoring of perf code (like the stats changes).
> >>
> >>Before I march down this path, any objections, opinions, etc?
> >
> >Can I see the code?
> 
> Let me clean it up over the weekend and send out an RFC for it.
> 
> David
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v7 3/3] KVM: perf: kvm events analysis tool

2012-09-13 Thread Arnaldo Carvalho de Melo
Em Wed, Sep 12, 2012 at 10:56:44PM -0600, David Ahern escreveu:
> >>  static const char * const kvm_usage[] = {
> >>+"perf kvm [] {top|record|report|diff|buildid-list|stat}",

> >The usage for the report/record sub commands of stat is never shown. e.g.,
> >$ perf kvm stat
> >--> shows help for perf-stat

> >$ perf kvm
> >--> shows the above and perf-kvm's usage
> 
> [I deleted this thread, so having to reply to one of my responses.
> hopefully noone is unduly harmed by this.]
> 
> I've been using this command a bit lately -- especially on nested
> virtualization -- and I think the syntax is quirky - meaning wrong.
> In my case I always follow up a record with a report and end up
> using a shell script wrapper that combines the 2 and running it
> repeatedly. e.g.,
> 
> $PERF kvm stat record -o $FILE -p $pid -- sleep $time
> [ $? -eq 0 ] && $PERF --no-pager kvm  -i $FILE stat report
> 
> As my daughter likes to say - awkward.
> 
> That suggests what is really needed is a 'live' mode - a continual
> updating of the output like perf top, not a record and analyze later
> mode. Which does come back to why I responded to this email -- the
> syntax is klunky and awkward.
> 
> So, I spent a fair amount of time today implementing a live mode.
> And after a lot of swearing at the tracepoint processing code I

What kind of swearing? I'm working on 'perf test' entries for
tracepoints to make sure we don't regress on the perf/libtraceevent
junction, doing that as prep work for further simplifying tracepoint
tools like sched, kvm, kmem, etc.

> finally have it working. And the format extends easily (meaning <
> day and the next step) to a perf-based kvm_stat replacement. Example
> syntax is:
> 
>perf kvm stat [-p |-a|...]
> 
> which defaults to an update delay of 1 second, and vmexit analysis.
> 
> The guts of the processing logic come from the existing kvm-events
> code. The changes focus on combining the record and report paths
> into one. The display needs some help (Arnaldo?), but it seems to
> work well.
> 
> I'd like to get opinions on what next? IMO, the record/report path
> should not get a foot hold from a backward compatibility perspective
> and having to maintain those options. I am willing to take the
> existing patches into git to maintain authorship and from there
> apply patches to make the live mode work - which includes a bit of
> refactoring of perf code (like the stats changes).
> 
> Before I march down this path, any objections, opinions, etc?

Can I see the code?

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v6 3/3] KVM: perf kvm events analysis tool

2012-08-10 Thread Arnaldo Carvalho de Melo
Em Fri, Aug 10, 2012 at 11:19:10AM +0800, Dong Hao escreveu:
> From: Xiao Guangrong 
> 
> Add 'perf kvm stat' support to analyze kvm vmexit/mmio/ioport smartly.

Some comments below about recent changes in my perf/core branch.
> +static void process_raw_event(struct thread *thread, void *data, u64 
> timestamp)
> +{
> + struct event_format *event;
> + int type;
> +
> + type = trace_parse_common_type(kvm_session->pevent, data);

type can be found in evsel->attr.config

> + event = pevent_find_event(kvm_session->pevent, type);

event in evsel->tp_format, i.e. no need to relookup via
pevent_find_event.

> + return handle_kvm_event(thread, event, data, timestamp);
> +}
> +
> +static int process_sample_event(struct perf_tool *tool __used,
> + union perf_event *event,
> + struct perf_sample *sample,
> + struct perf_evsel *evsel __used,
> + struct machine *machine)
> +{
> + struct thread *thread = machine__findnew_thread(machine, sample->tid);
> +
> + if (thread == NULL) {
> + pr_debug("problem processing %d event, skipping it.\n",
> + event->header.type);
> + return -1;
> + }
> +
> + process_raw_event(thread, sample->raw_data, sample->time);
> +
> + return 0;
> +}
> +
> +static struct perf_tool eops = {
> + .sample = process_sample_event,
> + .comm   = perf_event__process_comm,
> + .ordered_samples= true,
> +};
> +
> +static int get_cpu_isa(struct perf_session *session)
> +{
> + char *cpuid;
> + int isa;
> +
> + cpuid = perf_header__read_feature(session, HEADER_CPUID);
> +
> + if (!cpuid)
> + die("read HEADER_CPUID failed.\n");

Please don't use die, use pr_err and return -1, so that whatever cleanup
code the main perf tool needs to do gets done.

> + if (strstr(cpuid, "Intel"))
> + isa = 1;
> + else if (strstr(cpuid, "AMD"))
> + isa = 0;
> + else
> + die("CPU %s is not supported.\n", cpuid);

Ditto

> + free(cpuid);
> + return isa;
> +}
> +
> +static const char *file_name;
> +
> +static int read_events(void)
> +{
> + kvm_session = perf_session__new(file_name, O_RDONLY, 0, false, &eops);
> + if (!kvm_session)
> + die("Initializing perf session failed\n");

Ditto

> + if (!perf_session__has_traces(kvm_session, "kvm record"))
> + return -1;
> +
> + /*
> +  * Do not use 'isa' recorded in kvm_exit tracepoint since it is not
> +  * traced in the old kernel.
> +  */
> + cpu_isa = get_cpu_isa(kvm_session);
> +
> + return perf_session__process_events(kvm_session, &eops);
> +}
> +
> +static void verify_vcpu(int vcpu)
> +{
> + if (vcpu != -1 && vcpu < 0)
> + die("Invalid vcpu:%d.\n", vcpu);

Ditto

> +}
> +
> +static int kvm_events_report_vcpu(int vcpu)
> +{
> + init_kvm_event_record();
> + verify_vcpu(vcpu);
> + select_key();
> + register_kvm_events_ops();
> + setup_pager();
> +
> + read_events();
> +
> + sort_result(vcpu);
> + print_result(vcpu);
> + return 0;
> +}
> +
> +static const char * const record_args[] = {
> + "record",
> + "-R",
> + "-f",
> + "-m", "1024",
> + "-c", "1",
> + "-e", "kvm:kvm_entry",
> + "-e", "kvm:kvm_exit",
> + "-e", "kvm:kvm_mmio",
> + "-e", "kvm:kvm_pio",
> +};
> +
> +static const char * const new_event[] = {
> + "kvm_mmio_begin",
> + "kvm_mmio_done"
> +};
> +
> +static bool kvm_events_exist(const char *event)
> +{
> + char evt_path[MAXPATHLEN];
> + int fd;
> +
> + snprintf(evt_path, MAXPATHLEN, "%s/kvm/%s/id", tracing_events_path,
> +  event);
> +
> + fd = open(evt_path, O_RDONLY);
> +
> + if (fd < 0)
> + return false;
> +
> + close(fd);
> +
> + return true;
> +}
> +
> +static bool kvm_record_specified_guest(int argc, const char **argv)
> +{
> + int i;
> +
> + for (i = 0; i < argc; i++)
> + if (!strcmp(argv[i], "-p") || !strcmp(argv[i], "--pid"))
> + return true;
> +
> + return false;
> +}
> +
> +static int kvm_events_record(int argc, const char **argv)
> +{
> + unsigned int rec_argc, i, j;
> + const char **rec_argv;
> +
> + rec_argc = ARRAY_SIZE(record_args) + argc + 2;
> + rec_argc += ARRAY_SIZE(new_event) * 2;
> + rec_argv = calloc(rec_argc + 1, sizeof(char *));
> +
> + if (rec_argv == NULL)
> + return -ENOMEM;

Just like you do here :-)

> + for (i = 0; i < ARRAY_SIZE(record_args); i++)
> + rec_argv[i] = strdup(record_args[i]);
> +
> + /*
> +  * Append "-a" only if "-p"/"--pid" is not specified since they
> +  * are mutually exclusive.
> +  */
> + if (!kvm_record_specified_guest(argc, argv))
> + rec_argv[i++] = strdup("-a");
> +
> +

Re: [PATCH 3/3] KVM: perf: kvm events analysis tool

2012-02-16 Thread Arnaldo Carvalho de Melo
Em Wed, Feb 15, 2012 at 10:05:08PM -0700, David Ahern escreveu:
> On 2/15/12 9:59 PM, Xiao Guangrong wrote:
> >
> >
> >Okay, i will post the next version after collecting your new comments!
> >
> >Thanks for your time, David! :)
> >
> 
> I had more comments, but got sidetracked and forgot to come back to
> this. I still haven't looked at the code yet, but some comments from
> testing:
> 
> 1. The error message:
>   Warning: Error: expected type 5 but read 4
>   Warning: Error: expected type 5 but read 0
>   Warning: unknown op '}'
> 
> is fixed by this patch which has not yet made its way into perf:
> https://lkml.org/lkml/2011/9/4/41
> 
> The most recent request:
> https://lkml.org/lkml/2012/2/8/479
> 
> Arnaldo: the patch still applies cleanly (but with an offset of -2 lines).

I'll merge this one now, recall Steven asked me to, thanks for the
reminder,

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 5/5] perf kvm: Fix copy & paste error in description

2012-01-03 Thread Arnaldo Carvalho de Melo
From: Joerg Roedel 

The --host option certainly enables host-data collection.

Cc: Avi Kivity 
Cc: Ingo Molnar 
Cc: Peter Zijlstra 
Cc: Joerg Roedel 
Cc: kvm@vger.kernel.org
Link: http://lkml.kernel.org/r/1317816084-18026-5-git-send-email-g...@redhat.com
Signed-off-by: Gleb Natapov 
Signed-off-by: Joerg Roedel 
Signed-off-by: Arnaldo Carvalho de Melo 
---
 tools/perf/builtin-kvm.c |2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/tools/perf/builtin-kvm.c b/tools/perf/builtin-kvm.c
index 34d1e85..032324a 100644
--- a/tools/perf/builtin-kvm.c
+++ b/tools/perf/builtin-kvm.c
@@ -38,7 +38,7 @@ static const struct option kvm_options[] = {
OPT_BOOLEAN(0, "guest", &perf_guest,
"Collect guest os data"),
OPT_BOOLEAN(0, "host", &perf_host,
-   "Collect guest os data"),
+   "Collect host os data"),
OPT_STRING(0, "guestmount", &symbol_conf.guestmount, "directory",
   "guest mount directory under which every guest os"
   " instance has a subdir"),
-- 
1.7.8.rc0.35.gee6df

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[GIT PULL 0/5] perf/core fixes and improvements

2012-01-03 Thread Arnaldo Carvalho de Melo
Hi Ingo,

Please consider pulling from:

git://github.com/acmel/linux.git perf/core

Regards,

- Arnaldo

Joerg Roedel (1):
  perf kvm: Fix copy & paste error in description

Namhyung Kim (4):
  perf session: Remove impossible condition check
  perf stat: Introduce get_ratio_color() helper
  perf top: Fix a memory leak
  perf script: Kill script_spec__delete

 tools/perf/builtin-kvm.c|2 +-
 tools/perf/builtin-script.c |   13 +--
 tools/perf/builtin-stat.c   |   91 --
 tools/perf/builtin-top.c|1 -
 tools/perf/util/session.c   |3 +-
 5 files changed, 38 insertions(+), 72 deletions(-)

-- 
1.7.8.rc0.35.gee6df

--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [F.A.Q.] the advantages of a shared tool/kernel Git repository, tools/perf/ and tools/kvm/

2011-11-09 Thread Arnaldo Carvalho de Melo
Em Wed, Nov 09, 2011 at 02:25:09PM -0500, Jim Paris escreveu:
> Arnaldo Carvalho de Melo wrote:
> > Em Wed, Nov 09, 2011 at 11:40:01AM +0100, Gerd Hoffmann escreveu:
> > > As far I know it is pretty much impossible to figure the
> > > foreground/background colors of the terminal you are running on.  You

> > Glad to hear that, I thought I hadn't researched that much (I did). Hope
> > somebody appears and tell us how it is done :-)

> In xterm, '\e]10;?\e\\' and '\e]11;?\e\\' will report the colors, e.g.:

> #!/bin/bash
>   read -s -r -d \\ -p `printf '\e]10;?\e\\'` -t 1 fg
>   [ $? -ne 0 ] && fg="no response"
>   echo "foreground: $fg" | cat -v
>   read -s -r -d \\ -p `printf '\e]11;?\e\\'` -t 1 bg
>   [ $? -ne 0 ] && bg="no response"
>   echo "background: $bg" | cat -v

gnome-terminal:

[acme@felicio ~]$ ./a.sh
foreground: no response
background: no response
[acme@felicio ~]$

:-(

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [F.A.Q.] the advantages of a shared tool/kernel Git repository, tools/perf/ and tools/kvm/

2011-11-09 Thread Arnaldo Carvalho de Melo
Em Wed, Nov 09, 2011 at 01:46:42PM +0100, Peter Zijlstra escreveu:
> On Wed, 2011-11-09 at 10:33 -0200, Arnaldo Carvalho de Melo wrote:
> > 
> > Ingo, would that G+ page be useful for that?
> > 
> *groan*
> 
> Can we please keep things sane?

ROFL, I had to ask that :-P

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [F.A.Q.] the advantages of a shared tool/kernel Git repository, tools/perf/ and tools/kvm/

2011-11-09 Thread Arnaldo Carvalho de Melo
Em Wed, Nov 09, 2011 at 10:30:50AM -0200, Arnaldo Carvalho de Melo escreveu:
> Em Wed, Nov 09, 2011 at 01:26:34PM +0100, Gerd Hoffmann escreveu:
> > > Its fully configurable as of now, what we need is a set of .perfconfigs
> > > that show how people think its better, we try it, set it as the default,
> > > leave the others in tools/perf/Documentation/perfconfig/color.examples.

> > Yep, a set of examples works too.

> > The colors are not fully configurable yet though.  First, when switching
> > all five colorsets to "default, default" there are still things which
> > are colored (top bar, bottom bar, keys help display).  Second there is
> > no way to set terminal attributes (i.e. "top = bold" or "selected =
> > reverse").

> Ok, adding those to my TODO list.

> /me goes to check if http://perf.wiki.kernel.org is back working so that
> we can have a _public_ TODO list, perhaps it may attract more
> contributors :)

Oops, there is one, utterly old tho ;-\

I tried changing that and adding this entry but:

https://perf.wiki.kernel.org/articles/u/s/e/Special~UserLogin_94cd.html

Returns:

The requested URL /articles/u/s/e/Special~UserLogin_94cd.html was not
found on this server.

Ingo, would that G+ page be useful for that?

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [F.A.Q.] the advantages of a shared tool/kernel Git repository, tools/perf/ and tools/kvm/

2011-11-09 Thread Arnaldo Carvalho de Melo
Em Wed, Nov 09, 2011 at 01:26:34PM +0100, Gerd Hoffmann escreveu:
>   Hi,
> 
> >>> Plus allowing full .perfconfig configurability of all the relevant 
> >>> colors, for those with special taste.
> >>
> >> Sure.  Maybe also allow multiple color sections and pick them by $TERM
> >> or --colors switch, i.e. [colors "xterm"].
> > 
> > Its fully configurable as of now, what we need is a set of .perfconfigs
> > that show how people think its better, we try it, set it as the default,
> > leave the others in tools/perf/Documentation/perfconfig/color.examples.
> 
> Yep, a set of examples works too.
> 
> The colors are not fully configurable yet though.  First, when switching
> all five colorsets to "default, default" there are still things which
> are colored (top bar, bottom bar, keys help display).  Second there is
> no way to set terminal attributes (i.e. "top = bold" or "selected =
> reverse").

Ok, adding those to the TODO list.

/me goes to check if http://perf.wiki.kernel.org is back working so that
we can have a _public_ TODO list, perhaps it may attract more
contributors :)

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [F.A.Q.] the advantages of a shared tool/kernel Git repository, tools/perf/ and tools/kvm/

2011-11-09 Thread Arnaldo Carvalho de Melo
Em Wed, Nov 09, 2011 at 10:21:09AM +0100, Ingo Molnar escreveu:
> Eventually someone will do the right thing and implement 'perf trace' 
> (there's still the tip:tmp.perf/trace2 prototype branch) and users 

I'm working on it, reworking its patches into the new evlist/evsel
abstractions, etc.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [F.A.Q.] the advantages of a shared tool/kernel Git repository, tools/perf/ and tools/kvm/

2011-11-09 Thread Arnaldo Carvalho de Melo
Em Wed, Nov 09, 2011 at 11:40:01AM +0100, Gerd Hoffmann escreveu:
>   Hi,
> 
> > What we want to have is to have a set of distinctive colors - just 
> > two (background, foreground) colors are not enough - we also need 
> > colors to highlight certain information - we need 5-6 colors for the 
> > output to be maximally expressive. Is there a canonical way to handle 
> > that while still adapting to user preferences automatically by taking 
> > background/foreground color scheme of the xterm into account?
> 
> > I suspect to fix the worst of the fallout we could add some logic to 
> > detect low contrast combinations (too low color distance) and fall 
> > back to the foreground/background colors in that case.
> 
> As far I know it is pretty much impossible to figure the
> foreground/background colors of the terminal you are running on.  You

Glad to hear that, I thought I hadn't researched that much (I did). Hope
somebody appears and tell us how it is done :-)

> can try some guesswork based on $TERM (linux console usually has black
> background, xterm is white by default), but there will always be cases
> where it fails.
> 
> You can run without colors.  You can use bold to highlight things and
> reverse for the cursor.  Surely a bit limited and not as pretty as
> colored, but works for sure everywhere.
> 
> You can go for a linux-console style black background.  Pretty much any
> color is readable here, so you should have no problems at all to find
> the 5-6 colors you want.
> 
> You can go for a xterm-like light background, for example the lightgray
> used by older perf versions.  I like that background color, problem is
> with most colors the contrast is pretty low.  IMHO only red, blue and
> violet are readable on lightgray.  And black of course.
> 
> > Plus allowing full .perfconfig configurability of all the relevant 
> > colors, for those with special taste.
> 
> Sure.  Maybe also allow multiple color sections and pick them by $TERM
> or --colors switch, i.e. [colors "xterm"].

Its fully configurable as of now, what we need is a set of .perfconfigs
that show how people think its better, we try it, set it as the default,
leave the others in tools/perf/Documentation/perfconfig/color.examples.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [F.A.Q.] the advantages of a shared tool/kernel Git repository, tools/perf/ and tools/kvm/

2011-11-08 Thread Arnaldo Carvalho de Melo
Em Tue, Nov 08, 2011 at 04:38:48PM +0100, Gerd Hoffmann escreveu:
> Seems to have no effect, guess the distro perf is too old (RHEL-6).

> > [tui]
> > report = off

> That works.  I don't want turn off the tui altogether though, I actually
> like the interactive expanding+collapsing of the call graphs.  I just
> want turn off the colors.

> perf_color_default_config() in util/color.c seems to lookup a "color.ui"
> config variable.  Can I set that somehow?  Tried ui= in a [color]
> section -- no effect.

Ouch, that came from the code initialy stolen^Wcopied from git :-\

I don't think that will have any effect :-\
 
> > Ah, if you still need to configure the colors, use "default" so that it
> > will use whatever is the color configured in your
> > xterm/gnome-terminal/whatever profile.

> > For reference, the default set of colors now is:

> > .colorset = HE_COLORSET_TOP,
> > .name = "top",
> > .fg   = "red",
> > .bg   = "default",

> Bad idea IMO.  Setting only one of foreground+background gives pretty
> much unpredictable results.  My xterms have different background colors,
> the ones with a root shell happen to have a (dark) red background.
> Which results in red-on-dark-red text.  Not good.

> I'd strongly suggest to either set both background and foreground to
> default or to set both to a specific color.  When doing the latter make

That is the case for the normal one, two colorsets below the
HE_COLORSET_TOP one.

Humm, certainly there could be logic to figure it out if background ==
foreground and do something about it.

> sure the colors have enougth contrast so they are readable.

Problem is figuring out something that is considered a good default :-\
There will always be somebody that will complain.

When doing the coding to allow using the default xterm colors I tried
several of the gnome-terminal xterm profiles and all looked kinda sane
for the "top" (hottest functions, with most hits) and "medium" lines,
where we combine some chosen foreground color ("red" and "green").

Laziest solution would be: If the user customizes that much, could the
user please customize this as well? :-)

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [F.A.Q.] the advantages of a shared tool/kernel Git repository, tools/perf/ and tools/kvm/

2011-11-08 Thread Arnaldo Carvalho de Melo
Em Tue, Nov 08, 2011 at 02:40:42PM +0100, Gerd Hoffmann escreveu:
> > Indeed, documentation is lacking, I think coming from a kernel
> > standpoint I relied too much in the "documentation is source code"
> > mantra of old days.
 
> Sorry for the shameless plug, but as you are speaking of lacking

Thank you! Its easier when I get the questions for specific problems in
the documentation :-)

> documentation:  Where the heck is the perf config file documented, other
> than source code?  Reading the parser to figure how the config file is
> supposed to look like really isn't fun :(
 
> I'm looking for a way to disable the colors in the perf report tui.  Or
> configure them into something readable.  No, light green on light gray
> which is used by default isn't readable.

That was fixed in 3.2-rc1, where also we have:

[acme@felicio linux]$ cat tools/perf/Documentation/perfconfig.example
[colors]

# These were the old defaults
top = red, lightgray
medium = green, lightgray
normal = black, lightgray
selected = lightgray, magenta
code = blue, lightgray

[tui]

# Defaults if linked with libslang
report = on
annotate = on
top = on

[buildid]

# Default, disable using /dev/null
dir = /root/.debug
[acme@felicio linux]$

So you can use:

[tui]

report = off

To disable the TUI altogether or use:

$ perf report --stdio

Or tweak the colors to your liking.

By default the TUI now uses whatever color is configured for your xterm,
not something fixed as in the past, which was a common source of
complaints, that, unfortunately I only heard indirectly :-\

Ah, if you still need to configure the colors, use "default" so that it
will use whatever is the color configured in your
xterm/gnome-terminal/whatever profile.

For reference, the default set of colors now is (from
tools/perf/util/ui/browser.c):

static struct ui_browser__colorset {
const char *name, *fg, *bg;
int colorset;
} ui_browser__colorsets[] = {
{
.colorset = HE_COLORSET_TOP,
.name = "top",
.fg   = "red",
.bg   = "default",
},
{
.colorset = HE_COLORSET_MEDIUM,
.name = "medium",
.fg   = "green",
.bg   = "default",
},
{
.colorset = HE_COLORSET_NORMAL,
.name = "normal",
.fg   = "default",
.bg   = "default",
},
{
.colorset = HE_COLORSET_SELECTED,
.name = "selected",
.fg   = "black",
.bg   = "lightgray",
},
{
.colorset = HE_COLORSET_CODE,
.name = "code",
.fg   = "blue",
.bg   = "default",
},

It should all be fixed up now, together with many other improvements
that should make the TUI and stdio default user experience similar up
till you start using the navigation keys to do things that only are
possible with a TUI, like folding/unfolding callchains, etc.

Please let me know about any other problem you may find with it!

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [Qemu-devel] [PATCH] KVM: Add wrapper script around QEMU to test kernels

2011-11-08 Thread Arnaldo Carvalho de Melo
Em Tue, Nov 08, 2011 at 01:07:55PM +0100, Ingo Molnar escreveu:
> * Vince Weaver  wrote:
> > as mentioned before I have my own perf_event test suite with 20+ tests.
> >   http://web.eecs.utk.edu/~vweaver1/projects/perf-events/validation.html
 
> That should probably be moved into perf test. Arnaldo, any 
> objections?

I'd gladly take patches, I even have in my TODO list for me to volunteer
time to do that at some point.

If somebody else than me or Vince wants to do that... Assuming there is
no licensing problem and Vince doesn't objects for that to be done.

I know that at least the QE team at Red Hat uses it and I hope other QE
teams do it.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [F.A.Q.] the advantages of a shared tool/kernel Git repository, tools/perf/ and tools/kvm/

2011-11-08 Thread Arnaldo Carvalho de Melo
Em Tue, Nov 08, 2011 at 05:21:50AM -0500, Theodore Tso escreveu:
> 
> On Nov 8, 2011, at 4:32 AM, Ingo Molnar wrote:
> > 
> > No ifs and when about it, these are the plain facts:
> > 
> > - Better features, better ABIs: perf maintainers can enforce clean, 
> >   functional and usable tooling support *before* committing to an 
> >   ABI on the kernel side.

> "We don't have to be careful about breaking interface compatibility
> while we are developing new features".

My normal working environment is an MRG PREEMPT_RT kernel (2.6.33.9,
test kernels based on 3.0+) running on enterprise distros while I
develop the userspace part.

So no, at least for me, I don't keep updating the kernel part while
developing userspace.
 
> The flip side of this is that it's not obvious when an interface is
> stable, and when it is still subject to change.  It makes life much
> harder for any userspace code that doesn't live in the kernel.   And I
> think we do agree that moving all of userspace into a single git tree
> makes no sense, right?

Right, but that is the extreme as well, right?
 
> > - We have a shared Git tree with unified, visible version control. I
> >   can see kernel feature commits followed by tooling support, in a
> >   single flow of related commits:
> > 
> >  perf probe: Update perf-probe document
> >  perf probe: Support --del option
> >  trace-kprobe: Support delete probe syntax
> > 
> >   With two separate Git repositories this kind of connection between
> >   the tool and the kernel is inevitably weakened or lost.
 
> "We don't have to clearly document new interfaces between kernel and
> userspace, and instead rely on git commit order for people to figure
> out what's going on with some new interface"

Indeed, documentation is lacking, I think coming from a kernel
standpoint I relied too much in the "documentation is source code"
mantra of old days.

But I realize its a necessity and also that regression testing is as
well another necessity.

I introduced 'perf test' for this later need and rejoice everytime
people submit new test cases, like Jiri and Han did in the past, its
just that we need more of both, documentation and regression testing.

Unfortunately that is not so sexy and I have my hands full not just with
perf :-\
 
> > - Easier development, easier testing: if you work on a kernel 
> >   feature and on matching tooling support then it's *much* easier to
> >   work in a single tree than working in two or more trees in 
> >   parallel. I have worked on multi-tree features before, and except
> >   special exceptions they are generally a big pain to develop.
 
> I've developed in the split tree systems, and it's really not that
> hard.  It does mean you have to be explicit about designing interfaces
> up front, and then you have to have a good, robust way of negotiating
> what features are in the kernel, and what features are supposed by the
> userspace --- but if you don't do that then having good backwards and
> forwards compatibility between different versions of the tool simply
> doesn't exist.
 
> So at the end of the day it question is whether you want to be able to
> (for example) update e2fsck to get better ability to fix more file
> system corruptions, without needing to upgrade the kernel.   If you
> want to be able to use a newer, better e2fsck with an older,
> enterprise kernel, then you have use certain programming disciplines.
> That's where the work is, not in whether you have to maintain two git
> trees or a single git tree.

But it can as well be achieved with a single tree, or do you think
having a single tree makes that impossible to achieve? As I said I do
development basically using the split model at least for testing new
tools on older kernels.

People using the tools while developing mostly the kernel or both
kperf/uperf components do the test on the combined kernel + perf
sources.
 
> > - We are using and enforcing established quality control and coding
> >   principles of the kernel project. If we mess up then Linus pushes
> >   back on us at the last line of defense - and has pushed back on us
> >   in the past. I think many of the currently external kernel
> >   utilities could benefit from the resulting rise in quality.
> >   I've seen separate tool projects degrade into barely usable
> >   tinkerware - that i think cannot happen to perf, regardless of who
> >   maintains it in the future.
 
> That's basically saying that if you don't have someone competent
> managing the git tree and providing quality assurance, life gets hard.
> Sure.   But at the same time, does it scale to move all of userspace
> under one git tree and depending on Linus to push back? 

8 or 80 again :-\
 
> I mean, it would have been nice to move all of GNOME 3 under the Linux
> kernel, so Linus could have pushed back on behalf of all of us power

Sheesh, all of gnome? How closely related and used in kernel development
is gnome? gnome 3?

> users, but as much as many of us would have

Re: trace_printk() support in trace-cmd

2010-12-12 Thread Arnaldo Carvalho de Melo
Em Sun, Dec 12, 2010 at 07:42:06PM +0200, Avi Kivity escreveu:
> On 12/12/2010 07:36 PM, Arnaldo Carvalho de Melo wrote:
> >Em Sun, Dec 12, 2010 at 06:35:24PM +0200, Avi Kivity escreveu:
> >>  On 11/23/2010 05:45 PM, Steven Rostedt wrote:
> >>  >Again, the work around is to replace your trace_printks() with
> >>  >__trace_printk(_THIS_IP_, ...) or just modify the trace_printk() macro
> >>  >in include/linux/kernel.h to always use the __trace_printk() version.
> >>
> >>  This works; I'm using it for now (I tried to use 'perf probe', but I
> >>  get unpredictable results, like null pointer derefs).
> >
> >Can you tell us which functions, environment, etc?
> 
> Something around 2.6.27-rc4; example functions are FNAME(fetch) in
> arch/x86/kvm/paging_tmpl.h; compiled modular (which was Steven's
> guess as to why it fails).
> 
> (note, the failure is with trace-cmd, not /sys/kernel/debug/tracing).

I mean the "I tried to use 'perf probe'" part.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: trace_printk() support in trace-cmd

2010-12-12 Thread Arnaldo Carvalho de Melo
Em Sun, Dec 12, 2010 at 06:35:24PM +0200, Avi Kivity escreveu:
> On 11/23/2010 05:45 PM, Steven Rostedt wrote:
> >Again, the work around is to replace your trace_printks() with
> >__trace_printk(_THIS_IP_, ...) or just modify the trace_printk() macro
> >in include/linux/kernel.h to always use the __trace_printk() version.
> 
> This works; I'm using it for now (I tried to use 'perf probe', but I
> get unpredictable results, like null pointer derefs).

Can you tell us which functions, environment, etc?

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 3/3] perf events: Change perf parameter --pid to process-wide collection instead of thread-wide

2010-03-25 Thread Arnaldo Carvalho de Melo
Em Thu, Mar 25, 2010 at 04:47:44PM +0800, Zhang, Yanmin escreveu:
> On Thu, 2010-03-25 at 16:02 +0800, Li Zefan wrote:
> Thanks for reporting it. Arnaldo, could you pick up below patch?
> Zefan, Could you try it?

Sure thing, will stash them in today's push to Ingo, thanks!

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Arnaldo Carvalho de Melo
Em Wed, Mar 24, 2010 at 08:20:10PM +0200, Avi Kivity escreveu:
> On 03/24/2010 07:47 PM, Arnaldo Carvalho de Melo wrote:
>> Em Wed, Mar 24, 2010 at 06:09:30PM +0200, Avi Kivity escreveu:
>>
>>> Doesn't perf already has a dependency on naming conventions for finding
>>> debug information?
>>>  
>> It looks at several places, from most symbol rich (/usr/lib/debug/, aka
>> -debuginfo packages, where we have full symtabs) to poorest (the
>> packaged binary, where we may just have a .dynsym).
>>
>> In an ideal world, it would just get the build-id (a SHA1 cookie that is
>> in an ELF session inserted in every binary (aka DSOs), kernel module,
>> kallsyms or vmlinux file) and use that to look first in a local cache
>> (implemented in perf for a long time already) or in some symbol server.
>>
>> For instance, for a random perf.data file I collected here in my machine
>> I have:
>>
>> [a...@doppio linux-2.6-tip]$ perf buildid-list | grep libpthread
>> 5c68f7afeb33309c78037e374b0deee84dd441f6 /lib64/libpthread-2.10.2.so
>> [a...@doppio linux-2.6-tip]$
>>
>> So I don't have to access /lib64/libpthread-2.10.2.so directly, nor some
>> convention to get a debuginfo in a local file like:
>>
>> /usr/lib/debug/lib64/libpthread-2.10.2.so.debug
>>
>> Instead the tools look at:
>>
>> [a...@doppio linux-2.6-tip]$ l 
>> ~/.debug/.build-id/5c/68f7afeb33309c78037e374b0deee84dd441f6
>> lrwxrwxrwx 1 acme acme 73 2010-01-06 18:53 
>> /home/acme/.debug/.build-id/5c/68f7afeb33309c78037e374b0deee84dd441f6 ->  
>> ../../lib64/libpthread-2.10.2.so/5c68f7afeb33309c78037e374b0deee84dd441f6*
>>
>> To find the file for that specific build-id, not the one installed in my
>> machine (or on the different machine, of a different architecture) that
>> may be completely unrelated, a new one, or one for a different arch.

> Thanks.  I believe qemu could easily act as a symbol server for this use  
> case.

Agreed, but it doesn't even have to :-)

We just need to get the build-id in the PERF_RECORD_MMAP event somehow
and then get this symbol from elsewhere, say the same DVD/RHN
channel/Debian Repository/embedded developer toolkit image not
stripped/whatever.

Or it may already be in the local cache from last week's perf report
session :-)

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [RFC] Unify KVM kernel-space and user-space code into a single project

2010-03-24 Thread Arnaldo Carvalho de Melo
Em Wed, Mar 24, 2010 at 06:09:30PM +0200, Avi Kivity escreveu:
> Doesn't perf already has a dependency on naming conventions for finding  
> debug information?

It looks at several places, from most symbol rich (/usr/lib/debug/, aka
-debuginfo packages, where we have full symtabs) to poorest (the
packaged binary, where we may just have a .dynsym).

In an ideal world, it would just get the build-id (a SHA1 cookie that is
in an ELF session inserted in every binary (aka DSOs), kernel module,
kallsyms or vmlinux file) and use that to look first in a local cache
(implemented in perf for a long time already) or in some symbol server.

For instance, for a random perf.data file I collected here in my machine
I have:

[a...@doppio linux-2.6-tip]$ perf buildid-list | grep libpthread
5c68f7afeb33309c78037e374b0deee84dd441f6 /lib64/libpthread-2.10.2.so
[a...@doppio linux-2.6-tip]$

So I don't have to access /lib64/libpthread-2.10.2.so directly, nor some
convention to get a debuginfo in a local file like:

/usr/lib/debug/lib64/libpthread-2.10.2.so.debug

Instead the tools look at:

[a...@doppio linux-2.6-tip]$ l 
~/.debug/.build-id/5c/68f7afeb33309c78037e374b0deee84dd441f6
lrwxrwxrwx 1 acme acme 73 2010-01-06 18:53 
/home/acme/.debug/.build-id/5c/68f7afeb33309c78037e374b0deee84dd441f6 -> 
../../lib64/libpthread-2.10.2.so/5c68f7afeb33309c78037e374b0deee84dd441f6*

To find the file for that specific build-id, not the one installed in my
machine (or on the different machine, of a different architecture) that
may be completely unrelated, a new one, or one for a different arch.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] Enhance perf to collect KVM guest os statistics from host side

2010-03-23 Thread Arnaldo Carvalho de Melo
Em Tue, Mar 23, 2010 at 03:20:11PM +0100, Andi Kleen escreveu:
> Soeren Sandmann  writes:
> > I don't think the oprofile JIT interface solves any of these
> > problems. (In fact, I don't see why the JIT problem is even hard. The
> > JIT compiler can just generate a little ELF file with symbols in it,
> > and the profiler can pick it up through the mmap events that you get
> > through the perf interface).
> 
> That would require keeping those temporary ELF files for
> potentially unlimited time around (profilers today look at the ELF
> files at the final analysis phase, which might be weeks away)

'perf record' will traverse the perf.data file just collected and, if the
binaries have build-ids, will stash them in ~/.debug/, keyed by build-id
just like the -debuginfo packages do.

So only the binaries with hits. Also one can use 'perf archive' to
create a tar.bz2 file with the files with hits for the specified
perf.data file, that can then be transfered to another machine, whatever
arch, untarred at ~/.debug and then the report can be done there.

As it is done by build-id, multiple 'perf record' sessions share files
in the cache.

Right now the whole ELF file (or /proc/kallsyms copy) is stored if
collected from the DSO directly, or the bits that are stored in
-debuginfo files if we find it installed (so smaller). We could strip
that down further by storing just the ELF sections needed to make sense
of the symtab.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] Enhance perf to collect KVM guest os statistics from host side

2010-03-23 Thread Arnaldo Carvalho de Melo
Em Tue, Mar 23, 2010 at 02:49:01PM +0100, Andi Kleen escreveu:
> Soeren Sandmann  writes:
> > To fix that problem, it seems like we need some way to have python
> > export what is going on. Maybe the same mechanism could be used to
> > both access what is going on in qemu and python.
> 
> oprofile already has an interface to let JITs export
> information about the JITed code. C Python is not a JIT,
> but presumably one of the python JITs could do it.
> 
> http://oprofile.sourceforge.net/doc/devel/index.html
> 
> I know it's not envogue anymore and you won't be a approved 
> cool kid if you do, but you could just use oprofile? 

perf also has supports for this and Pekka Enberg's jato uses it:

http://penberg.blogspot.com/2009/06/jato-has-profiler.html

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] Enhance perf to collect KVM guest os statistics from host side

2010-03-23 Thread Arnaldo Carvalho de Melo
Em Tue, Mar 23, 2010 at 11:14:41AM +0800, Zhang, Yanmin escreveu:
> On Mon, 2010-03-22 at 13:44 -0300, Arnaldo Carvalho de Melo wrote:
> > Em Mon, Mar 22, 2010 at 03:24:47PM +0800, Zhang, Yanmin escreveu:
> > > On Fri, 2010-03-19 at 09:21 +0100, Ingo Molnar wrote:
> > > Then, perf could access all files. It's possible because guest os instance
> > > happens to be multi-threading in a process. One of the defects is the 
> > > accessing to
> > > guest os becomes slow or impossible when guest os is very busy.
> > 
> > If the MMAP events on the guest included a cookie that could later be
> > used to query for the symtab of that DSO, we wouldn't need to access the
> > guest FS at all, right?

> It depends on specific sub commands. As for 'perf kvm top', developers
> want to see the profiling immediately. Even with 'perf kvm record',
> developers also want to

That is not a problem, if you have the relevant buildids in your cache
(Look in your machine at ~/.debug/), it will be as fast as ever.

If you use a distro that has its userspace with build-ids, you probably
use it always without noticing :-)

> see results quickly. At least I'm eager for the results when
> investigating a performance issue.

Sure thing.
 
> > With build-ids and debuginfo-install like tools the symbol
> > resolution could be performed by using the cookies (build-ids) as
> > keys to get to the *-debuginfo packages with matching symtabs (and
> > DWARF for source annotation, etc).

> We can't make sure guest os uses the same os images, or don't know
> where we could find the original DVD images being used to install
> guest os.

You don't have to have guest and host sharing the same OS image, you
just have to somehow populate your buildid cache with what you need, be
it using sshfs or what Ingo is suggesting once, or using what your
vendor provides (debuginfo packages). And you just have to do it once,
for the relevant apps, to have it in your buildid cache.
 
> Current perf does save build id, including both kernls's and other
> application lib/executables.

Yeah, I know, I implemented it. :-)
 
> > We have that for the kernel as:

> > [a...@doppio linux-2.6-tip]$ l /sys/kernel/notes 
> > -r--r--r-- 1 root root 36 2010-03-22 13:14 /sys/kernel/notes
> > [a...@doppio linux-2.6-tip]$ l /sys/module/ipv6/sections/.note.gnu.build-id 
> > -r--r--r-- 1 root root 4096 2010-03-22 13:38 
> > /sys/module/ipv6/sections/.note.gnu.build-id
> > [a...@doppio linux-2.6-tip]$

> > That way we would cover DSOs being reinstalled in long running 'perf
> > record' sessions too.

> That's one of objectives of perf to support long running.

But it doesn't fully supports right now, as I explained, build-ids are
collected at the end of the record session, because we have to open the
DSOs that had hits to get the 20 bytes cookie we need, the build-id.

If we had it in the PERF_RECORD_MMAP record, we would close this race,
and the added cost at load time should be minimal, to get the ELF
section with it and put it somewhere in task struct.

If only we could coalesce it a bit to reclaim this:

[a...@doppio linux-2.6-tip]$ pahole -C task_struct 
../build/v2.6.34-rc1-tip+/kernel/sched.o  | tail -5
/* size: 5968, cachelines: 94, members: 150 */
/* sum members: 5943, holes: 7, sum holes: 25 */
/* bit holes: 1, sum bit holes: 28 bits */
/* last cacheline: 16 bytes */
};
[a...@doppio linux-2.6-tip]$ 

8-)

Or at least get just one of those 4 bytes holes then we could stick it
at the end to get our build-id there, accessing it would be done only
at PERF_RECORD_MMAP injection time, i.e. close to the time when we
actually are loading the executable mmap, i.e. close to the time when
the loader is injecting the build-id, I guess the extra memory and
processing costs would be in the noise.

> > This was discussed some time ago but would require help from the bits
> > that load DSOs.

> > build-ids then would be first class citizens.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] Enhance perf to collect KVM guest os statistics from host side

2010-03-22 Thread Arnaldo Carvalho de Melo
Em Mon, Mar 22, 2010 at 03:24:47PM +0800, Zhang, Yanmin escreveu:
> On Fri, 2010-03-19 at 09:21 +0100, Ingo Molnar wrote:
> > So some sort of --guestmount option would be the natural solution, which 
> > points to the guest system's root: and a Qemu enumeration of guest mounts 
> > (which would be off by default and configurable) from which perf can pick 
> > up 
> > the target guest all automatically. (obviously only under allowed 
> > permissions 
> > so that such access is secure)
> If sshfs could access /proc/ and /sys correctly, here is a design:
> --guestmount points to a directory which consists of a list of 
> sub-directories.
> Every sub-directory's name is just the qemu process id of guest os. 
> Admin/developer
> mounts every guest os instance's root directory to corresponding 
> sub-directory.
> 
> Then, perf could access all files. It's possible because guest os instance
> happens to be multi-threading in a process. One of the defects is the 
> accessing to
> guest os becomes slow or impossible when guest os is very busy.

If the MMAP events on the guest included a cookie that could later be
used to query for the symtab of that DSO, we wouldn't need to access the
guest FS at all, right?

With build-ids and debuginfo-install like tools the symbol resolution
could be performed by using the cookies (build-ids) as keys to get to
the *-debuginfo packages with matching symtabs (and DWARF for source
annotation, etc).

We have that for the kernel as:

[a...@doppio linux-2.6-tip]$ l /sys/kernel/notes 
-r--r--r-- 1 root root 36 2010-03-22 13:14 /sys/kernel/notes
[a...@doppio linux-2.6-tip]$ l /sys/module/ipv6/sections/.note.gnu.build-id 
-r--r--r-- 1 root root 4096 2010-03-22 13:38 
/sys/module/ipv6/sections/.note.gnu.build-id
[a...@doppio linux-2.6-tip]$

That way we would cover DSOs being reinstalled in long running 'perf
record' sessions too.

This was discussed some time ago but would require help from the bits
that load DSOs.

build-ids then would be first class citizens.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 3/3] perf events: Change perf parameter --pid to process-wide collection instead of thread-wide

2010-03-18 Thread Arnaldo Carvalho de Melo
Em Thu, Mar 18, 2010 at 10:35:48AM -0300, Arnaldo Carvalho de Melo escreveu:
> Em Thu, Mar 18, 2010 at 05:31:06PM +0800, Zhang, Yanmin escreveu:
> > From: Zhang, Yanmin 
> > 
> > Parameter --pid (or -p) of perf currently means a thread-wide collection.
> > For exmaple, if a process whose id is  has 10 threads, 'perf top -p 
> > '
> > just collects the main thread statistics. That's misleading. Users are
> > used to attach a whole process when debugging a process by gdb. To follow
> > normal usage style, the patch change --pid to process-wide collection and
> > add --tid (-t) to mean a thread-wide collection.
> > 
> > Usage example is:
> > #perf top -p 
> > #perf record -p  -f sleep 10
> > #perf stat -p  -f sleep 10
> > Above commands collect the statistics of all threads of process .
> > 
> > Signed-off-by: Zhang Yanmin 
> 
> Just did visual inspection of the three patches, all sane, except for
> some coding style nits, don't worry right now for that, I'll fix them up

Nah, didn't fix them up, left it for some followup patch to avoid
risking introducing problems, applied as is, did some testing and pushed
to Ingo, thanks!

> myself, but please take those into account int the future, highlight
> below.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 3/3] perf events: Change perf parameter --pid to process-wide collection instead of thread-wide

2010-03-18 Thread Arnaldo Carvalho de Melo
Em Thu, Mar 18, 2010 at 05:31:06PM +0800, Zhang, Yanmin escreveu:
> From: Zhang, Yanmin 
> 
> Parameter --pid (or -p) of perf currently means a thread-wide collection.
> For exmaple, if a process whose id is  has 10 threads, 'perf top -p '
> just collects the main thread statistics. That's misleading. Users are
> used to attach a whole process when debugging a process by gdb. To follow
> normal usage style, the patch change --pid to process-wide collection and
> add --tid (-t) to mean a thread-wide collection.
> 
> Usage example is:
> #perf top -p 
> #perf record -p  -f sleep 10
> #perf stat -p  -f sleep 10
> Above commands collect the statistics of all threads of process .
> 
> Signed-off-by: Zhang Yanmin 

Just did visual inspection of the three patches, all sane, except for
some coding style nits, don't worry right now for that, I'll fix them up
myself, but please take those into account int the future, highlight
below.
 
> ---
> 
> diff -Nraup linux-2.6_tip0317_statrecord/tools/perf/builtin-record.c 
> linux-2.6_tip0317_statrecordpid/tools/perf/builtin-record.c
> --- linux-2.6_tip0317_statrecord/tools/perf/builtin-record.c  2010-03-18 
> 13:48:39.578181540 +0800
> +++ linux-2.6_tip0317_statrecordpid/tools/perf/builtin-record.c   
> 2010-03-18 14:28:41.449631936 +0800
> + mmap_array[nr_cpu][counter][thread_index].mask = 
> mmap_pages*page_size - 1;
> + mmap_array[nr_cpu][counter][thread_index].base = 
> mmap(NULL, (mmap_pages+1)*page_size,

Spaces around +, *,  etc

> + PROT_READ|PROT_WRITE, MAP_SHARED, 
> fd[nr_cpu][counter][thread_index], 0);
> + if (mmap_array[nr_cpu][counter][thread_index].base == 
> MAP_FAILED) {
> + error("failed to mmap with %d (%s)\n", errno, 
> strerror(errno));
> + exit(-1);
> + }
> + } else {
> + all_tids=malloc(sizeof(pid_t));

Ditto here for =

> + if (!all_tids)
> + return -ENOMEM;
> +
> + all_tids[0] = target_tid;
> + thread_num = 1;
> + }
> +
> + for (i = 0; i < MAX_NR_CPUS; i++) {
> + for (j = 0; j < MAX_COUNTERS; j++) {
> + fd[i][j] = malloc(sizeof(int)*thread_num);
> + mmap_array[i][j] = malloc(
> + sizeof(struct mmap_data)*thread_num);

Ditto

> + if (!fd[i][j] || !mmap_array[i][j])
> + return -ENOMEM;
> + }
> + }
> + event_array = malloc(
> + sizeof(struct pollfd)*MAX_NR_CPUS*MAX_COUNTERS*thread_num);

Ditto

Should be, I suggest:

event_array = malloc((sizeof(struct pollfd) * MAX_NR_CPUS *
  MAX_COUNTERS * thread_num));


Anyway, I'll fix some of these while merging, now.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH] Enhance perf to collect KVM guest os statistics from host side

2010-03-18 Thread Arnaldo Carvalho de Melo
Em Thu, Mar 18, 2010 at 09:03:25AM +0100, Ingo Molnar escreveu:
> 
> * Zhang, Yanmin  wrote:
> 
> > I worked out 3 new patches against tip/master tree of Mar. 17th.
> 
> Cool! Mind sending them as a series of patches instead of attachment? That 
> makes it easier to review them. Also, the Signed-off-by lines seem to be 
> missing plus we need a per patch changelog as well.

Yeah, please, and I hadn't merged them, so the resend was the best thing to do.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: KVM usability

2010-03-07 Thread Arnaldo Carvalho de Melo
Em Sun, Mar 07, 2010 at 08:15:40PM +0200, Avi Kivity escreveu:
> On 03/07/2010 08:01 PM, Arnaldo Carvalho de Melo wrote:
> >Em Sun, Mar 07, 2010 at 11:35:31AM +0200, Avi Kivity escreveu:
> >>perf really is wonderful, but to be really competitive, and usable to
> >>more developers, it needs to be in a graphical environment.  I want
> >>'perf report' output to start out collapsed and drill down by clicking
> >>on a tree widget.  Clicking on a function name opens its definition.
> >>'perf annotate' should display annotations on my editor window, not in a
> >>pager.  I should be able to check events on a list, not using 'perf
> >>list'.
> >Do you really think that more kernel developers would use perf more
> >frequently if it had some GUI?
> 
> Not much.  Is perf's target kernel developers exclusively?  Who are
> we writing this kernel for?

No, we aren't writing this tool only for kernel developers exclusively,
but that wasn't my question, it was badly formulated, sorry, I shouldn't
have included "kernel" in it :-\
 
> No wonder everything is benchmarked using kbuild.
> 
> >I plan to work on a ncurses tool combining aspects of the existing perf
> >tools, integrating them more, like you suggest above, but even having
> >worked on a pygtk tool that is close to the kernel [1], I'm unsure if
> >doing it using gtk or QT would be something that would entice more
> >developers to use it.
> 
> Even for kernel developers there are advantages in a GUI, namely
> that features are easily discovered, the amount of information is
> easily controlled, and in that you can interact (not redo everything
> from scratch every time you want to change something).  The
> difference between a curses based tool and a true GUI are minimal
> for this audience.

Ok, I agree with you about easier discoverability of features, path
shortened from report to annotate to starting the editor right at the
line where some event of interest happened, will try to keep the
routines not much coupled with ncurses, but definetely ncurses will be
the first step.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: KVM usability

2010-03-07 Thread Arnaldo Carvalho de Melo
Em Sun, Mar 07, 2010 at 11:35:31AM +0200, Avi Kivity escreveu:
> perf really is wonderful, but to be really competitive, and usable to  
> more developers, it needs to be in a graphical environment.  I want  
> 'perf report' output to start out collapsed and drill down by clicking  
> on a tree widget.  Clicking on a function name opens its definition.   
> 'perf annotate' should display annotations on my editor window, not in a  
> pager.  I should be able to check events on a list, not using 'perf 
> list'.

Do you really think that more kernel developers would use perf more
frequently if it had some GUI?

I plan to work on a ncurses tool combining aspects of the existing perf
tools, integrating them more, like you suggest above, but even having
worked on a pygtk tool that is close to the kernel [1], I'm unsure if
doing it using gtk or QT would be something that would entice more
developers to use it.

- Arnaldo

[1] http://www.osadl.org/Single-View.111+M52212cb1379.0.html
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: Enhance perf to support KVM

2010-03-02 Thread Arnaldo Carvalho de Melo
Em Tue, Mar 02, 2010 at 05:46:03PM +0100, Paolo Bonzini escreveu:
> On 02/26/2010 03:23 PM, Ingo Molnar wrote:
>> I do think tools/X and tools/libc would make quite a bit of sense - this is
>> one of the better design aspects of FreeBSD et al. It's a mistake that it's
>> not being done.
>
> I don't see what it would buy to have tools/libc.  You cannot force  
> users to update kernel-space and user-space in lockstep (Apple forced  
> you to do that sometimes when I used Macs at work, and it was very very  
> inconvenient), so it's not like libc would be able to always assume the  
> latest system calls.  There is (relatively) a lot of  
> backwards-compatibility code in libc; it's ugly code, but you have to  
> live with it.
>
> Any case in which you need lockstep upgrades of kernel and libc for  
> bisectability is a bug in libc, and I haven't seen it happen enough to  
> be a problem.

You imply lockstep updates because both are on the same source tree? I
don't see why that would be required, there is an ABI contract to be
respected no matter where the sources for the signatories live.

perf lives in the kernel sources and while developing it I use all sorts
of combinations of userland and kernel perf bits so as not to be
rebooting the many machines of different arches where I test the
userspace bits all the time with a new kernel, even if it indeed has new
in-kernel perf bits.

As an instance where I could have worked on tools/libc recently was the
recvmmsg syscall I introduced.

Till it got into glibc (IIRC it was, but I wasn't involved in it, google
alert told me) interested parties had to use 'man syscall' and pick the
syscall number for their specific architecture, etc.

I could've done both the kernel and userspace bits in the same patch.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: KVM usability

2010-03-01 Thread Arnaldo Carvalho de Melo
Em Mon, Mar 01, 2010 at 06:48:07AM -1000, Zachary Amsden escreveu:
> On 02/27/2010 07:25 AM, Ingo Molnar wrote:
>> I'm not talking about moving it into a kernel _module_ - albeit that
>> alone is a worthwile thing to do for any performance sensitive hw
>> component.
>>
>> I was talking about the option of a clean, stripped down Qemu base
>> hosted in the kernel proper, in linux/tools/kvm/ or so. If i were
>> running a virtualization effort it would be the first place i'd
>> consider to put my tooling into.

> So ripping out a clean part interface like PCI bus infrastructure and  
> using it in the kernel, for example, does nothing except put that  
> infrastructure in two different places, because everything the kernel  
> does, userspace will have to do again anyway.  So now you have twice as  
> much code involving the same idea and you have to keep the pieces in  
> sync and from trampling each other.
>
> The only parts that warrant such complexity and high risk for bugs are  
> performance critical things like the PIT and APIC.

I guess there is some misunderstanding here, the tools/ directory that
lives in the kernel _sources_, has no kernel source, its all userspace
stuff.

- Arnaldo
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html