Re: [Qemu-devel] [PATCH] KVM: Add wrapper script around QEMU to test kernels
On Tue, 8 Nov 2011, Arnaldo Carvalho de Melo wrote: > Em Tue, Nov 08, 2011 at 01:07:55PM +0100, Ingo Molnar escreveu: > > * Vince Weaver wrote: > > > as mentioned before I have my own perf_event test suite with 20+ tests. > > > http://web.eecs.utk.edu/~vweaver1/projects/perf-events/validation.html > > > That should probably be moved into perf test. Arnaldo, any > > objections? > > I'd gladly take patches, I even have in my TODO list for me to volunteer > time to do that at some point. > > If somebody else than me or Vince wants to do that... Assuming there is > no licensing problem and Vince doesn't objects for that to be done. I have no objections, though I don't really have time right now to do the work myself. The test code is licensed dual GPLv2/BSD. I should stick that in the package somewhere if I haven't already. My testcases mostly are testing things necessary for proper PAPI functionality and are by no means complete. There are huge areas of perf_event functionality that are not well tested, especially the overflow code. Vince
Re: [Qemu-devel] [PATCH] KVM: Add wrapper script around QEMU to test kernels
On Mon, 7 Nov 2011, Ingo Molnar wrote: > I think we needed to do only one revert along the way in the past two > years, to fix an unintended ABI breakage in PowerTop. Considering the > total complexity of the perf ABI our compatibility track record is > *very* good. There have been more breakages, as you know. It's just they weren't caught in time so they were declared to be grandfathered in rather than fixed. > Pekka, Vince has meanwhile become the resident perf critic on lkml, > always in it when it comes to some perf-bashing: For what it's worth you'll find commits from me in the qemu tree, and I also oppose the merge of kvm-tool into the Linux tree. > ... and you have argued against perf from the very first day on, when > you were one of the perfmon developers - and IMO in hindsight you've > been repeatedly wrong about most of your design arguments. I can't find an exact e-mail, but I seem to recall my arguments were that Pentium 4 support would be hard (it was), that in-kernel generalized events were a bad idea (I still think that, try talking to the ARM guys sometime about that) and that making access to raw events hard (by not using a naming library) was silly. I'm sure I probably said other things that were eventually addressed. > The PAPI project has the (fundamental) problem that you are still > doing it in the old-style sw design fashion, with many months long > delays in testing, and then you are blaming the problems you > inevitably meet with that model on *us*. The fundamental problem with the PAPI project is that we only have 3 full-time developers, and we have to make sure PAPI runs on about 10 different platforms, of which perf_events/Linux is only one. Time I waste tracking down perf_event ABI regressions and DoS bugs takes away from actual useful userspace PAPI development. > There was one PAPI incident i remember where it took you several > *months* to report a regression in a regular PAPI test-case (no > actual app affected as far as i know). No other tester ever ran the > PAPI testcases so nobody else reported it. We have a huge userbase. They run on some pretty amazing machines and do some tests that strain perf libraries to the limit. They also tend to use distro kernels, assuming they even have moved to 2.6.31+ kernels yet. When these power users report problems, they aren't going to be against the -tip tree. > Nobody but you tests PAPI so you need to become *part* of the > upstream development process, which releases a new upstream kernel > every 3 months. PAPI is a free software project, with the devel tree available from CVS. It takes maybe 15 minutes to run the full PAPI regression suite. I encourage you or any perf developer to try it and report any issues. I can only be so comprehensive. I didn't find the current NMI-watchdog regression right away because my git tree builds didn't have it enabled. It wasn't until there started being 3.0 distro kernels that people started reporting the problem to us. > Also, as i mentioned it several times before, you are free to add an > arbitrary number of ABI test-cases to 'perf test' and we can promise > that we run that. Right now it consists of a few tests: as mentioned before I have my own perf_event test suite with 20+ tests. http://web.eecs.utk.edu/~vweaver1/projects/perf-events/validation.html I do run it often. It tends to be reactionary though, as I can only add a test for a bug once I know about it. I also have more up-to date perf documentation than the kernel does: http://web.eecs.utk.edu/~vweaver1/projects/perf-events/programming.html and a cpu compatability matrix: http://web.eecs.utk.edu/~vweaver1/projects/perf-events/support.html I didn't really want to turn this into yet another perf flamewar. I just didn't want the implication that perf being in kernel is all rainbows and unicorns to go unchallenged. Vince
Re: [Qemu-devel] [PATCH] KVM: Add wrapper script around QEMU to test kernels
On Mon, 7 Nov 2011, Pekka Enberg wrote: > I've never heard ABI incompatibility used as an argument for perf. Ingo? Never overtly. They're too clever for that. In any case, as a primary developer of a library (PAPI) that uses the perf_events ABI I have to say that having perf in the kernel has been a *major* pain for us. Unlike the perf developers, we *do* have to maintain backwards compatability. And we have a lot of nasty code in PAPI to handle this. Entirely because the perf_events ABI is not stable. It's mostly stable, but there are enough regressions to be a pain. It's problem enough that there's no way to know what version of the perf_event abi you are running against and we have to guess based on kernel version. This gets "fun" because all of the vendors have backported seemingly random chunks of perf_event code to their older kernels. And it often does seem as the perf developers don't care when something breaks in perf_events if it doesn't affect perf users. For example, the new NMI watchdog severely breaks perf_event event allocation if you are using FORMAT_GROUP. perf doesn't use this though, so none of the kernel developers seem to care. And unless I can quickly come up with a patch as an outsider, a few kernel versions will go by and the kernel devs will declare "well it was broken so long, now we don't have to fix it". Fun. Vince
Re: [Qemu-devel] [patch] performance counters from inside of Qemu
On Sat, 6 Nov 2010, Stefan Hajnoczi wrote: > On Thu, Nov 4, 2010 at 7:20 PM, Vince Weaver wrote: > > This is mostly a proof of concept, I'm not sure if anyone is interested in > > this. It could in theory be useful for tracking down performance problems > > from iside of Qemu using existing tools. > > This patch handles uniprocessor guests only? yes; I typically only runs Qemu in uniprocessor mode so I hadn't thought about CMP support. In theory it shouldn't require too many changes to the actual MSR code because the kernel handles saving/restoring the counter values on context switch, but it would involve separate per-core simulated instruction counters. Hmmm. > There was a patch series a few months back by Yanmin Zhang to > implement a paravirt perf interface. The subject line was "para virt > interface of perf to support kvm guest os statistics collection in > guest os". It might be interesting to follow that up if you haven't > seen it already. Thanks for pointing that out! I had been looking for something like that and somehow missed it at the time. That patch seems to export a para-virtualized counter interface. While useful, it would require code changes for external tools, and it also wouldn't allow for tools that program raw events into the counters (as opposed to the handful of predefined kernel perf-events ones). Thanks for the feedback. Vince
[Qemu-devel] [patch] performance counters from inside of Qemu
Hello
The following patch enables simulated perf event support inside of Qemu
for x86_64 systems. It enables support for the AMD performance MSRs enough
to return values for the "retired_instructions" (both user and kernel) and
"cpu_clk_unhalted" events.
This is mostly a proof of concept, I'm not sure if anyone is interested in
this. It could in theory be useful for tracking down performance problems
from iside of Qemu using existing tools.
The main missing component is lack of interrupts on counter overflow.
This is possible to implement, I just haven't.
The eventual goal of my group is to enable performance counter support
inside of KVM, but I was experimenting using Qemu first.
Usage:
* Apply the patch
* Boot into Qemu, using a kernel using 2.6.32 or newer.
Use the "-cpu phenom" option to select an AMD machine.
* Use the "perf" utility (or other method of accessing perf counters,
such as PAPI) as you normally would.
Sample output using a benchmark that is exactly 1 million instructions
(it counts more due to kernel overhead):
vi...@debian:~$ perf stat ./million
Performance counter stats for './million':
9.147038 task-clock-msecs # 0.635 CPUs
0 context-switches # 0.000 M/sec
0 CPU-migrations # 0.000 M/sec
2 page-faults # 0.000 M/sec
24990915 cycles # 2732.132 M/sec
1070162 instructions # 0.043 IPC
0 cache-references # 0.000 M/sec
0 cache-misses # 0.000 M/sec
0.014401652 seconds time elapsed
vi...@debian:~$ exit
diff --git a/target-i386/cpu.h b/target-i386/cpu.h
index 2440d65..aac0efd 100644
--- a/target-i386/cpu.h
+++ b/target-i386/cpu.h
@@ -341,6 +341,16 @@
#define MSR_KERNELGSBASE0xc102
#define MSR_TSC_AUX 0xc103
+#define MSR_PERFEVTSEL00xc001
+#define MSR_PERFEVTSEL10xc0010001
+#define MSR_PERFEVTSEL20xc0010002
+#define MSR_PERFEVTSEL30xc0010003
+
+#define MSR_PERFCTR0 0xc0010004
+#define MSR_PERFCTR1 0xc0010005
+#define MSR_PERFCTR2 0xc0010006
+#define MSR_PERFCTR3 0xc0010007
+
#define MSR_VM_HSAVE_PA 0xc0010117
/* cpuid_features bits */
diff --git a/target-i386/helper.c b/target-i386/helper.c
index 26ea1e5..f2aa2d7 100644
--- a/target-i386/helper.c
+++ b/target-i386/helper.c
@@ -31,6 +31,20 @@
//#define DEBUG_MMU
+long long global_ins_count[3] = {0,0,0};
+
+void helper_insn_count(unsigned int cpl);
+
+void helper_insn_count(unsigned int cpl) {
+ if (cpl==0) {
+ global_ins_count[1]++;
+ }
+ else if (cpl==3) {
+ global_ins_count[0]++;
+ }
+ /* FIXME -- handle overflow interrupts */
+}
+
/* NOTE: must be called outside the CPU execute loop */
void cpu_reset(CPUX86State *env)
{
diff --git a/target-i386/helper.h b/target-i386/helper.h
index 6b518ad..6739007 100644
--- a/target-i386/helper.h
+++ b/target-i386/helper.h
@@ -1,5 +1,7 @@
#include "def-helper.h"
+DEF_HELPER_1(insn_count, void, i32)
+
DEF_HELPER_FLAGS_1(cc_compute_all, TCG_CALL_PURE, i32, int)
DEF_HELPER_FLAGS_1(cc_compute_c, TCG_CALL_PURE, i32, int)
diff --git a/target-i386/op_helper.c b/target-i386/op_helper.c
index 43fbd0c..a80d875 100644
--- a/target-i386/op_helper.c
+++ b/target-i386/op_helper.c
@@ -3005,6 +3005,24 @@ void helper_rdmsr(void)
{
}
#else
+
+extern long long global_ins_count[3];
+
+static uint64_t perf_msrs[8];
+
+static struct counter_info_t {
+int enabled;
+long long last_value;
+int umask;
+int emask;
+int os;
+int usr;
+} counter_info[4];
+
+#define PERF_ENABLED 1ULL<<22
+#define PERF_USR 1ULL<<16
+#define PERF_OS 1ULL<<17
+
void helper_wrmsr(void)
{
uint64_t val;
@@ -3126,6 +3144,88 @@ void helper_wrmsr(void)
case MSR_TSC_AUX:
env->tsc_aux = val;
break;
+ case MSR_PERFEVTSEL0:
+ case MSR_PERFEVTSEL1:
+ case MSR_PERFEVTSEL2:
+ case MSR_PERFEVTSEL3:
+{
+ int enable;
+ int umask,emask,usr,os;
+ int counter=(int)ECX&0x7;
+
+ emask=val&0xff;
+ umask=(val>>8)&0xff;
+ enable=!!(val&PERF_ENABLED);
+ usr=!!(val&PERF_USR);
+ os=!!(val&PERF_OS);
+
+ if (enable) {
+ counter_info[counter].emask=emask;
+ counter_info[counter].umask=umask;
+ counter_info[counter].enabled=1;
+ counter_info[counter].last_value=0;
+
+ /* retired instructions */
+ if (emask==0xc0) {
+ if (usr) {
+counter_info[counter].last_value+=global_ins_count[0];
+ }
+
Re: [Qemu-devel] Latest git version failed to run 403.gcc benchmark in SPEC CPU2006
Hello have you tried older versions of qemu, in an attempt to see if this is a recent problem? As of about a year ago I had qemu-sparc32plus running most of SPEC2006 properly, validated against hardware perf counters. gcc definitely worked. I had issues with calculix, dealII, soplex, wrf, zeusmp and astar but the rest all worked and gave retired instruction counts close to that of real hardware. I can probably look up what version of qemu I used for that work. Vince
Re: Res: [Qemu-devel] full dynamic instruction trace for MIPS target
On Tue, 6 Apr 2010, Boris Cámara wrote:
>The aproach you are using on
>http://www.csl.cornell.edu/~vince/projects/qemu-trace/ to get the PC dump
>is similar to mine but as you dont disable the TB caches It is not a full
>execution trace.
I can assure you that it does in fact work.
> You only dump the PC when they are compiled and inserted
> on the TB cache.
no, look closer. My code at instrumentation time inserts a helper-op
after each instruction. This is like a virtual instruction that lives in
the TB and calls my counting function. So each time the TB is re-executed
the calls happen again, as they are part of the TB instruction stream.
> Did you get correct values for your counters in bbvs[bb] ?
yes. And the resuts match valgrind, pin, and hardware performance
counters.
Vince
____
De: Vince Weaver
Para: Boris Cámara
Cc: qemu-devel@nongnu.org
Enviadas: Segunda-feira, 5 de Abril de 2010 22:41:52
Assunto: Re: [Qemu-devel] full dynamic instruction trace for MIPS target
> I think the correct way to get the full instruction trace on a MIPS
> emulated processor is:
the way you describe is slow because you are constantly re-generating the
TBs. The best way to do this is to add your instrumentation to the TBs.
I have code that does that for a recent version of Qemu here:
http://www.csl.cornell.edu/~vince/projects/qemusim/
although it outputs Basic-Block vectors, not a full memory trace like you
want. It has been validated to match proper instruction counts using
hardware performnce counters though.
I also have code creating full instruction/memory traces for Qemu MIPS
that can be found here:
http://www.csl.cornell.edu/~vince/projects/qemu-trace/
but it's against Qemu from 2007 pre-dating the TCG changeover so of
limited use probably. I hvae some code somewhere that updated this to
work with TCG but I don't know what happened to it.
Vince
Veja quais são os assuntos do momento no Yahoo! +Buscados
http://br.maisbuscados.yahoo.com
--
/* Vince Weaver vi...@csl.cornell.edu http://csl.cornell.edu/~vince */
main(){char O,o[66]="|\n\\/_ ",*I=o+7,l[]="B!FhhBHCWE9C?cJFKET$+h'Iq*chT"
,i=0,_;while(_=l[i++])for(O=0;O++<_>>5;)*I=*(I++-(_&31));*I=0;puts(o+5);}
Re: [Qemu-devel] full dynamic instruction trace for MIPS target
> I think the correct way to get the full instruction trace on a MIPS > emulated processor is: the way you describe is slow because you are constantly re-generating the TBs. The best way to do this is to add your instrumentation to the TBs. I have code that does that for a recent version of Qemu here: http://www.csl.cornell.edu/~vince/projects/qemusim/ although it outputs Basic-Block vectors, not a full memory trace like you want. It has been validated to match proper instruction counts using hardware performnce counters though. I also have code creating full instruction/memory traces for Qemu MIPS that can be found here: http://www.csl.cornell.edu/~vince/projects/qemu-trace/ but it's against Qemu from 2007 pre-dating the TCG changeover so of limited use probably. I hvae some code somewhere that updated this to work with TCG but I don't know what happened to it. Vince
Re: [Qemu-devel] [PATCH] User mode: Handle x86_64 vsyscall
On Sun, 7 Feb 2010, Richard Henderson wrote: > > I imagine that QEMU's VDSO would not have the complicated bits that the > kernel's version does, where it arranges to read the clock without going into > kernel space. I imagine QEMU would simply stuff a normal syscall sequence in > there, which would automatically be emulated in the normal way. For what it's worth, this is how various other systems I'm aware of handle x86_64 VDSOs (both Valgrind and the m5 simulator do it this way). Vince
[Qemu-devel] [patch] alpha-linux-user stat64 issue
Hello
The stat64/fstat64 syscalls are broken for alpha linux-user.
This is because Alpha, even though it is native 64-bits, has a stat64
syscall that is different than regular stat. This means that the
"TARGET_LONG_BITS==64" check in syscall.c isn't enough. Below is
a patch that fixes things for me, although it might not be the cleanest
fix.
This issue keeps sixtrack and fma3d spec2k benchmarks from running.
Vince
Signed-off-by: Vince Weaver
diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index 1acf1f5..f2dd39e 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -4004,7 +4004,7 @@ static inline abi_long host_to_target_stat64(void
*cpu_env,
} else
#endif
{
-#if TARGET_LONG_BITS == 64
+#if (TARGET_LONG_BITS == 64) && (!defined(TARGET_ALPHA))
struct target_stat *target_st;
#else
struct target_stat64 *target_st;
Re: [Qemu-devel] target-alpha: An approach to fp insn qualifiers
On Mon, 14 Dec 2009, Richard Henderson wrote: > On 12/14/2009 04:31 PM, Richard Henderson wrote: > > Hmph. One more patch for correctness. With this 183.equake runs correctly. I just finished running all of spec2k Alpha through Qemu. With these patches installed (the 4 fpu ones, and the 5 from the earlier thread) on top of current Qemu, pretty much all of the floating point issues have been fixed! Before, at least 19 of the spec2k benchmarks didn't run for various reasons. Now only two fail, sixtrack and fma3d. And I think those two are probably syscall related, though I haven't had chance to look in detail. #sixtrackAt line 343 of file daten.f Fortran runtime error: End of file #fma3d FMA-3D> Fatal Error Reported. Job Terminated. So it would be great if the alpha patches could be merged. Vince
Re: [Qemu-devel] target-alpha: An approach to fp insn qualifiers
On Mon, 14 Dec 2009, Richard Henderson wrote: > On 12/14/2009 04:31 PM, Richard Henderson wrote: > > Hmph. One more patch for correctness. With this 183.equake runs correctly. > I couldn't remember all the hoops to get runspec.pl to work, to do the whole > testsuite, but I did run this one by hand. This is great! About a year ago I had been steadily working on getting all of spec2k to run under alpha qemu but then other projects came up and I didn't have time. I'll re-run once the patches get applied to see if any more of the benchmarks have issues. My notes from before your patches showed the following benchmarks had issues: perlbmk.perfect WRONG_OUTPUT eon.rushmeier wrong_results eon.kajiyawrong_results eon.cook wrong_results art.110 never finished art.470 never finished vpr.place Arithmetic trap. vpr.route Arithmetic trap. mesa Arithmetic trap. wupwise wrong results (NaN) lucas Arithmetic trap. equakewrong results (NaN) sixtrack Fortran runtime error: Invalid string input in item 1 facerec Arithmetic trap. fma3d MMU data fault ammp Arithmetic trap. galgelbroken? twolf Arithmetic trap. apsi MMU data fault Vince
[Qemu-devel] [patch] fix mmap 0.9.1 mmap regression in linux-user
Hello The 0.9.1 release segfaults on certain benchmarks under linux-user emulation (x86 on x86). Specifically "facerec" from the SPEC2000 benchmarks, and over half of the SPEC2006 benchmarks. I tracked this down to a problem in the mmap() code in linux-user. I've attached a patch that reverts behavior back to that from 0.9.0, which makes the benchmarks all run for me. The problem is that after a large number of mmap() calls, the mmap_find_vma() fuction sometimes completely destroys the memory map table. I can produce debug info on this if needed. What the patch does is make the code ignore the results of mmap_find_vma(). I'm not familiar enough with the qemu mmap() code to fix this properly... Thanks Vince --- ./qemu-0.9.1-stock/linux-user/mmap.c2008-01-06 14:38:42.0 -0500 +++ ./qemu-0.9.1/linux-user/mmap.c 2008-01-17 13:27:11.0 -0500 @@ -250,8 +250,8 @@ /* Note: we prefer to control the mapping address. It is especially important if qemu_host_page_size > qemu_real_host_page_size */ -p = mmap(g2h(mmap_start), - host_len, prot, flags | MAP_FIXED, fd, host_offset); +p = mmap(real_start ? g2h(real_start) : NULL, + host_len, prot, flags, fd, host_offset); if (p == MAP_FAILED) return -1; /* update start so that it points to the file position at 'offset' */
Re: [Qemu-devel] sh4 mov.b fix
> I think this patch is needed for proper implementation of > the sh4 mov.b @(disp,Rm),R0 instruction. and of course, I managed to send a reversed patch. That will teach me to send out things like that late at night. Here's the proper patch. Vince --- ./qemu-snapshot-2007-08-14_05/target-sh4/translate.c.orig 2007-08-24 01:37:56.0 -0400 +++ ./qemu-snapshot-2007-08-14_05/target-sh4/translate.c2007-08-24 01:48:47.0 -0400 @@ -879,10 +879,10 @@ gen_op_stw_T0_T1(ctx); return; case 0x8400: /* mov.b @(disp,Rn),R0 */ - gen_op_movl_rN_T0(REG(0)); - gen_op_movl_rN_T1(REG(B7_4)); - gen_op_addl_imm_T1(B3_0); - gen_op_stb_T0_T1(ctx); + gen_op_movl_rN_T0(REG(B7_4)); + gen_op_addl_imm_T0(B3_0); + gen_op_ldb_T0_T0(ctx); + gen_op_movl_T0_rN(REG(0)); return; case 0x8500: /* mov.w @(disp,Rn),R0 */ gen_op_movl_rN_T0(REG(B7_4));
[Qemu-devel] sh4 mov.b fix
Hello I think this patch is needed for proper implementation of the sh4 mov.b @(disp,Rm),R0 instruction. Vince --- ./qemu-snapshot-2007-08-14_05/target-sh4/translate.c2007-08-24 01:48:47.0 -0400 +++ ./qemu-snapshot-2007-08-14_05/target-sh4/translate.c.orig 2007-08-24 01:37:56.0 -0400 @@ -879,10 +879,10 @@ gen_op_stw_T0_T1(ctx); return; case 0x8400: /* mov.b @(disp,Rn),R0 */ - gen_op_movl_rN_T0(REG(B7_4)); - gen_op_addl_imm_T0(B3_0); - gen_op_ldb_T0_T0(ctx); - gen_op_movl_T0_rN(REG(0)); + gen_op_movl_rN_T0(REG(0)); + gen_op_movl_rN_T1(REG(B7_4)); + gen_op_addl_imm_T1(B3_0); + gen_op_stb_T0_T1(ctx); return; case 0x8500: /* mov.w @(disp,Rn),R0 */ gen_op_movl_rN_T0(REG(B7_4));
[Qemu-devel] dumping processor state after each instruction executed
Hello, I was wondering if it is possible to insert some code into the translated basic-block in order to dump the processor state to a file after every single instcution? This would be every single executed instruction, (meaning it couldn't be just dump the state once at translation time). I'm trying to use qemu as a frontend to the TAXI tool (http://www.eecs.umich.edu/~vlaovic/L150_vlaovic_s.pdf) while doing some computer architecture work. I've figured out how to add a helper function to the load/store routines to get a memory trace file, but getting an instruction trace is proving to be a bit harder. Thanks for any help, Vince Weaver [EMAIL PROTECTED] ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
