Re: [Xen-devel] [PATCH v3] xen: sched: removal of redundant check in Credit

2016-12-28 Thread George Dunlap
On 17/12/16 01:46, Dario Faggioli wrote: > On Sat, 2016-12-17 at 00:53 +0530, Praveen Kumar wrote: >> The patch gets rid of a redundant check in csched_vcpu_acct. In fact, >> the function is only called from csched_tick, which already checks >> that current is not the idle vcpu. The patch also adds

[Xen-devel] [xen-unstable test] 103982: tolerable FAIL

2016-12-28 Thread osstest service owner
flight 103982 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/103982/ Failures :-/ but no regressions. Regressions which are regarded as allowable (not blocking): test-armhf-armhf-libvirt-xsm 13 saverestore-support-checkfail like 103975 test-amd64-i386-xl-qemuu-wi

Re: [Xen-devel] [PATCH] arm64: Do not include linux/uaccess.h from assembler files

2016-12-28 Thread Thomas Petazzoni
Hello, On Mon, 26 Dec 2016 02:31:08 -0800, Guenter Roeck wrote: > Including linux/uaccess.h from assembler files in arm64 builds results > in the following build errors. > > In file included from arm64/include/asm/asm-offsets.h:1:0, > from arch/arm64/include/asm/assembler.h:26, >

[Xen-devel] [xen-unstable-coverity test] 103985: all pass - PUSHED

2016-12-28 Thread osstest service owner
flight 103985 xen-unstable-coverity real [real] http://logs.test-lab.xenproject.org/osstest/logs/103985/ Perfect :-) All tests in this flight passed as required version targeted for testing: xen ee524f2bfa681ad116b8ae925fa8f3f18ee12ba5 baseline version: xen ad89

[Xen-devel] [distros-debian-squeeze test] 68280: tolerable FAIL

2016-12-28 Thread Platform Team regression test user
flight 68280 distros-debian-squeeze real [real] http://osstest.xs.citrite.net/~osstest/testlogs/logs/68280/ Failures :-/ but no regressions. Regressions which are regarded as allowable (not blocking): test-amd64-amd64-amd64-squeeze-netboot-pygrub 9 debian-di-install fail like 68252 test-amd64-

[Xen-devel] [ovmf test] 103984: all pass - PUSHED

2016-12-28 Thread osstest service owner
flight 103984 ovmf real [real] http://logs.test-lab.xenproject.org/osstest/logs/103984/ Perfect :-) All tests in this flight passed as required version targeted for testing: ovmf 31d060d94e65c41ccca2b68f7908c0c3b4ac3df4 baseline version: ovmf 1f20b2988cb223907e64a

Re: [Xen-devel] [PATCH 02/10] x86/guest-walk: use unambiguous register names

2016-12-28 Thread George Dunlap
On Tue, Dec 20, 2016 at 10:36 AM, Jan Beulich wrote: > This is in preparation of eliminating the mis-naming of 64-bit fields > with 32-bit register names (eflags instead of rflags etc). Use the > guaranteed 32-bit underscore prefixed names for now where appropriate. > > Signed-off-by: Jan Beulich

Re: [Xen-devel] [RFC] Xen PV Drivers Lifecycle

2016-12-28 Thread George Dunlap
On Tue, Dec 20, 2016 at 11:09 PM, Stefano Stabellini wrote: > On Tue, 20 Dec 2016, Jan Beulich wrote: >> >>> On 20.12.16 at 01:47, wrote: >> > ## Design Phase >> > >> > The first step toward acceptance of a new PV protocol is to write a >> > design document and send it to xen-devel. It should cov

[Xen-devel] [linux-4.1 test] 103978: regressions - FAIL

2016-12-28 Thread osstest service owner
flight 103978 linux-4.1 real [real] http://logs.test-lab.xenproject.org/osstest/logs/103978/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-armhf-armhf-libvirt 11 guest-start fail REGR. vs. 101737 test-amd64-i386-xl-qe

Re: [Xen-devel] [PATCH] x86/emulate: Don't assume that addr_size == 32 implies protected mode

2016-12-28 Thread George Dunlap
On 16/12/16 10:34, Jan Beulich wrote: On 16.12.16 at 10:55, wrote: >> Callers of x86_emulate() generally define addr_size based on the code >> segment. In vm86 mode, the code segment is set by the hardware to be >> 16-bits; but it is entirely possible to enable protected mode, set the >> CS

Re: [Xen-devel] [PATCH 02/10] x86/guest-walk: use unambiguous register names

2016-12-28 Thread Jan Beulich
>>> George Dunlap 12/28/16 12:18 PM >>> >On Tue, Dec 20, 2016 at 10:36 AM, Jan Beulich wrote: >> This is in preparation of eliminating the mis-naming of 64-bit fields >> with 32-bit register names (eflags instead of rflags etc). Use the >> guaranteed 32-bit underscore prefixed names for now where

Re: [Xen-devel] [PATCH] x86/emulate: Don't assume that addr_size == 32 implies protected mode

2016-12-28 Thread Jan Beulich
>>> George Dunlap 12/28/16 2:34 PM >>> >On 16/12/16 10:34, Jan Beulich wrote: > On 16.12.16 at 10:55, wrote: >>> Callers of x86_emulate() generally define addr_size based on the code >>> segment. In vm86 mode, the code segment is set by the hardware to be >>> 16-bits; but it is entirely poss

Re: [Xen-devel] [PATCH v3] xen: sched: removal of redundant check in Credit

2016-12-28 Thread Dario Faggioli
On Wed, 2016-12-28 at 09:02 +, George Dunlap wrote: > On 17/12/16 01:46, Dario Faggioli wrote: > > On Sat, 2016-12-17 at 00:53 +0530, Praveen Kumar wrote: > > > Signed-off-by: Praveen Kumar > > > Acked-by: Dario Faggioli > > > > > Better than before. But still, if I: > >  - save this mail as

[Xen-devel] [ovmf baseline-only test] 68282: all pass

2016-12-28 Thread Platform Team regression test user
This run is configured for baseline tests only. flight 68282 ovmf real [real] http://osstest.xs.citrite.net/~osstest/testlogs/logs/68282/ Perfect :-) All tests in this flight passed as required version targeted for testing: ovmf 31d060d94e65c41ccca2b68f7908c0c3b4ac3df4 baseline v

[Xen-devel] [ovmf test] 103986: all pass - PUSHED

2016-12-28 Thread osstest service owner
flight 103986 ovmf real [real] http://logs.test-lab.xenproject.org/osstest/logs/103986/ Perfect :-) All tests in this flight passed as required version targeted for testing: ovmf af8ba51aca4e0b41a359fe467fb5c5b9baa75a05 baseline version: ovmf 31d060d94e65c41ccca2b

Re: [Xen-devel] [xl restore and migrating problem - hardware compatibility]

2016-12-28 Thread Julien Grall
(Adding Ian and Wei) Hi Andrew, On 23/12/16 17:00, Andrew Cooper wrote: On 23/12/16 16:32, Ing. Ricardo Brisighelli wrote: Hi, i'm gentoo user and try with xen 4.6.3 and 4.7.1 in both version have the same problem. My cpu is AMD A10-7860K This issue has been reported before ("[Xen-devel] "X

Re: [Xen-devel] [xl restore and migrating problem - hardware compatibility]

2016-12-28 Thread Julien Grall
create ^ severity it blocker affects it 4.6, 4.7, 4.8 thanks On 23/12/16 17:00, Andrew Cooper wrote: On 23/12/16 16:32, Ing. Ricardo Brisighelli wrote: Hi, i'm gentoo user and try with xen 4.6.3 and 4.7.1 in both version have the same problem. My cpu is AMD A10-7860K This issue has been rep

[Xen-devel] Processed: Re: [xl restore and migrating problem - hardware compatibility]

2016-12-28 Thread xen
Processing commands for x...@bugs.xenproject.org: > create ^ Created new bug #56 rooted at `' Title: `Re: [Xen-devel] [xl restore and migrating problem - hardware compatibility]' > severity it blocker Change severity for #56 to `blocker' > affects it 4.6, 4.7, 4.8 Bug #56 affects `4.6, 4.7, 4.8'

Re: [Xen-devel] [PATCH] xen/arm: fix rank/vgic locks inversion bug

2016-12-28 Thread Julien Grall
Hi Stefano, On 20/12/16 19:38, Stefano Stabellini wrote: On Tue, 20 Dec 2016, Julien Grall wrote: On 20/12/2016 00:22, Stefano Stabellini wrote: On Mon, 19 Dec 2016, Julien Grall wrote: On 19/12/2016 23:30, Stefano Stabellini wrote: On Mon, 19 Dec 2016, Julien Grall wrote: 2) We run gic_upd

Re: [Xen-devel] [PATCH v2 3/4] arm, vgic_migrate_irq: take the right vgic lock

2016-12-28 Thread Julien Grall
Hi Stefano, On 22/12/16 02:15, Stefano Stabellini wrote: Always take the vgic lock of the old vcpu. When more than one irq migration is requested before the first one completes, take the vgic lock of the oldest vcpu. Write the new vcpu id into the rank from vgic_migrate_irq, protected by the ol

[Xen-devel] [ovmf baseline-only test] 68283: all pass

2016-12-28 Thread Platform Team regression test user
This run is configured for baseline tests only. flight 68283 ovmf real [real] http://osstest.xs.citrite.net/~osstest/testlogs/logs/68283/ Perfect :-) All tests in this flight passed as required version targeted for testing: ovmf af8ba51aca4e0b41a359fe467fb5c5b9baa75a05 baseline v

Re: [Xen-devel] [PATCH v2 4/4] The locking order is: first rank lock, then vgic lock. The order is respected everywhere, except for gic_update_one_lr.

2016-12-28 Thread Julien Grall
Hi Stefano, On 22/12/16 02:15, Stefano Stabellini wrote: gic_update_one_lr is called with the vgic lock held, but it calls vgic_get_target_vcpu, which tries to obtain the rank lock. This can cause deadlocks. We already have a version of vgic_get_target_vcpu that doesn't take the rank lock: __vg

Re: [Xen-devel] [PATCH v2 1/4] xen/arm: fix GIC_INVALID_LR

2016-12-28 Thread Julien Grall
Hi Stefano, On 22/12/16 02:15, Stefano Stabellini wrote: GIC_INVALID_LR should be 0xff, but actually, defined as ~(uint8_t)0, is 0x. Fix the problem by placing the ~ operator before the cast. Signed-off-by: Stefano Stabellini Reviewed-by: Julien Grall Regards, -- Julien Grall ___

Re: [Xen-devel] Xen ARM - Exposing a PL011 to the guest

2016-12-28 Thread Julien Grall
Hi Stefano, On 21/12/16 22:12, Stefano Stabellini wrote: On Wed, 21 Dec 2016, Julien Grall wrote: On 20/12/2016 20:53, Stefano Stabellini wrote: On Tue, 20 Dec 2016, Julien Grall wrote: On 19/12/2016 21:24, Stefano Stabellini wrote: On Mon, 19 Dec 2016, Christoffer Dall wrote: On Fri, Dec 1

Re: [Xen-devel] Xen ARM - Exposing a PL011 to the guest

2016-12-28 Thread Julien Grall
On 28/12/16 17:49, Julien Grall wrote: On 21/12/16 22:12, Stefano Stabellini wrote: On Wed, 21 Dec 2016, Julien Grall wrote: On 20/12/2016 20:53, Stefano Stabellini wrote: On Tue, 20 Dec 2016, Julien Grall wrote: On 19/12/2016 21:24, Stefano Stabellini wrote: On Mon, 19 Dec 2016, Christoff

[Xen-devel] [linux-3.18 test] 103983: tolerable FAIL - PUSHED

2016-12-28 Thread osstest service owner
flight 103983 linux-3.18 real [real] http://logs.test-lab.xenproject.org/osstest/logs/103983/ Failures :-/ but no regressions. Regressions which are regarded as allowable (not blocking): test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stopfail like 101675 test-armhf-armhf-libvirt-xsm

Re: [Xen-devel] Xen: ARM: Support for mapping ECAM PCIe Config Space Specified In Static ACPI Table

2016-12-28 Thread Julien Grall
(CC Andrew and Jan) Hi Stefano, On 20/12/16 22:33, Stefano Stabellini wrote: On Tue, 20 Dec 2016, Julien Grall wrote: Hi Stefano, On 20/12/2016 00:54, Stefano Stabellini wrote: On Mon, 19 Dec 2016, Julien Grall wrote: On 16/12/2016 15:49, Julien Grall wrote: On 14/12/16 08:00, Jiandi An wr

[Xen-devel] [linux-4.1 test] 103988: regressions - FAIL

2016-12-28 Thread osstest service owner
flight 103988 linux-4.1 real [real] http://logs.test-lab.xenproject.org/osstest/logs/103988/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-armhf-armhf-libvirt 11 guest-start fail REGR. vs. 101737 Tests which are faili

[Xen-devel] (no subject)

2016-12-28 Thread Ronald Rojas
The first 57 commits are merged from previous work done by George Dunlap at (https://github.com/gwd/schedbench) and implement manipulating Cpu pool. The last 2 commits merge his work onto the Xen tree and implement finding system information and throwing errors. CC: xen-devel CC: Ian Jackson

[Xen-devel] [PATCH RFC 01/59] Initial controller framework

2016-12-28 Thread Ronald Rojas
From: George Dunlap Go code to execute a worker and parse its output. The go json marshaller needs capitalized structure elements; make the output match. Also add fflush()-es to make sure that output actually gets to the controller in a timely manner. Signed-off-by: George Dunlap --- Makefil

[Xen-devel] [PATCH RFC 02/59] controller: Revamp communication structure

2016-12-28 Thread Ronald Rojas
From: George Dunlap Two general-purporse channels rather than one per worker. Set up two workers, gather and collate the information at the central station. Have the worker print out a report at start-of-day, so we get timing information for the first actual report. Catch SIGINT as a shorthand

[Xen-devel] [PATCH RFC 05/59] controller: Add WorkerParams argument to Init in Worker interface

2016-12-28 Thread Ronald Rojas
From: George Dunlap For now just make it a plain []string; later make it more abstrcted. Signed-off-by: George Dunlap --- main.go | 22 +- processworker.go | 4 ++-- 2 files changed, 19 insertions(+), 7 deletions(-) diff --git a/main.go b/main.go index a4986cf..9

[Xen-devel] [PATCH RFC 04/59] Controller: Move process worker into its own file

2016-12-28 Thread Ronald Rojas
From: George Dunlap Also fix Makefile rule to use correct automatic variable. Signed-off-by: George Dunlap --- Makefile | 4 ++-- main.go | 59 -- processworker.go | 66 3

[Xen-devel] [PATCH RFC 08/59] controller: Exit after second SIGINT

2016-12-28 Thread Ronald Rojas
From: George Dunlap Otherwise you have to kill -9 the process. Signed-off-by: George Dunlap --- main.go | 14 +++--- 1 file changed, 11 insertions(+), 3 deletions(-) diff --git a/main.go b/main.go index 04c8467..dbd6276 100644 --- a/main.go +++ b/main.go @@ -62,6 +62,8 @@ func (ws *Wo

[Xen-devel] [PATCH RFC 06/59] Reorganize to enable "Dist" directory

2016-12-28 Thread Ronald Rojas
From: George Dunlap Make a toplevel Makefile which defaults to 'make dist'. Add "make dist" targets in all subdirs, which copy the distribution files into $PWD/dist. Modify processworker.go such that it looks for worker-proc in the same directory. Signed-off-by: George Dunlap --- Makefile

[Xen-devel] [PATCH RFC 07/59] controller: Introduce basic Xen functionality

2016-12-28 Thread Ronald Rojas
From: George Dunlap Rough-and-ready execution of xl commands, but they work, amazingly. Signed-off-by: George Dunlap --- Makefile | 2 +- main.go | 3 +- xenworker.go | 231 +++ 3 files changed, 234 insertions(+), 2 deletions

[Xen-devel] [PATCH RFC 25/59] main: Change default workload to something a bit more extreme

2016-12-28 Thread Ronald Rojas
From: George Dunlap Signed-off-by: George Dunlap --- main.go | 19 +-- 1 file changed, 9 insertions(+), 10 deletions(-) diff --git a/main.go b/main.go index bb46dbc..61d3949 100644 --- a/main.go +++ b/main.go @@ -29,15 +29,15 @@ func main() { switch(os.Args[1]) {

[Xen-devel] [PATCH RFC 10/59] controller: First cut at BenchmarkParams

2016-12-28 Thread Ronald Rojas
From: George Dunlap Struct for configurable worker sets, as well as a set runtime. Signed-off-by: George Dunlap --- main.go | 75 ++--- 1 file changed, 53 insertions(+), 22 deletions(-) diff --git a/main.go b/main.go index 89e9cf3..7

[Xen-devel] [PATCH RFC 20/59] xenworker: Use libxl_domain_unpause rather than forking xl

2016-12-28 Thread Ronald Rojas
From: George Dunlap Unpause is a time-sensitive operation; use libxl to unpause directly rather than forking and execing xl for each VM. Signed-off-by: George Dunlap --- libxl.go | 14 ++ xenworker.go | 12 ++-- 2 files changed, 16 insertions(+), 10 deletions(-) diff -

[Xen-devel] [PATCH RFC 15/59] controller: Move "running" code to a separate file

2016-12-28 Thread Ronald Rojas
From: George Dunlap Signed-off-by: George Dunlap --- Makefile | 2 +- benchmark.go | 153 -- run.go | 178 +++ 3 files changed, 179 insertions(+), 154 deletions(-) create mode 10

[Xen-devel] [PATCH RFC 45/59] libxl: Implement Destroy, Add/Remove operations

2016-12-28 Thread Ronald Rojas
From: George Dunlap Also: * Implement useful Bitmap operations: SetRange(), ClearRange(), IsEmpty(), and And(). * Normalize CpupoolInfo element naming. * Implement CpupoolMakeFree, which will scan through cpupools freeing the cpus in the map. Signed-off-by: George Dunlap --- libxl.go | 18

[Xen-devel] [PATCH RFC 17/59] controller: Collect and display statistics on WorkerSets

2016-12-28 Thread Ronald Rojas
From: George Dunlap Introduce WorkerSetSummary, to contain WorkerSummary's and collect data for the set. For now "if false"-out reporting of individual worker averages; we can add this back in as an option later. Signed-off-by: George Dunlap --- benchmark.go | 156

[Xen-devel] [PATCH RFC 03/59] controller: Initial attempt to generalize process / vm creation

2016-12-28 Thread Ronald Rojas
From: George Dunlap Create a "Worker" interface, and have ProcessWorker implement that interface. Rename "Worker" to "WorkerReport" to make room for the interface. Signed-off-by: George Dunlap --- main.go | 45 +++-- 1 file changed, 27 insertions(+), 18

[Xen-devel] [PATCH RFC 19/59] Add basic libxl framework, get domain cpu_time

2016-12-28 Thread Ronald Rojas
From: George Dunlap Introduce libxl "Context" class with open, close, and dominfo. Create a global variable in xenworker.go to hold the context; open on first worker creation, close on last worker destruction. Add a new element to WorkerReport, Cputime, and print it out. For now, include hard-

[Xen-devel] [PATCH RFC 26/59] Use kops rather than mops

2016-12-28 Thread Ronald Rojas
From: George Dunlap 1 million operations on my test box takes about 3ms -- meaning the minimum granularity for how long to burn cpu is far longer than typical. Make this kops instead, giving us a minimum granularity of 3us. Update most of the default workers to have similar patterns but on a 1/

[Xen-devel] [PATCH RFC 24/59] report: Lots of changes

2016-12-28 Thread Ronald Rojas
From: George Dunlap Store raw worker reports in the worker summary for easy printing. Add "level" argument to textreport to easly switch between different levels of granularity. Report total throughput, time, and cputime for workers. When running, print plain time and aggregate cputime in addi

[Xen-devel] [PATCH RFC 23/59] run: Don't collect results reported after command to stop guests is issued

2016-12-28 Thread Ronald Rojas
From: George Dunlap Particularly on heavily-loaded systems, this can lead to skew as some guests have already stopped processing. Signed-off-by: George Dunlap --- run.go | 9 +++-- 1 file changed, 7 insertions(+), 2 deletions(-) diff --git a/run.go b/run.go index 4222001..41408dd 100644 -

[Xen-devel] [PATCH RFC 18/59] controller: Add cpupool global config

2016-12-28 Thread Ronald Rojas
From: George Dunlap Define WorkerConfig for worker configuration, add "Pool" as a pool in which to place the worker (currently implemented in Xen only). Add WorkerConfig options at the WorkerSet, BenchmarkRun, and BenchmarkPlan levels; allow local levels to "override" global config options. The

[Xen-devel] [PATCH RFC 44/59] libxl: Implement CpupoolCreate

2016-12-28 Thread Ronald Rojas
From: George Dunlap Implement CpupoolCreate. For now, follow the lead of "xl cpupool-create", by automatically generating a new uuid each time a new cpupool is created, and by not allowing the user to attempt to set the poolid. This requires us to implement bytmapGotoC, which C-allocates the ap

[Xen-devel] [PATCH RFC 57/59] tools/blktap2: remove unused inclusion of sys/sysctl.l

2016-12-28 Thread Ronald Rojas
From: Alistair Francis That header file is not used. Removing it would avoid build error with musl libc, which doesn't have that header file. Signed-off-by: Alistair Francis Reviewed-by: Doug Goldstein [ wei: rewrote commit message ] Acked-by: Wei Liu --- tools/blktap2/drivers/block-remus.c

[Xen-devel] [PATCH RFC 52/59] libxl: Implement Bitmap.String()

2016-12-28 Thread Ronald Rojas
From: George Dunlap Implement Bitmap.String, which will convert a bitmap into a string parse-able by libxl. Signed-off-by: George Dunlap --- libxl.go | 59 ++- 1 file changed, 58 insertions(+), 1 deletion(-) diff --git a/libxl.go b/libxl

[Xen-devel] [PATCH RFC 11/59] Refactor to move towards benchmark "plans" and data analysis

2016-12-28 Thread Ronald Rojas
From: George Dunlap A bit of a roll-up of lots of bits. Signed-off-by: George Dunlap --- Makefile | 4 +- benchmark.go | 256 +++ main.go | 197 ++--- 3 files changed, 302 insertions(+),

[Xen-devel] [PATCH RFC 12/59] Basic 'report' functionality

2016-12-28 Thread Ronald Rojas
From: George Dunlap Go through the raw data and re-calculate throughput for each timeframe. Record for each worker the min and max of these; then take an average over the whole run. Have the report go through each and show the worker setup, as well as the avg, min, and max for each worker. Als

[Xen-devel] [PATCH RFC 21/59] Report utilization statistics

2016-12-28 Thread Ronald Rojas
From: George Dunlap Refactor the min/max into a separate type. Signed-off-by: George Dunlap --- benchmark.go | 119 +-- run.go | 4 +- 2 files changed, 78 insertions(+), 45 deletions(-) diff --git a/benchmark.go b/benchmark.go in

[Xen-devel] [PATCH RFC 22/59] Use tsc for time rather than rumpkernel clock_gettime()

2016-12-28 Thread Ronald Rojas
From: George Dunlap The rumpkernel clock_gettime() drifts drastically when the system is overloaded. Switch to using a TSC timesource instead: * Use tsc_mode='native' in guest config * Read the host tsc from /proc/cpuinfo when starting a run * Pass it as an argument to the worker * Implement no

[Xen-devel] [PATCH RFC 13/59] Add GPL headers / COPYING file (v2 only)

2016-12-28 Thread Ronald Rojas
From: George Dunlap Signed-off-by: George Dunlap --- benchmark.go | 18 ++ main.go | 19 +++ processworker.go | 18 ++ xenworker.go | 18 ++ 4 files changed, 73 insertions(+) diff --git a/benchmark.go b/benchmark.

[Xen-devel] [PATCH RFC 40/59] controller: Add / update GPL text

2016-12-28 Thread Ronald Rojas
From: George Dunlap Add it to files which are missing it; clarify the wording regarding version 2 only in others. Signed-off-by: George Dunlap --- benchmark.go | 2 +- main.go | 2 +- processworker.go | 2 +- xenworker.go | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-

[Xen-devel] [PATCH RFC 38/59] controller: Allow multiple schedulers in the same benchmark file

2016-12-28 Thread Ronald Rojas
From: George Dunlap First, add a RunConfig element to BenchmarkRun, which contains a (string) name of the scheduler for this run. Add run.Ready(), which if RunConfig.Scheduler is non-null, will check to see if the cpupool in which the VMs will run (either WorkerConfig.Pool or PoolID 0) has the s

[Xen-devel] [PATCH RFC 09/59] controller: Refactor creation and stopping of workers into WorkerList methods

2016-12-28 Thread Ronald Rojas
From: George Dunlap Signed-off-by: George Dunlap --- main.go | 59 ++- 1 file changed, 42 insertions(+), 17 deletions(-) diff --git a/main.go b/main.go index dbd6276..89e9cf3 100644 --- a/main.go +++ b/main.go @@ -55,10 +55,43 @@ func Rep

[Xen-devel] [PATCH RFC 53/59] controller: Add WorkerConfig.SoftAffinity

2016-12-28 Thread Ronald Rojas
From: George Dunlap If the SoftAffinity is present, pass it as "cpus_soft = ..." in the config file. Signed-off-by: George Dunlap --- benchmark.go | 1 + xenworker.go | 4 2 files changed, 5 insertions(+) diff --git a/benchmark.go b/benchmark.go index bd513fd..d5a0ac8 100644 --- a/benchm

[Xen-devel] [PATCH RFC 36/59] controller: Make 'dummy' at the level of 'run' rather than xenworker

2016-12-28 Thread Ronald Rojas
From: George Dunlap Get rid of xenworker_dummy.go and use stubs.go instead. This is much cleaner (as evidenced by the size of the file), and allows us to use libxl-based things in run.go without having to make a load of dummy libxl stub functions. Signed-off-by: George Dunlap --- Makefile

[Xen-devel] [PATCH RFC 33/59] controller: Add verbosity argument and update README with new instructions

2016-12-28 Thread Ronald Rojas
From: George Dunlap Signed-off-by: George Dunlap --- main.go | 13 - 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/main.go b/main.go index ddb768c..2aa8bae 100644 --- a/main.go +++ b/main.go @@ -30,6 +30,7 @@ func main() { Args = Args[1:] filename :

[Xen-devel] [PATCH RFC 27/59] report: Allow report verbosity to be specified

2016-12-28 Thread Ronald Rojas
From: George Dunlap Take a third argument to control the level of verbosity. Also add helpful deltas when reporting the raw results. Signed-off-by: George Dunlap --- benchmark.go | 17 +++-- main.go | 11 --- 2 files changed, 23 insertions(+), 5 deletions(-) diff --g

[Xen-devel] [PATCH RFC 35/59] libxl: Add ListCpupool

2016-12-28 Thread Ronald Rojas
From: George Dunlap Add libxl bindings for libxl_list_cpupool(). Define the Scheduler type (based on libxl_schedler) with appropriate constants and string functions. Punt on the cpumap part of CpupoolInfo for now. Also create "xltest" command for testing libxl bindings Signed-off-by: George D

[Xen-devel] [PATCH RFC 37/59] libxl.go: Provide a single global context by default

2016-12-28 Thread Ronald Rojas
From: George Dunlap Provide a single global context by default, Ctx. Allow this to be called with Ctx.Open() multiple times (but Close() only once). Move xenworker.go to use this global context instead; and open it when starting the benchmark (rather than opening and closing it for every run, w

[Xen-devel] [PATCH RFC 16/59] controller: Rename an element in BenchmarkRun to be more accurate

2016-12-28 Thread Ronald Rojas
From: George Dunlap Signed-off-by: George Dunlap --- benchmark.go | 6 +++--- main.go | 6 +++--- run.go | 2 +- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/benchmark.go b/benchmark.go index 2e03fe5..aaa4c98 100644 --- a/benchmark.go +++ b/benchmark.go @@ -84,7 +84

[Xen-devel] [PATCH RFC 31/59] Make a binary that can run reports on a system without libxl

2016-12-28 Thread Ronald Rojas
From: George Dunlap To simplify report-generation and iteration, make a binary that doesn't link against libxl. Create a "dummy" Xen library that just returns errors and warnings. A better way to do this is probably to do some dlopen hackery (so we can use the same binary), but that's for anoth

[Xen-devel] [PATCH RFC 58/59] remove irrelevant files from old repository

2016-12-28 Thread Ronald Rojas
Signed-off-by: Ronald Rojas --- Makefile | 31 -- benchmark.go | 467 -- htmlreport.go | 238 -- libxl.go | 720 - main.go

[Xen-devel] [PATCH RFC 43/59] libxl: Add bitmap support

2016-12-28 Thread Ronald Rojas
From: George Dunlap Make the structure compatible so that we can just copy things out or in. Use it to complete the Cpupool structure. Signed-off-by: George Dunlap --- libxl.go | 104 +++ 1 file changed, 98 insertions(+), 6 deletions

[Xen-devel] [PATCH RFC 14/59] benchmark: Store data in terms of worker sets and worker ids

2016-12-28 Thread Ronald Rojas
From: George Dunlap ...rather than having a single global index, in preparation for summarizing sets of workers as well as individual workers. To do this: - Define a WorkerId type which consists of the worker set + worker id within that set. Pass this to the workers and use this in the rep

[Xen-devel] [PATCH RFC 51/59] controller: Make and/or modify cpupools when possible

2016-12-28 Thread Ronald Rojas
From: George Dunlap Make it possible for schedbench to modify cpupools in order to satisfy run constraints. Make a "RunConfig" option which contains the name of the pool, the scheduler, and the number of cpus. As with WorkerConfig, make these automatically inherited from larger levels to smalle

[Xen-devel] [PATCH RFC 32/59] controller: Allow specification of an input file

2016-12-28 Thread Ronald Rojas
From: George Dunlap So that we can analyze data later if we want. Signed-off-by: George Dunlap --- main.go | 194 +++- 1 file changed, 106 insertions(+), 88 deletions(-) diff --git a/main.go b/main.go index f8d77cf..ddb768c 100644 --

[Xen-devel] [PATCH RFC 55/59] plan: Make the matrix generation more programmatic

2016-12-28 Thread Ronald Rojas
From: George Dunlap Rather than have nested loops, keep a running list of all the configs we've created so far. Then for each existing new element of the matrix, combine it with all the existing columns. Signed-off-by: George Dunlap --- plan.go | 50 +++

[Xen-devel] [PATCH RFC 50/59] libxl: Fix Bitmap.Max(), make Test() / Clear() more robust

2016-12-28 Thread Ronald Rojas
From: George Dunlap Bitmap.Max() should return the largest currently addressable bit; at the moment it returns the total number of bits (which is one greater than the largest addressable bit). This also implicitly fixes a bug in Test() and Clear() which would cause an out-of-range error when tes

[Xen-devel] [PATCH RFC 30/59] htmlreport: Include utilization scatterplots

2016-12-28 Thread Ronald Rojas
From: George Dunlap Reformat RunRaw so that it's a more generic scatterplot, with the caller specifying the javascript tag, title, and axes. Keep them as a straight list to be displayed rather than defining them on a per-run basis. Collect a slice of utilization points at the same time as colle

[Xen-devel] [PATCH RFC 47/59] libxl: Reorganize code

2016-12-28 Thread Ronald Rojas
From: George Dunlap Put all type and enumeration definitions at the top of the file, separated into "builtins" and "idl-generated". Also define Uuid as C.libxl_uud. (Not sure if this gives us the functionality we need without having to manually copy things in.) Finally, get rid of Bitmap.Alloc

[Xen-devel] [PATCH RFC 28/59] controller: Handle worker early death

2016-12-28 Thread Ronald Rojas
From: George Dunlap Log raw worker output. In the event of an unexpected worker death, dump the output and stop further processing. Also fix an assert that caused workers to die if the timer was too exact. Signed-off-by: George Dunlap --- benchmark.go | 7 --- processworker.go | 18

[Xen-devel] [PATCH RFC 29/59] report: Add basic html report

2016-12-28 Thread Ronald Rojas
From: George Dunlap Use Google's chartapi to create a (mostly) self-contained html file. Start with just scatterplots of the raw data for proof-of-concept. Signed-off-by: George Dunlap --- Makefile | 2 +- benchmark.go | 1 + htmlreport.go | 233 ++

[Xen-devel] [PATCH RFC 41/59] libxl.go: Link statically rather than dynamically

2016-12-28 Thread Ronald Rojas
From: George Dunlap Link statically rather than dynamically, because Go prefers that. This also gets rid of the need for schedbench-report, so disable that from the build (although keep the rule and the dummy file around just in case. Also update README.md. Signed-off-by: George Dunlap --- M

[Xen-devel] [PATCH RFC 34/59] controller: Make a useful config file

2016-12-28 Thread Ronald Rojas
From: George Dunlap Allow a simple "skeleton" config file which can be adapted for the particular use case. Have 'plan' expand this with the real benchmark plan. Include "sample.bench" as a template to modify. Also update the README.md accordingly. Signed-off-by: George Dunlap --- Makefile

[Xen-devel] [PATCH RFC 42/59] plan: Allow "templating" from other runs

2016-12-28 Thread Ronald Rojas
From: George Dunlap Allow 'plan' to use an existing run as a "template": load the template, discard the runs, and save the result in the new file. Use like this: ./schedbench -t 18.bench -f 20.bench plan This will make 20 with an empty plan identical to that in 18. Signed-off-by: George Dunla

[Xen-devel] [PATCH RFC 46/59] libxl: Reorganize bitmapGotoC

2016-12-28 Thread Ronald Rojas
From: George Dunlap No need to "allocate" the structure elsewhere; passing as a return value makes usage a lot cleaner. Signed-off-by: George Dunlap --- libxl.go | 15 +++ 1 file changed, 7 insertions(+), 8 deletions(-) diff --git a/libxl.go b/libxl.go index 92064ca..16f4645 10064

[Xen-devel] [PATCH RFC 48/59] libxl: Add Ctx.CheckOpen

2016-12-28 Thread Ronald Rojas
From: George Dunlap To standardize the error in the case that Ctx isn't open. Add checks to all functions which call libxl functions directly. Signed-off-by: George Dunlap --- libxl.go | 48 +--- 1 file changed, 45 insertions(+), 3 deletions(-) di

[Xen-devel] [PATCH RFC 39/59] libxl.go: Put common link flags in libxl.go

2016-12-28 Thread Ronald Rojas
From: George Dunlap ...so that the only things added in the makefile are extra flags required when linking against non-installed builds. Add in the text of the GPL while we're at it. Signed-off-by: George Dunlap --- Makefile | 3 +-- libxl.go | 19 +++ 2 files changed, 20 ins

[Xen-devel] [PATCH RFC 49/59] libxl: Implement libxl_cpupool_info and Scheduler.FromString()

2016-12-28 Thread Ronald Rojas
From: George Dunlap Unify the libxl_cpupoolinfo -> CpupoolInfo translation in a function. Also make bitmapCToGo pass-by-value rather than pass-by-reference, since it only has two elements and doesn't need to be modified. Signed-off-by: George Dunlap --- libxl.go | 64 +

[Xen-devel] [PATCH RFC 54/59] controller/run: Add RunConfig.NumaDisable

2016-12-28 Thread Ronald Rojas
From: George Dunlap If RunConfig.NumaDisable is set, get each worker's WorkerConfig.SoftAffinity to the cpumap of the cpupool the VM is going to be running in. Signed-off-by: George Dunlap --- benchmark.go | 4 run.go | 35 --- 2 files changed, 36 in

[Xen-devel] [PATCH RFC 59/59] tools/xenlight: Create interface for xenlight info

2016-12-28 Thread Ronald Rojas
Create interface to interact with libxl_verions_info and libxl_physinfo. Also introduce proper error handling. Signed-off-by: Ronald Rojas --- tools/golang/xenlight/libxl.go| 720 -- tools/golang/xenlight/xenlight.go | 1000 + 2 fi

[Xen-devel] [PATCH RFC 56/59] controller/plan: Add NumaDisable to SimpleMatrix

2016-12-28 Thread Ronald Rojas
From: George Dunlap Signed-off-by: George Dunlap --- plan.go | 22 ++ 1 file changed, 22 insertions(+) diff --git a/plan.go b/plan.go index 736d9f3..b8e0c6b 100644 --- a/plan.go +++ b/plan.go @@ -26,6 +26,7 @@ type PlanSimpleMatrix struct { Schedulers []string

[Xen-devel] [linux-4.1 test] 103991: regressions - FAIL

2016-12-28 Thread osstest service owner
flight 103991 linux-4.1 real [real] http://logs.test-lab.xenproject.org/osstest/logs/103991/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-armhf-armhf-libvirt 11 guest-start fail REGR. vs. 101737 Regressions which are

[Xen-devel] [ovmf test] 103993: all pass - PUSHED

2016-12-28 Thread osstest service owner
flight 103993 ovmf real [real] http://logs.test-lab.xenproject.org/osstest/logs/103993/ Perfect :-) All tests in this flight passed as required version targeted for testing: ovmf d9bb3ca3c2a1bd3f7b78adf27e82e5605f665e16 baseline version: ovmf af8ba51aca4e0b41a359f

[Xen-devel] [qemu-mainline baseline-only test] 68281: regressions - FAIL

2016-12-28 Thread Platform Team regression test user
This run is configured for baseline tests only. flight 68281 qemu-mainline real [real] http://osstest.xs.citrite.net/~osstest/testlogs/logs/68281/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-amd64-libvirt-vhd 6 xen-boot

[Xen-devel] [linux-3.18 baseline-only test] 68284: regressions - FAIL

2016-12-28 Thread Platform Team regression test user
This run is configured for baseline tests only. flight 68284 linux-3.18 real [real] http://osstest.xs.citrite.net/~osstest/testlogs/logs/68284/ Regressions :-( Tests which did not succeed and are blocking, including tests which could not be run: test-amd64-i386-pair 21 guest-migrate/src_host/

[Xen-devel] [qemu-mainline test] 103992: tolerable FAIL - PUSHED

2016-12-28 Thread osstest service owner
flight 103992 qemu-mainline real [real] http://logs.test-lab.xenproject.org/osstest/logs/103992/ Failures :-/ but no regressions. Regressions which are regarded as allowable (not blocking): test-armhf-armhf-libvirt-xsm 13 saverestore-support-checkfail like 103980 test-armhf-armhf-libvirt-q