Re: [Xen-devel] [PATCH v9.1 02/16] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
On 17-10-24 06:27:26, Jan Beulich wrote: > >>> On 24.10.17 at 11:10, wrote: > > This patch renames PSR sysctl/domctl interfaces and related xsm policy to > > make them be general for all resource allocation features but not only > > for CAT. Then, we can resuse the interfaces for all allocation features. > > > > Basically, it changes 'psr_cat_op' to 'psr_alloc', and remove 'CAT_' from > > some > > macros. E.g.: > > 1. psr_cat_op -> psr_alloc > > 2. XEN_DOMCTL_psr_cat_op -> XEN_DOMCTL_psr_alloc > > 3. XEN_SYSCTL_psr_cat_op -> XEN_SYSCTL_psr_alloc > > 4. XEN_DOMCTL_PSR_CAT_SET_L3_CBM -> XEN_DOMCTL_PSR_SET_L3_CBM > > 5. XEN_SYSCTL_PSR_CAT_get_l3_info -> XEN_SYSCTL_PSR_get_l3_info > > > > Signed-off-by: Yi Sun > > Reviewed-by: Wei Liu > > Reviewed-by: Roger Pau Monné > > Acked-by: Jan Beulich > > Acked-by: Daniel De Graaf > > --- > > CC: Jan Beulich > > CC: Andrew Cooper > > CC: Wei Liu > > CC: Ian Jackson > > CC: Daniel De Graaf > > CC: Roger Pau Monné > > CC: Chao Peng > > > > v9: > > - rename 'psr_cat_op' to 'psr_alloc' in xen.if. > > Even if this was just a simple oversight and an easy rename, I think > strictly speaking it invalidates Daniel's ack. > > Jan Sorry for that. Hi, Daniel, Could you please review the change again? Thank you! BRs, Sun Yi ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v9.1 02/16] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
This patch renames PSR sysctl/domctl interfaces and related xsm policy to
make them be general for all resource allocation features but not only
for CAT. Then, we can resuse the interfaces for all allocation features.
Basically, it changes 'psr_cat_op' to 'psr_alloc', and remove 'CAT_' from some
macros. E.g.:
1. psr_cat_op -> psr_alloc
2. XEN_DOMCTL_psr_cat_op -> XEN_DOMCTL_psr_alloc
3. XEN_SYSCTL_psr_cat_op -> XEN_SYSCTL_psr_alloc
4. XEN_DOMCTL_PSR_CAT_SET_L3_CBM -> XEN_DOMCTL_PSR_SET_L3_CBM
5. XEN_SYSCTL_PSR_CAT_get_l3_info -> XEN_SYSCTL_PSR_get_l3_info
Signed-off-by: Yi Sun
Reviewed-by: Wei Liu
Reviewed-by: Roger Pau Monné
Acked-by: Jan Beulich
Acked-by: Daniel De Graaf
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Ian Jackson
CC: Daniel De Graaf
CC: Roger Pau Monné
CC: Chao Peng
v9:
- rename 'psr_cat_op' to 'psr_alloc' in xen.if.
v7:
- add single trailing underscore for internal variabled in macro.
(suggested by Jan Beulich)
- add parentheses for input parameters of marcro.
(suggested by Jan Beulich)
- adjust the postion of macro.
(suggested by Jan Beulich)
v6:
- move macro definition into the function and undefine it after use.
(suggested by Roger Pau Monné)
- do not bump sysctl version because it has been bumped for 4.10.
(suggested by Roger Pau Monné)
v5:
- remove domctl version number upgrade.
(suggested by Jan Beulich)
- restore 'XEN_SYSCTL_PSR_CAT_L3_CDP'.
(suggested by Jan Beulich)
- define a local macro to complete psr get value flow.
(suggested by Roger Pau Monné)
- remove 'Reviewed-by' and 'Acked-by'.
(suggested by Wei Liu)
v4:
- remove 'ALLOC_' from names.
(suggested by Roger Pau Monné)
- fix comments.
(suggested by Roger Pau Monné)
v3:
- remove 'op/OP' from names and modify some names from 'PSR_CAT' to
'PSR_ALLOC'.
(suggested by Roger Pau Monné)
v1:
- add description about what to be changed in commit message.
(suggested by Wei Liu)
- bump sysctl/domctl version numbers.
(suggested by Wei Liu)
---
tools/flask/policy/modules/dom0.te | 4 +--
tools/flask/policy/modules/xen.if | 2 +-
tools/libxc/xc_psr.c| 50 +-
xen/arch/x86/domctl.c | 71 ++---
xen/arch/x86/sysctl.c | 28 +++
xen/include/public/domctl.h | 24 ++---
xen/include/public/sysctl.h | 12 +++
xen/xsm/flask/hooks.c | 8 ++---
xen/xsm/flask/policy/access_vectors | 8 ++---
9 files changed, 103 insertions(+), 104 deletions(-)
diff --git a/tools/flask/policy/modules/dom0.te
b/tools/flask/policy/modules/dom0.te
index 1643b40..07de3d5 100644
--- a/tools/flask/policy/modules/dom0.te
+++ b/tools/flask/policy/modules/dom0.te
@@ -14,7 +14,7 @@ allow dom0_t xen_t:xen {
tmem_control getscheduler setscheduler
};
allow dom0_t xen_t:xen2 {
- resource_op psr_cmt_op psr_cat_op pmu_ctrl get_symbol
+ resource_op psr_cmt_op psr_alloc pmu_ctrl get_symbol
get_cpu_levelling_caps get_cpu_featureset livepatch_op
gcov_op set_parameter
};
@@ -39,7 +39,7 @@ allow dom0_t dom0_t:domain {
};
allow dom0_t dom0_t:domain2 {
set_cpuid gettsc settsc setscheduler set_max_evtchn set_vnumainfo
- get_vnumainfo psr_cmt_op psr_cat_op set_gnttab_limits
+ get_vnumainfo psr_cmt_op psr_alloc set_gnttab_limits
};
allow dom0_t dom0_t:resource { add remove };
diff --git a/tools/flask/policy/modules/xen.if
b/tools/flask/policy/modules/xen.if
index 5543749..cb48a6c 100644
--- a/tools/flask/policy/modules/xen.if
+++ b/tools/flask/policy/modules/xen.if
@@ -52,7 +52,7 @@ define(`create_domain_common', `
settime setdomainhandle getvcpucontext set_misc_info };
allow $1 $2:domain2 { set_cpuid settsc setscheduler setclaim
set_max_evtchn set_vnumainfo get_vnumainfo cacheflush
- psr_cmt_op psr_cat_op soft_reset set_gnttab_limits };
+ psr_cmt_op psr_alloc soft_reset set_gnttab_limits };
allow $1 $2:security check_context;
allow $1 $2:shadow enable;
allow $1 $2:mmu { map_read map_write adjust memorymap physmap pinpage
mmuext_op updatemp };
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index edec4d1..78deba0 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -258,27 +258,27 @@ int xc_psr_cat_set_domain_data(xc_interface *xch,
uint32_t domid,
switch ( type )
{
case XC_PSR_CAT_L3_CBM:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM;
+cmd = XEN_DOMCTL_PSR_SET_L3_CBM;
break;
case XC_PSR_CAT_L3_CBM_CODE:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_CODE;
+cmd =
[Xen-devel] Is that possible to merge MBA into Xen 4.10?
Hi, all, As you may know, MBA patch set has got enough Reviewed-by/Acked-by in last week. It is ready to be merged. This is a feature for Skylake, Intel has launched Skylake and KVM already supported MBA, so including it in Xen 4.10 will quickly fill this gap. MBA missed the 4.10 feature freeze date for only a few days due to lack of timely review for earlier versions which slowed down the patch iteration notably. It seems maintainers are very busy recently so that the review progress for 4.10 is slower than before. So I am wondering if it is possible to merge it into 4.10? This patch set mainly touches codes related to PSR in tools/domctl/sysctl/hypervisor. It does not touch other features. So, the risk is low to merge it. Thank you! BRs, Sun Yi ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v9 10/16] tools: implement the new libxc get hw info interface
This patch implements a new libxc get hw info interface and corresponding
data structures. It also changes libxl_psr.c to call this new interface.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Wei Liu
---
CC: Wei Liu
CC: Ian Jackson
CC: Roger Pau Monné
CC: Chao Peng
v6:
- remove unnecessary spaces in brackets.
(suggested by Wei Liu)
- use assert to check input lvl.
(suggested by Roger Pau Monné)
v5:
- directly define 'xc_psr_hw_info' as union type.
(suggested by Roger Pau Monné)
- converge L2 and L3 cases in 'xc_psr_get_hw_info'.
(suggested by Roger Pau Monné)
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- remove 'XC_PSR_FEAT_UNKNOWN' which is not necessary.
(suggested by Roger Pau Monné)
- remove 'FEAT_' from enum item names.
(suggested by Roger Pau Monné)
- remove 'xc_' from struct name.
(suggested by Roger Pau Monné)
- adjust codes to reduce indentation.
(suggested by Roger Pau Monné)
- assert for not happened case.
(suggested by Roger Pau Monné)
- add LOGE to show errno.
(suggested by Roger Pau Monné)
v3:
- rename 'psr_cat/PSR_CAT' to 'psr_alloc/PSR_ALLOC' and remove 'op/OP'
from name.
(suggested by Roger Pau Monné)
- remove 'info' from 'xc_cat_info' and 'xc_mba_info'.
(suggested by Roger Pau Monné)
- set errno in 'xc_psr_get_hw_info'.
(suggested by Roger Pau Monné)
- remove 'inline'.
(suggested by Roger Pau Monné)
- remove 'psr' from 'libxl__psr_feat_type_to_libxc_psr_feat_type' to make
function name shorter.
(suggested by Roger Pau Monné)
- check 'xc_type' in 'libxl_psr_cat_get_info'.
(suggested by Roger Pau Monné)
v2:
- split this patch out from a big patch in v1.
(suggested by Wei Liu)
- change 'CAT_INFO' and 'MBA_INFO' to 'CAT' and 'MBA'.
(suggested by Chao Peng)
---
tools/libxc/include/xenctrl.h | 27 ++---
tools/libxc/xc_psr.c | 55 +++
tools/libxl/libxl_psr.c | 38 --
3 files changed, 95 insertions(+), 25 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 666db0b..9fc3348 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2494,6 +2494,28 @@ enum xc_psr_cat_type {
};
typedef enum xc_psr_cat_type xc_psr_cat_type;
+enum xc_psr_feat_type {
+XC_PSR_CAT_L3,
+XC_PSR_CAT_L2,
+XC_PSR_MBA,
+};
+typedef enum xc_psr_feat_type xc_psr_feat_type;
+
+union xc_psr_hw_info {
+struct {
+uint32_t cos_max;
+uint32_t cbm_len;
+bool cdp_enabled;
+} cat;
+
+struct {
+uint32_t cos_max;
+uint32_t thrtl_max;
+bool linear;
+} mba;
+};
+typedef union xc_psr_hw_info xc_psr_hw_info;
+
int xc_psr_cmt_attach(xc_interface *xch, uint32_t domid);
int xc_psr_cmt_detach(xc_interface *xch, uint32_t domid);
int xc_psr_cmt_get_domain_rmid(xc_interface *xch, uint32_t domid,
@@ -2515,9 +2537,8 @@ int xc_psr_cat_set_domain_data(xc_interface *xch,
uint32_t domid,
int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
xc_psr_cat_type type, uint32_t target,
uint64_t *data);
-int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
-uint32_t *cos_max, uint32_t *cbm_len,
-bool *cdp_enabled);
+int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
+ xc_psr_feat_type type, xc_psr_hw_info *hw_info);
int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 78deba0..2335842 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -323,37 +323,52 @@ int xc_psr_cat_get_domain_data(xc_interface *xch,
uint32_t domid,
return rc;
}
-int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
-uint32_t *cos_max, uint32_t *cbm_len, bool
*cdp_enabled)
+int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
+ xc_psr_feat_type type, xc_psr_hw_info *hw_info)
{
int rc = -1;
DECLARE_SYSCTL;
+if ( !hw_info )
+{
+errno = EINVAL;
+return rc;
+}
+
sysctl.cmd = XEN_SYSCTL_psr_alloc;
sysctl.u.psr_alloc.target = socket;
-switch ( lvl )
+switch ( type )
{
-case 2:
-sysctl.u.psr_alloc.cmd = XEN_SYSCTL_PSR_get_l2_info;
+case XC_PSR_CAT_L2:
+case XC_PSR_CAT_L3:
+
[Xen-devel] [PATCH v9 07/16] x86: implement get value interface for MBA
This patch implements get value domctl interface for MBA.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Jan Beulich
---
CC: Andrew Cooper
CC: Jan Beulich
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v5:
- use newly defined macro to get MBA thrtl.
(suggested by Roger Pau Monné)
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
v3:
- change 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
---
xen/arch/x86/domctl.c | 4
xen/include/public/domctl.h | 1 +
2 files changed, 5 insertions(+)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index ddd3b6d..f7d299d 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1491,6 +1491,10 @@ long arch_do_domctl(
ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L2_CBM, copyback);
break;
+case XEN_DOMCTL_PSR_GET_MBA_THRTL:
+ret = domctl_psr_get_val(d, domctl, PSR_TYPE_MBA_THRTL, copyback);
+break;
+
#undef domctl_psr_get_val
default:
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index eaff9b3..44cf0b5 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1074,6 +1074,7 @@ struct xen_domctl_psr_alloc {
#define XEN_DOMCTL_PSR_GET_L3_DATA5
#define XEN_DOMCTL_PSR_SET_L2_CBM 6
#define XEN_DOMCTL_PSR_GET_L2_CBM 7
+#define XEN_DOMCTL_PSR_GET_MBA_THRTL 9
uint32_t cmd; /* IN: XEN_DOMCTL_PSR_* */
uint32_t target;/* IN */
uint64_t data; /* IN/OUT */
--
1.9.1
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v9 06/16] x86: implement get hw info flow for MBA
This patch implements get HW info flow for MBA including its callback
function and sysctl interface.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Jan Beulich
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v7:
- change 'PSR_INFO_IDX_MBA_FLAG' to 'PSR_INFO_IDX_MBA_FLAGS'.
(suggested by Jan Beulich)
v5:
- use ASSERT in 'mba_get_feat_info'.
(suggested by Roger Pau Monné)
- correct initialization format of 'data[PSR_INFO_ARRAY_SIZE]'.
(suggested by Roger Pau Monné and Jan Beulich)
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- initialize 'data[PSR_INFO_ARRAY_SIZE]' to 0 to prevent to leak stack data.
(suggested by Roger Pau Monné)
v3:
- replace 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
v2:
- use 'XEN_SYSCTL_PSR_MBA_LINEAR' to set MBA feature HW info.
(suggested by Chao Peng)
v1:
- sort 'PSR_INFO_IDX_' macros as feature.
(suggested by Chao Peng)
- rename 'PSR_INFO_IDX_MBA_LINEAR' to 'PSR_INFO_IDX_MBA_FLAG'.
- rename 'linear' in 'struct mba_info' to 'flags' for future extension.
---
xen/arch/x86/psr.c | 14 +-
xen/arch/x86/sysctl.c | 21 -
xen/include/asm-x86/psr.h | 2 ++
xen/include/public/sysctl.h | 8
4 files changed, 43 insertions(+), 2 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 872bade..a4901d8 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -263,6 +263,10 @@ static enum psr_feat_type psr_type_to_feat_type(enum
psr_type type)
feat_type = FEAT_TYPE_L2_CAT;
break;
+case PSR_TYPE_MBA_THRTL:
+feat_type = FEAT_TYPE_MBA;
+break;
+
default:
ASSERT_UNREACHABLE();
}
@@ -481,7 +485,15 @@ static const struct feat_props l2_cat_props = {
static bool mba_get_feat_info(const struct feat_node *feat,
uint32_t data[], unsigned int array_len)
{
-return false;
+ASSERT(array_len == PSR_INFO_ARRAY_SIZE);
+
+data[PSR_INFO_IDX_COS_MAX] = feat->cos_max;
+data[PSR_INFO_IDX_MBA_THRTL_MAX] = feat->mba.thrtl_max;
+
+if ( feat->mba.linear )
+data[PSR_INFO_IDX_MBA_FLAGS] |= XEN_SYSCTL_PSR_MBA_LINEAR;
+
+return true;
}
static void mba_write_msr(unsigned int cos, uint32_t val,
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index 6d48cac..ffad585 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -174,7 +174,7 @@ long arch_do_sysctl(
case XEN_SYSCTL_psr_alloc:
switch ( sysctl->u.psr_alloc.cmd )
{
-uint32_t data[PSR_INFO_ARRAY_SIZE];
+uint32_t data[PSR_INFO_ARRAY_SIZE] = { };
case XEN_SYSCTL_PSR_get_l3_info:
{
@@ -214,6 +214,25 @@ long arch_do_sysctl(
break;
}
+case XEN_SYSCTL_PSR_get_mba_info:
+{
+ret = psr_get_info(sysctl->u.psr_alloc.target,
+ PSR_TYPE_MBA_THRTL, data, ARRAY_SIZE(data));
+if ( ret )
+break;
+
+sysctl->u.psr_alloc.u.mba_info.cos_max =
+ data[PSR_INFO_IDX_COS_MAX];
+sysctl->u.psr_alloc.u.mba_info.thrtl_max =
+ data[PSR_INFO_IDX_MBA_THRTL_MAX];
+sysctl->u.psr_alloc.u.mba_info.flags =
+ data[PSR_INFO_IDX_MBA_FLAGS];
+
+if ( __copy_field_to_guest(u_sysctl, sysctl, u.psr_alloc) )
+ret = -EFAULT;
+break;
+}
+
default:
ret = -EOPNOTSUPP;
break;
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index 3cf544a..c2257da 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -39,6 +39,8 @@
#define PSR_INFO_IDX_COS_MAX0
#define PSR_INFO_IDX_CAT_CBM_LEN1
#define PSR_INFO_IDX_CAT_FLAGS 2
+#define PSR_INFO_IDX_MBA_THRTL_MAX 1
+#define PSR_INFO_IDX_MBA_FLAGS 2
#define PSR_INFO_ARRAY_SIZE 3
struct psr_cmt_l3 {
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index a50e345..f7f26c3 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -698,6 +698,7 @@ struct xen_sysctl_pcitopoinfo {
#define XEN_SYSCTL_PSR_get_l3_info 0
#define XEN_SYSCTL_PSR_get_l2_info 1
+#define XEN_SYSCTL_PSR_get_mba_info 2
struct xen_sysctl_psr_alloc {
uint32_t cmd; /* IN: XEN_SYSCTL_PSR_* */
uint32_t target;/* IN */
@@ -708,6 +709,13 @@ struct xen_sysctl_psr_alloc {
#define XEN_SYSCTL_PSR_CAT_L3_CDP (1u << 0)
[Xen-devel] [PATCH v9 15/16] tools: implement new generic set value interface and MBA set value command
This patch implements new generic set value interfaces in libxc and libxl.
These interfaces are suitable for all allocation features. It also adds a
new MBA set value command in xl.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Wei Liu
---
CC: Wei Liu
CC: Ian Jackson
CC: Roger Pau Monné
CC: Chao Peng
v5:
- move xc_type definition and value get out of the loop.
(suggested by Roger Pau Monné)
v4:
- remove 'ALLOC_' from macro name.
(suggested by Roger Pau Monné)
- adjust place of argc check and return EXIT_FAILURE.
(suggested by Roger Pau Monné)
- fix indentation issue.
(suggested by Roger Pau Monné)
- move same type local variables declaration to a single line.
(suggested by Roger Pau Monné)
v3:
- add 'const' for 'opts[]' in 'main_psr_mba_set'.
(suggested by Roger Pau Monné)
- replace 'libxl_psr_cbm_type' to 'libxl_psr_type' for newly defined
interfaces.
(suggested by Roger Pau Monné)
---
tools/libxc/include/xenctrl.h | 6 ++---
tools/libxc/xc_psr.c | 9 ---
tools/libxl/libxl_psr.c | 52
tools/xl/xl.h | 1 +
tools/xl/xl_cmdtable.c| 6 +
tools/xl/xl_psr.c | 55 +++
6 files changed, 96 insertions(+), 33 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index fc0d4d2..90bee20 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2532,9 +2532,9 @@ int xc_psr_cmt_get_data(xc_interface *xch, uint32_t rmid,
uint32_t cpu,
uint64_t *tsc);
int xc_psr_cmt_enabled(xc_interface *xch);
-int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_type type, uint32_t target,
- uint64_t data);
+int xc_psr_set_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_type type, uint32_t target,
+ uint64_t data);
int xc_psr_get_domain_data(xc_interface *xch, uint32_t domid,
xc_psr_type type, uint32_t target,
uint64_t *data);
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 37a6feb..1a0ab63 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -248,9 +248,9 @@ int xc_psr_cmt_enabled(xc_interface *xch)
return 0;
}
-int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_type type, uint32_t target,
- uint64_t data)
+int xc_psr_set_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_type type, uint32_t target,
+ uint64_t data)
{
DECLARE_DOMCTL;
uint32_t cmd;
@@ -269,6 +269,9 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t
domid,
case XC_PSR_CAT_L2_CBM:
cmd = XEN_DOMCTL_PSR_SET_L2_CBM;
break;
+case XC_PSR_MBA_THRTL:
+cmd = XEN_DOMCTL_PSR_SET_MBA_THRTL;
+break;
default:
errno = EINVAL;
return -1;
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index 7c560bc..9ced7d1 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -328,32 +328,7 @@ int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
libxl_psr_cbm_type type, libxl_bitmap *target_map,
uint64_t cbm)
{
-GC_INIT(ctx);
-int rc;
-int socketid, nr_sockets;
-
-rc = libxl__count_physical_sockets(gc, &nr_sockets);
-if (rc) {
-LOGED(ERROR, domid, "failed to get system socket count");
-goto out;
-}
-
-libxl_for_each_set_bit(socketid, *target_map) {
-xc_psr_type xc_type = libxl__psr_type_to_libxc_psr_type(type);
-
-if (socketid >= nr_sockets)
-break;
-
-if (xc_psr_cat_set_domain_data(ctx->xch, domid, xc_type,
- socketid, cbm)) {
-libxl__psr_alloc_log_err_msg(gc, errno, type);
-rc = ERROR_FAIL;
-}
-}
-
-out:
-GC_FREE;
-return rc;
+return libxl_psr_set_val(ctx, domid, type, target_map, cbm);
}
int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
@@ -453,7 +428,30 @@ int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
libxl_psr_type type, libxl_bitmap *target_map,
uint64_t val)
{
-return ERROR_FAIL;
+GC_INIT(ctx);
+int rc, socketid, nr_sockets;
+xc_psr_type xc_type = libxl__psr_type_to_libxc_psr_type(type);
+
+rc = libxl__count_physical_sockets(gc, &nr_sockets);
+if (rc) {
+LOG(ERROR, "failed to get system socket count");
+goto out;
+}
+
+libxl_for_each_set_bit(s
[Xen-devel] [PATCH v9 13/16] tools: rename 'xc_psr_cat_type' to 'xc_psr_type'
This patch renames 'xc_psr_cat_type' to 'xc_psr_type' so that
the structure name is common for all allocation features.
Signed-off-by: Yi Sun
Acked-by: Wei Liu
Reviewed-by: Chao Peng
Reviewed-by: Roger Pau Monné
---
CC: Ian Jackson
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v5:
- remove a duplicated ';'.
(suggested by Roger Pau Monné)
v4:
- move assignment of xc_type to its declaration place.
(suggested by Roger Pau Monné)
v3:
- change 'xc_psr_val_type' to 'xc_psr_type'.
(suggested by Roger Pau Monné)
---
tools/libxc/include/xenctrl.h | 8
tools/libxc/xc_psr.c | 4 ++--
tools/libxl/libxl_psr.c | 11 +--
3 files changed, 11 insertions(+), 12 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 9fc3348..8451f6d 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2486,13 +2486,13 @@ enum xc_psr_cmt_type {
};
typedef enum xc_psr_cmt_type xc_psr_cmt_type;
-enum xc_psr_cat_type {
+enum xc_psr_type {
XC_PSR_CAT_L3_CBM = 1,
XC_PSR_CAT_L3_CBM_CODE = 2,
XC_PSR_CAT_L3_CBM_DATA = 3,
XC_PSR_CAT_L2_CBM = 4,
};
-typedef enum xc_psr_cat_type xc_psr_cat_type;
+typedef enum xc_psr_type xc_psr_type;
enum xc_psr_feat_type {
XC_PSR_CAT_L3,
@@ -2532,10 +2532,10 @@ int xc_psr_cmt_get_data(xc_interface *xch, uint32_t
rmid, uint32_t cpu,
int xc_psr_cmt_enabled(xc_interface *xch);
int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_type type, uint32_t target,
uint64_t data);
int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_type type, uint32_t target,
uint64_t *data);
int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
xc_psr_feat_type type, xc_psr_hw_info *hw_info);
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 2335842..aa07fe0 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -249,7 +249,7 @@ int xc_psr_cmt_enabled(xc_interface *xch)
return 0;
}
int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_type type, uint32_t target,
uint64_t data)
{
DECLARE_DOMCTL;
@@ -284,7 +284,7 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t
domid,
}
int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_type type, uint32_t target,
uint64_t *data)
{
int rc;
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index b053abd..c54cb6f 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -303,11 +303,11 @@ out:
return rc;
}
-static inline xc_psr_cat_type libxl__psr_cbm_type_to_libxc_psr_cat_type(
+static inline xc_psr_type libxl__psr_cbm_type_to_libxc_psr_type(
libxl_psr_cbm_type type)
{
-BUILD_BUG_ON(sizeof(libxl_psr_cbm_type) != sizeof(xc_psr_cat_type));
-return (xc_psr_cat_type)type;
+BUILD_BUG_ON(sizeof(libxl_psr_cbm_type) != sizeof(xc_psr_type));
+return (xc_psr_type)type;
}
int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
@@ -325,12 +325,11 @@ int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
}
libxl_for_each_set_bit(socketid, *target_map) {
-xc_psr_cat_type xc_type;
+xc_psr_type xc_type = libxl__psr_cbm_type_to_libxc_psr_type(type);
if (socketid >= nr_sockets)
break;
-xc_type = libxl__psr_cbm_type_to_libxc_psr_cat_type(type);
if (xc_psr_cat_set_domain_data(ctx->xch, domid, xc_type,
socketid, cbm)) {
libxl__psr_cat_log_err_msg(gc, errno);
@@ -349,7 +348,7 @@ int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
{
GC_INIT(ctx);
int rc = 0;
-xc_psr_cat_type xc_type = libxl__psr_cbm_type_to_libxc_psr_cat_type(type);
+xc_psr_type xc_type = libxl__psr_cbm_type_to_libxc_psr_type(type);
if (xc_psr_cat_get_domain_data(ctx->xch, domid, xc_type,
target, cbm_r)) {
--
1.9.1
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v9 01/16] docs: create Memory Bandwidth Allocation (MBA) feature document
This patch creates MBA feature document in doc/features/. It describes key points to implement MBA which is described in details in Intel SDM "Introduction to Memory Bandwidth Allocation". Signed-off-by: Yi Sun Reviewed-by: Roger Pau Monné --- CC: Jan Beulich CC: Andrew Cooper CC: Wei Liu CC: Ian Jackson CC: Daniel De Graaf CC: Roger Pau Monné CC: Konrad Rzeszutek Wilk CC: Chao Peng CC: Julien Grall v6: - fix some words. (suggested by Roger Pau Monné) v5: - correct some words. (suggested by Roger Pau Monné) - change 'xl psr-mba-set 1 0xa' to 'xl psr-mba-set 1 10'. (suggested by Roger Pau Monné) v4: - add 'domain-name' as parameter of 'psr-mba-show/psr-mba-set'. (suggested by Roger Pau Monné) - fix some wordings. (suggested by Roger Pau Monné) - explain how user can know the MBA_MAX. (suggested by Roger Pau Monné) - move the description of 'Linear mode/Non-linear mode' into section of 'psr-mba-show'. (suggested by Roger Pau Monné) - change 'per-thread' to 'per-hyper-thread' to make it clearer. (suggested by Roger Pau Monné) - upgrade revision number. v3: - remove 'closed-loop' related description. (suggested by Roger Pau Monné) - explain 'linear' and 'non-linear' before mentioning them. (suggested by Roger Pau Monné) - adjust desription of 'psr-mba-set'. (suggested by Roger Pau Monné) - explain 'MBA_MAX'. (suggested by Roger Pau Monné) - remove 'n<64'. (suggested by Roger Pau Monné) - fix some wordings. (suggested by Roger Pau Monné) - add context in 'Testing' part to make things more clear. (suggested by Roger Pau Monné) v2: - declare 'HW' in Terminology. (suggested by Chao Peng) - replace 'COS ID of VCPU' to 'COS ID of domain'. (suggested by Chao Peng) - replace 'COS register' to 'Thrtl MSR'. (suggested by Chao Peng) - add description for 'psr-mba-show' to state that the decimal value is shown for linear mode but hexadecimal value is shown for non-linear mode. (suggested by Chao Peng) - remove content in 'Areas for improvement'. (suggested by Chao Peng) - use '<>' to specify mandatory argument to a command. (suggested by Wei Liu) v1: - remove a special character to avoid the error when building pandoc. --- docs/features/intel_psr_mba.pandoc | 297 + 1 file changed, 297 insertions(+) create mode 100644 docs/features/intel_psr_mba.pandoc diff --git a/docs/features/intel_psr_mba.pandoc b/docs/features/intel_psr_mba.pandoc new file mode 100644 index 000..86df661 --- /dev/null +++ b/docs/features/intel_psr_mba.pandoc @@ -0,0 +1,297 @@ +% Intel Memory Bandwidth Allocation (MBA) Feature +% Revision 1.8 + +\clearpage + +# Basics + + + Status: **Tech Preview** + +Architecture(s): Intel x86 + + Component(s): Hypervisor, toolstack + + Hardware: MBA is supported on Skylake Server and beyond + + +# Terminology + +* CAT Cache Allocation Technology +* CBM Capacity BitMasks +* CDP Code and Data Prioritization +* COS/CLOSClass of Service +* HW Hardware +* MBA Memory Bandwidth Allocation +* MSRsMachine Specific Registers +* PSR Intel Platform Shared Resource +* THRTL Throttle value or delay value + +# Overview + +The Memory Bandwidth Allocation (MBA) feature provides indirect and approximate +control over memory bandwidth available per-core. This feature provides OS/ +hypervisor the ability to slow misbehaving apps/domains by using a credit-based +throttling mechanism. + +# User details + +* Feature Enabling: + + Add "psr=mba" to boot line parameter to enable MBA feature. + +* xl interfaces: + + 1. `psr-mba-show [domain-id|domain-name]`: + + Show memory bandwidth throttling for domain. Under different modes, it + shows different type of data. + + There are two modes: + Linear mode: the input precision is defined as 100-(MBA_MAX). For instance, + if the MBA_MAX value is 90, the input precision is 10%. Values not an even + multiple of the precision (e.g., 12%) will be rounded down (e.g., to 10% + delay applied) by HW automatically. The response of throttling value is + linear. + + Non-linear mode: input delay values are powers-of-two from zero to the + MBA_MAX value from CPUID. In this case any values not a power of two will + be rounded down the next nearest power of two by HW automatically. The + response of throttli
[Xen-devel] [PATCH v9 03/16] x86: rename 'cbm_type' to 'psr_type' to make it general
This patch renames 'cbm_type' to 'psr_type' to generalize it.
Then, we can reuse this for all psr allocation features.
Signed-off-by: Yi Sun
Reviewed-by: Wei Liu
Reviewed-by: Roger Pau Monné
Acked-by: Jan Beulich
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v5:
- correct character of reviewer's name.
(suggested by Jan Beulich)
v4:
- fix words in commit message.
(suggested by Roger Pau Monné)
v3:
- replace 'psr_val_type' to 'psr_type' and remove '_VAL' from the enum
items.
(suggested by Roger Pau Monné)
v2:
- replace 'PSR_VAL_TYPE_{L3, L2}' to 'PSR_VAL_TYPE_{L3, L2}_CBM'.
(suggested by Chao Peng)
---
xen/arch/x86/domctl.c | 16 ++--
xen/arch/x86/psr.c| 62 +--
xen/arch/x86/sysctl.c | 4 +--
xen/include/asm-x86/psr.h | 18 +++---
4 files changed, 52 insertions(+), 48 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 64bb79c..ddd3b6d 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1444,25 +1444,25 @@ long arch_do_domctl(
case XEN_DOMCTL_PSR_SET_L3_CBM:
ret = psr_set_val(d, domctl->u.psr_alloc.target,
domctl->u.psr_alloc.data,
- PSR_CBM_TYPE_L3);
+ PSR_TYPE_L3_CBM);
break;
case XEN_DOMCTL_PSR_SET_L3_CODE:
ret = psr_set_val(d, domctl->u.psr_alloc.target,
domctl->u.psr_alloc.data,
- PSR_CBM_TYPE_L3_CODE);
+ PSR_TYPE_L3_CODE);
break;
case XEN_DOMCTL_PSR_SET_L3_DATA:
ret = psr_set_val(d, domctl->u.psr_alloc.target,
domctl->u.psr_alloc.data,
- PSR_CBM_TYPE_L3_DATA);
+ PSR_TYPE_L3_DATA);
break;
case XEN_DOMCTL_PSR_SET_L2_CBM:
ret = psr_set_val(d, domctl->u.psr_alloc.target,
domctl->u.psr_alloc.data,
- PSR_CBM_TYPE_L2);
+ PSR_TYPE_L2_CBM);
break;
#define domctl_psr_get_val(d, domctl, type, copyback) ({\
@@ -1476,19 +1476,19 @@ long arch_do_domctl(
})
case XEN_DOMCTL_PSR_GET_L3_CBM:
-ret = domctl_psr_get_val(d, domctl, PSR_CBM_TYPE_L3, copyback);
+ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L3_CBM, copyback);
break;
case XEN_DOMCTL_PSR_GET_L3_CODE:
-ret = domctl_psr_get_val(d, domctl, PSR_CBM_TYPE_L3_CODE,
copyback);
+ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L3_CODE, copyback);
break;
case XEN_DOMCTL_PSR_GET_L3_DATA:
-ret = domctl_psr_get_val(d, domctl, PSR_CBM_TYPE_L3_DATA,
copyback);
+ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L3_DATA, copyback);
break;
case XEN_DOMCTL_PSR_GET_L2_CBM:
-ret = domctl_psr_get_val(d, domctl, PSR_CBM_TYPE_L2, copyback);
+ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L2_CBM, copyback);
break;
#undef domctl_psr_get_val
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 1abb9b4..7d8295e 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -100,24 +100,24 @@ static const struct feat_props {
unsigned int cos_num;
/*
- * An array to save all 'enum cbm_type' values of the feature. It is
+ * An array to save all 'enum psr_type' values of the feature. It is
* used with cos_num together to get/write a feature's COS registers
* values one by one.
*/
-enum cbm_type type[MAX_COS_NUM];
+enum psr_type type[MAX_COS_NUM];
/*
* alt_type is 'alternative type'. When this 'alt_type' is input, the
* feature does some special operations.
*/
-enum cbm_type alt_type;
+enum psr_type alt_type;
/* get_feat_info is used to return feature HW info through sysctl. */
bool (*get_feat_info)(const struct feat_node *feat,
uint32_t data[], unsigned int array_len);
/* write_msr is used to write out feature MSR register. */
-void (*write_msr)(unsigned int cos, uint32_t val, enum cbm_type type);
+void (*write_msr)(unsigned int cos, uint32_t val, enum psr_type type);
} *feat_props[FEAT_TYPE_NUM];
/*
@@ -215,13 +215,13 @@ static void free_socket_resources(unsigned int socket)
bitmap_zero(info->dom_set, DOMID_IDLE + 1);
}
-static enum psr_feat_type psr_cbm_type_to_feat_type(enum cbm_type type)
+static enum psr_feat_type psr_type_to_feat_type(enum psr_type type)
{
enum psr_feat_type feat_
[Xen-devel] [PATCH v9 14/16] tools: implement new generic get value interface and MBA get value command
This patch implements generic get value interfaces in libxc and libxl.
It also refactors the get value flow in xl to make it be suitable for all
allocation features. Based on that, a new MBA get value command is added in xl.
Signed-off-by: Yi Sun
Acked-by: Wei Liu
Reviewed-by: Roger Pau Monné
---
CC: Ian Jackson
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v6:
- fix one coding style issue.
(suggested by Roger Pau Monné)
v5:
- start a newline for "CDP" because it exceeds 80 characters.
(suggested by Roger Pau Monné)
- remove a duplicated ';'.
(suggested by Roger Pau Monné)
- remove a extra newline.
(suggested by Roger Pau Monné)
- correct words in log message.
(suggested by Roger Pau Monné)
v4:
- use designated initializers for 'feat_name[]'.
(suggested by Roger Pau Monné)
- use LOG in 'libxl__psr_alloc_log_err_msg'.
(suggested by Roger Pau Monné)
v3:
- replace 'libxl_psr_cbm_type' to 'libxl_psr_type' in newly defined
interfaces.
(suggested by Roger Pau Monné)
v2:
- change 'CAT_INFO'/'MBA_INFO' to 'CAT'/'MBA'. The related structure names
are changed too.
(suggested by Chao Peng)
---
tools/libxc/include/xenctrl.h | 7 +-
tools/libxc/xc_psr.c | 9 +-
tools/libxl/libxl_psr.c | 58 -
tools/xl/xl.h | 1 +
tools/xl/xl_cmdtable.c| 5 ++
tools/xl/xl_psr.c | 185 ++
6 files changed, 183 insertions(+), 82 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 8451f6d..fc0d4d2 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2491,6 +2491,7 @@ enum xc_psr_type {
XC_PSR_CAT_L3_CBM_CODE = 2,
XC_PSR_CAT_L3_CBM_DATA = 3,
XC_PSR_CAT_L2_CBM = 4,
+XC_PSR_MBA_THRTL = 5,
};
typedef enum xc_psr_type xc_psr_type;
@@ -2534,9 +2535,9 @@ int xc_psr_cmt_enabled(xc_interface *xch);
int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
xc_psr_type type, uint32_t target,
uint64_t data);
-int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_type type, uint32_t target,
- uint64_t *data);
+int xc_psr_get_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_type type, uint32_t target,
+ uint64_t *data);
int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
xc_psr_feat_type type, xc_psr_hw_info *hw_info);
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index aa07fe0..37a6feb 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -283,9 +283,9 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t
domid,
return do_domctl(xch, &domctl);
}
-int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_type type, uint32_t target,
- uint64_t *data)
+int xc_psr_get_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_type type, uint32_t target,
+ uint64_t *data)
{
int rc;
DECLARE_DOMCTL;
@@ -305,6 +305,9 @@ int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t
domid,
case XC_PSR_CAT_L2_CBM:
cmd = XEN_DOMCTL_PSR_GET_L2_CBM;
break;
+case XC_PSR_MBA_THRTL:
+cmd = XEN_DOMCTL_PSR_GET_MBA_THRTL;
+break;
default:
errno = EINVAL;
return -1;
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index c54cb6f..7c560bc 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -71,16 +71,30 @@ static void libxl__psr_cmt_log_err_msg(libxl__gc *gc, int
err)
LOGE(ERROR, "%s", msg);
}
-static void libxl__psr_cat_log_err_msg(libxl__gc *gc, int err)
+static void libxl__psr_alloc_log_err_msg(libxl__gc *gc,
+ int err,
+ libxl_psr_type type)
{
+/*
+ * Index is 'libxl_psr_type' so we set two 'CDP' to correspond to
+ * DATA and CODE.
+ */
+const char * const feat_name[] = {
+[LIBXL_PSR_CBM_TYPE_UNKNOWN] = "UNKNOWN",
+[LIBXL_PSR_CBM_TYPE_L3_CBM] = "L3 CAT",
+[LIBXL_PSR_CBM_TYPE_L3_CBM_CODE...LIBXL_PSR_CBM_TYPE_L3_CBM_DATA] =
+ "CDP",
+[LIBXL_PSR_CBM_TYPE_L2_CBM] = "L2 CAT",
+[LIBXL_PSR_CBM_TYPE_MBA_THRTL] = "MBA",
+};
char *msg;
switch (err) {
case ENODEV:
-msg = "CAT is not supported in this system";
+msg = "is not supported
[Xen-devel] [PATCH v9 05/16] x86: implement data structure and CPU init flow for MBA
This patch implements main data structures of MBA. Like CAT features, MBA HW info has cos_max which means the max thrtl register number, and thrtl_max which means the max throttle value (delay value). It also has a flag to represent if the throttle value is linear or non-linear. One thrtl register of MBA stores a throttle value for one or more domains. The throttle value means the delay applied to traffic between L2 cache and next cache level. This patch also implements init flow for MBA and register stub callback functions. Signed-off-by: Yi Sun Reviewed-by: Roger Pau Monné Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Andrew Cooper CC: Wei Liu CC: Roger Pau Monné CC: Chao Peng v8: - remove unnecessary line split. (suggested by Jan Beulich) - use MASK_EXTR(). (suggested by Jan Beulich) v7: - modify commit message. (suggested by Jan Beulich) - move the changes about check of 'cat_init_feature' in 'psr_cpu_init' into previous patch. (suggested by Jan Beulich) v6: - restore type of 'mba_write_msr' to 'void'. v5: - move out some CAT codes optimization to a new patch. (suggested by Jan Beulich) - modify commit message. (suggested by Jan Beulich) - change print type of 'linear' to be %d. (suggested by Jan Beulich) - change type of 'mba_write_msr' to uint32_t. - move printk in 'mba_init_feature' to reduce one return path. (suggested by Roger Pau Monné) - move the MBA format string in printk to a new line. (suggested by Roger Pau Monné) v4: - modify commit message. (suggested by Roger Pau Monné) - fix a comment. (suggested by Roger Pau Monné) - join two checks in a single if. (suggested by Roger Pau Monné) - remove redundant initialization of 'feat->cos_reg_val[0]'. (suggested by Roger Pau Monné) - change 'reg_b' to 'ebx'. (suggested by Jan Beulich) - change type of 'mba_init_feature' from 'int' to 'bool'. (suggested by Roger Pau Monné) - change type of 'cat_init_feature' from 'int' to 'bool'. v3: - replace 'psr_val_type' to 'psr_type'. Also, change 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'. (suggested by Roger Pau Monné) - replace 'MBA_LINEAR' to 'MBA_LINEAR_MASK' to make the name more clear. (suggested by Roger Pau Monné) - replase 'cat_info'/'mba_info' to 'cat'/'mba' to make the names shorter. (suggested by Roger Pau Monné) - change type of 'linear' to 'bool'. (suggested by Roger Pau Monné) - make format string of printf in one line. (suggested by Roger Pau Monné) v2: - modify commit message to replace 'cos register' to 'thrtl register' to make it accurate. (suggested by Chao Peng) - restore the place of the sentence to assign value to 'feat->cbm_len' because the MBA init flow is splitted out as a separate function in v1. (suggested by Chao Peng) - add comment to explain what the MBA thrtl defaul value '0' stands for. (suggested by Chao Peng) - check 'thrtl_max' under linear mode. It could not be euqal or larger than 100. (suggested by Chao Peng) v1: - rebase codes onto L2 CAT v15. - move comment to appropriate place. (suggested by Chao Peng) - implement 'mba_init_feature' and keep 'cat_init_feature'. (suggested by Chao Peng) - keep 'regs.b' into a local variable to avoid reading CPUID every time. (suggested by Chao Peng) --- xen/arch/x86/psr.c | 127 +++- xen/include/asm-x86/msr-index.h | 1 + xen/include/asm-x86/psr.h | 2 + 3 files changed, 114 insertions(+), 16 deletions(-) diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c index 58d203f..872bade 100644 --- a/xen/arch/x86/psr.c +++ b/xen/arch/x86/psr.c @@ -27,13 +27,16 @@ * - CMT Cache Monitoring Technology * - COS/CLOSClass of Service. Also mean COS registers. * - COS_MAX Max number of COS for the feature (minus 1) + * - MBA Memory Bandwidth Allocation * - MSRsMachine Specific Registers * - PSR Intel Platform Shared Resource + * - THRTL_MAX Max throttle value (delay value) of MBA */ #define PSR_CMT(1u << 0) #define PSR_CAT(1u << 1) #define PSR_CDP(1u << 2) +#define PSR_MBA(1u << 3) #define CAT_CBM_LEN_MASK 0x1f #define CAT_COS_MAX_MASK 0x @@ -60,10 +63,14 @@ */ #define MAX_COS_NUM 2 +#define MBA_LINEAR_MASK(1u << 2) +#define MBA_THRTL_MAX_MASK 0xfff + enum psr_feat_type
[Xen-devel] [PATCH v9 04/16] x86: a few optimizations to psr codes
This patch refines psr codes:
1. Change type of 'cat_init_feature' to 'bool' to remove the pointless
returning of error code.
2. Move printk in 'cat_init_feature' to reduce a return path.
3. Define a local variable 'feat_mask' in 'psr_cpu_init' to reduce calling of
'cpuid_count_leaf()'.
4. Change 'PSR_INFO_IDX_CAT_FLAG' to 'PSR_INFO_IDX_CAT_FLAGS'.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Jan Beulich
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v7:
- adjust the check to 'cat_init_feature' in 'psr_cpu_init'.
(suggested by Jan Beulich)
- change 'PSR_INFO_IDX_CAT_FLAG' to 'PSR_INFO_IDX_CAT_FLAGS'.
(suggested by Jan Beulich)
v6:
- restore 'write_msr()' type to 'void'.
(suggested by Jan Beulich and Roger Pau Monné)
- change 'ebx' in 'psr_cpu_init' to 'feat_mask'.
(suggested by Jan Beulich and Roger Pau Monné)
v5:
- create this patch to make codes clearer.
(suggested by Jan Beulich and Roger Pau Monné)
---
xen/arch/x86/psr.c| 45 ++---
xen/arch/x86/sysctl.c | 4 ++--
xen/include/asm-x86/psr.h | 2 +-
3 files changed, 25 insertions(+), 26 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 7d8295e..58d203f 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -273,10 +273,10 @@ static bool psr_check_cbm(unsigned int cbm_len, unsigned
long cbm)
}
/* CAT common functions implementation. */
-static int cat_init_feature(const struct cpuid_leaf *regs,
-struct feat_node *feat,
-struct psr_socket_info *info,
-enum psr_feat_type type)
+static bool cat_init_feature(const struct cpuid_leaf *regs,
+ struct feat_node *feat,
+ struct psr_socket_info *info,
+ enum psr_feat_type type)
{
const char *const cat_feat_name[FEAT_TYPE_NUM] = {
[FEAT_TYPE_L3_CAT] = "L3 CAT",
@@ -286,7 +286,7 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
/* No valid value so do not enable feature. */
if ( !regs->a || !regs->d )
-return -ENOENT;
+return false;
feat->cbm_len = (regs->a & CAT_CBM_LEN_MASK) + 1;
feat->cos_max = min(opt_cos_max, regs->d & CAT_COS_MAX_MASK);
@@ -296,7 +296,7 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
case FEAT_TYPE_L3_CAT:
case FEAT_TYPE_L2_CAT:
if ( feat->cos_max < 1 )
-return -ENOENT;
+return false;
/* We reserve cos=0 as default cbm (all bits within cbm_len are 1). */
feat->cos_reg_val[0] = cat_default_val(feat->cbm_len);
@@ -313,7 +313,7 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
uint64_t val;
if ( feat->cos_max < 3 )
-return -ENOENT;
+return false;
/* Cut half of cos_max when CDP is enabled. */
feat->cos_max = (feat->cos_max - 1) >> 1;
@@ -332,20 +332,18 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
}
default:
-return -ENOENT;
+return false;
}
/* Add this feature into array. */
info->features[type] = feat;
-if ( !opt_cpu_info )
-return 0;
-
-printk(XENLOG_INFO "%s: enabled on socket %u, cos_max:%u, cbm_len:%u\n",
- cat_feat_name[type], cpu_to_socket(smp_processor_id()),
- feat->cos_max, feat->cbm_len);
+if ( opt_cpu_info )
+printk(XENLOG_INFO "%s: enabled on socket %u, cos_max:%u,
cbm_len:%u\n",
+ cat_feat_name[type], cpu_to_socket(smp_processor_id()),
+ feat->cos_max, feat->cbm_len);
-return 0;
+return true;
}
static bool cat_get_feat_info(const struct feat_node *feat,
@@ -356,7 +354,7 @@ static bool cat_get_feat_info(const struct feat_node *feat,
data[PSR_INFO_IDX_COS_MAX] = feat->cos_max;
data[PSR_INFO_IDX_CAT_CBM_LEN] = feat->cbm_len;
-data[PSR_INFO_IDX_CAT_FLAG] = 0;
+data[PSR_INFO_IDX_CAT_FLAGS] = 0;
return true;
}
@@ -383,7 +381,7 @@ static bool l3_cdp_get_feat_info(const struct feat_node
*feat,
if ( !cat_get_feat_info(feat, data, array_len) )
return false;
-data[PSR_INFO_IDX_CAT_FLAG] |= XEN_SYSCTL_PSR_CAT_L3_CDP;
+data[PSR_INFO_IDX_CAT_FLAGS] |= XEN_SYSCTL_PSR_CAT_L3_CDP;
return true;
}
@@ -1413,6 +1411,7 @@ static void psr_cpu_init(void)
unsigned int socket, cpu = smp_processor_id();
struct feat_node *feat;
struct cpuid_leaf regs;
+uint32_t feat_mask;
if ( !psr_alloc_feat_enabled() || !boot_cpu_ha
[Xen-devel] [PATCH v9 16/16] docs: add MBA description in docs
This patch adds MBA description in related documents. Signed-off-by: Yi Sun Acked-by: Wei Liu Reviewed-by: Roger Pau Monné --- CC: Ian Jackson CC: Wei Liu CC: Roger Pau Monné CC: Chao Peng v5: - remove 'closed-loop' in 'xl-psr.markdown' (suggested by Roger Pau Monné) v4: - modify description of MBA in 'xl.pod.1.in' to be same as feature doc. (suggested by Roger Pau Monné) - fix words issue. (suggested by Roger Pau Monné) v2: - state the value type shown by 'psr-mba-show'. For linear mode, it shows decimal value. For non-linear mode, it shows hexadecimal value. (suggested by Chao Peng) --- docs/man/xl.pod.1.in | 33 + docs/misc/xl-psr.markdown | 62 +++ 2 files changed, 95 insertions(+) diff --git a/docs/man/xl.pod.1.in b/docs/man/xl.pod.1.in index 486a24f..7fd35c9 100644 --- a/docs/man/xl.pod.1.in +++ b/docs/man/xl.pod.1.in @@ -1850,6 +1850,39 @@ processed. =back +=head2 Memory Bandwidth Allocation + +Intel Skylake and later server platforms offer capabilities to configure and +make use of the Memory Bandwidth Allocation (MBA) mechanisms, which provides +OS/VMMs the ability to slow misbehaving apps/VMs by using a credit-based +throttling mechanism. In the Xen implementation, MBA is used to control memory +bandwidth on VM basis. To enforce bandwidth on a specific domain, just set +throttling value (THRTL) for the domain. + +=over 4 + +=item B [I] I I + +Set throttling value (THRTL) for a domain. For how to specify I +please refer to L<http://xenbits.xen.org/docs/unstable/misc/xl-psr.html>. + +B + +=over 4 + +=item B<-s SOCKET>, B<--socket=SOCKET> + +Specify the socket to process, otherwise all sockets are processed. + +=back + +=item B [I] + +Show MBA settings for a certain domain or all domains. For linear mode, it +shows the decimal value. For non-linear mode, it shows hexadecimal value. + +=back + =head1 IGNORED FOR COMPATIBILITY WITH XM xl is mostly command-line compatible with the old xm utility used with diff --git a/docs/misc/xl-psr.markdown b/docs/misc/xl-psr.markdown index 04dd957..3d196ed 100644 --- a/docs/misc/xl-psr.markdown +++ b/docs/misc/xl-psr.markdown @@ -186,6 +186,68 @@ Setting data CBM for a domain: Setting the same code and data CBM for a domain: `xl psr-cat-set ` +## Memory Bandwidth Allocation (MBA) + +Memory Bandwidth Allocation (MBA) is a new feature available on Intel +Skylake and later server platforms that allows an OS or Hypervisor/VMM to +slow misbehaving apps/VMs by using a credit-based throttling mechanism. To +enforce bandwidth on a specific domain, just set throttling value (THRTL) +into Class of Service (COS). MBA provides two THRTL mode. One is linear mode +and the other is non-linear mode. + +In the linear mode the input precision is defined as 100-(THRTL_MAX). Values +not an even multiple of the precision (e.g., 12%) will be rounded down (e.g., +to 10% delay by the hardware). + +If linear values are not supported then input delay values are powers-of-two +from zero to the THRTL_MAX value from CPUID. In this case any values not a power +of two will be rounded down the next nearest power of two. + +For example, assuming a system with 2 domains: + + * A THRTL of 0x0 for every domain means each domain can access the whole cache + without any delay. This is the default. + + * Linear mode: Giving one domain a THRTL of 0xC and the other domain's 0 means + that the first domain gets 10% delay to access the cache and the other one + without any delay. + + * Non-linear mode: Giving one domain a THRTL of 0xC and the other domain's 0 + means that the first domain gets 8% delay to access the cache and the other + one without any delay. + +For more detailed information please refer to Intel SDM chapter +"Introduction to Memory Bandwidth Allocation". + +In Xen's implementation, THRTL can be configured with libxl/xl interfaces but +COS is maintained in hypervisor only. The cache partition granularity is per +domain, each domain has COS=0 assigned by default, the corresponding THRTL is +0, which means all the cache resource can be accessed without delay. + +### xl interfaces + +System MBA information such as maximum COS and maximum THRTL can be obtained by: + +`xl psr-hwinfo --mba` + +The simplest way to change a domain's THRTL from its default is running: + +`xl psr-mba-set [OPTIONS] ` + +In a multi-socket system, the same thrtl will be set on each socket by default. +Per socket thrtl can be specified with the `--socket SOCKET` option. + +Setting the THRTL may not be successful if insufficient COS is available. In +such case unused COS(es) may be freed by setting THRTL of all related domains to +its default value(0). + +Per domain THRTL settings can be shown by: + +`xl psr-mba-show [OPTIONS] ` + +For linear mode, it shows the decimal v
[Xen-devel] [PATCH v9 02/16] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
This patch renames PSR sysctl/domctl interfaces and related xsm policy to
make them be general for all resource allocation features but not only
for CAT. Then, we can resuse the interfaces for all allocation features.
Basically, it changes 'psr_cat_op' to 'psr_alloc', and remove 'CAT_' from some
macros. E.g.:
1. psr_cat_op -> psr_alloc
2. XEN_DOMCTL_psr_cat_op -> XEN_DOMCTL_psr_alloc
3. XEN_SYSCTL_psr_cat_op -> XEN_SYSCTL_psr_alloc
4. XEN_DOMCTL_PSR_CAT_SET_L3_CBM -> XEN_DOMCTL_PSR_SET_L3_CBM
5. XEN_SYSCTL_PSR_CAT_get_l3_info -> XEN_SYSCTL_PSR_get_l3_info
Signed-off-by: Yi Sun
Reviewed-by: Wei Liu
Reviewed-by: Roger Pau Monné
Acked-by: Jan Beulich
Acked-by: Daniel De Graaf
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Ian Jackson
CC: Daniel De Graaf
CC: Roger Pau Monné
CC: Chao Peng
v7:
- add single trailing underscore for internal variabled in macro.
(suggested by Jan Beulich)
- add parentheses for input parameters of marcro.
(suggested by Jan Beulich)
- adjust the postion of macro.
(suggested by Jan Beulich)
v6:
- move macro definition into the function and undefine it after use.
(suggested by Roger Pau Monné)
- do not bump sysctl version because it has been bumped for 4.10.
(suggested by Roger Pau Monné)
v5:
- remove domctl version number upgrade.
(suggested by Jan Beulich)
- restore 'XEN_SYSCTL_PSR_CAT_L3_CDP'.
(suggested by Jan Beulich)
- define a local macro to complete psr get value flow.
(suggested by Roger Pau Monné)
- remove 'Reviewed-by' and 'Acked-by'.
(suggested by Wei Liu)
v4:
- remove 'ALLOC_' from names.
(suggested by Roger Pau Monné)
- fix comments.
(suggested by Roger Pau Monné)
v3:
- remove 'op/OP' from names and modify some names from 'PSR_CAT' to
'PSR_ALLOC'.
(suggested by Roger Pau Monné)
v1:
- add description about what to be changed in commit message.
(suggested by Wei Liu)
- bump sysctl/domctl version numbers.
(suggested by Wei Liu)
---
tools/flask/policy/modules/dom0.te | 4 +--
tools/libxc/xc_psr.c| 50 +-
xen/arch/x86/domctl.c | 71 ++---
xen/arch/x86/sysctl.c | 28 +++
xen/include/public/domctl.h | 24 ++---
xen/include/public/sysctl.h | 12 +++
xen/xsm/flask/hooks.c | 8 ++---
xen/xsm/flask/policy/access_vectors | 8 ++---
8 files changed, 102 insertions(+), 103 deletions(-)
diff --git a/tools/flask/policy/modules/dom0.te
b/tools/flask/policy/modules/dom0.te
index 1643b40..07de3d5 100644
--- a/tools/flask/policy/modules/dom0.te
+++ b/tools/flask/policy/modules/dom0.te
@@ -14,7 +14,7 @@ allow dom0_t xen_t:xen {
tmem_control getscheduler setscheduler
};
allow dom0_t xen_t:xen2 {
- resource_op psr_cmt_op psr_cat_op pmu_ctrl get_symbol
+ resource_op psr_cmt_op psr_alloc pmu_ctrl get_symbol
get_cpu_levelling_caps get_cpu_featureset livepatch_op
gcov_op set_parameter
};
@@ -39,7 +39,7 @@ allow dom0_t dom0_t:domain {
};
allow dom0_t dom0_t:domain2 {
set_cpuid gettsc settsc setscheduler set_max_evtchn set_vnumainfo
- get_vnumainfo psr_cmt_op psr_cat_op set_gnttab_limits
+ get_vnumainfo psr_cmt_op psr_alloc set_gnttab_limits
};
allow dom0_t dom0_t:resource { add remove };
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index edec4d1..78deba0 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -258,27 +258,27 @@ int xc_psr_cat_set_domain_data(xc_interface *xch,
uint32_t domid,
switch ( type )
{
case XC_PSR_CAT_L3_CBM:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM;
+cmd = XEN_DOMCTL_PSR_SET_L3_CBM;
break;
case XC_PSR_CAT_L3_CBM_CODE:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_CODE;
+cmd = XEN_DOMCTL_PSR_SET_L3_CODE;
break;
case XC_PSR_CAT_L3_CBM_DATA:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_DATA;
+cmd = XEN_DOMCTL_PSR_SET_L3_DATA;
break;
case XC_PSR_CAT_L2_CBM:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM;
+cmd = XEN_DOMCTL_PSR_SET_L2_CBM;
break;
default:
errno = EINVAL;
return -1;
}
-domctl.cmd = XEN_DOMCTL_psr_cat_op;
+domctl.cmd = XEN_DOMCTL_psr_alloc;
domctl.domain = domid;
-domctl.u.psr_cat_op.cmd = cmd;
-domctl.u.psr_cat_op.target = target;
-domctl.u.psr_cat_op.data = data;
+domctl.u.psr_alloc.cmd = cmd;
+domctl.u.psr_alloc.target = target;
+domctl.u.psr_alloc.data = data;
return do_domctl(xch, &domctl);
}
@@ -294,31 +294,31 @@ int xc_psr_cat_get_domain_data(xc_interface *xch,
uint32_t domid,
switch ( type )
{
case XC_PSR_CAT_L3_CBM:
-cmd = XEN_D
[Xen-devel] [PATCH v9 11/16] tools: implement the new libxl get hw info interface
This patch implements the new libxl get hw info interface,
'libxl_psr_get_hw_info', which is suitable to all psr allocation
features. It also implements corresponding list free function,
'libxl_psr_hw_info_list_free' and makes 'libxl_psr_cat_get_info' call
'libxl_psr_get_hw_info' to avoid redundant code in libxl_psr.c.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Wei Liu
---
CC: Wei Liu
CC: Ian Jackson
CC: Roger Pau Monné
CC: Chao Peng
v5:
- change 'if (rc < 0)' to 'if (rc)'.
(suggested by Roger Pau Monné)
v4:
- remove 'xc_' from struct name.
(suggested by Roger Pau Monné)
- fix words in commit message.
(suggested by Roger Pau Monné)
- change type of 'libxl__hw_info_to_libxl_cat_info' to void and use
assert to check invalid type. Then, remove check for
'libxl__hw_info_to_libxl_cat_info'.
(suggested by Roger Pau Monné)
- change type of 'libxl__xc_hw_info_to_libxl_hw_info' to void and use
assert to check invalid type. Then, remove check for
'libxl__xc_hw_info_to_libxl_hw_info'.
(suggested by Roger Pau Monné)
v3:
- remove casting.
(suggested by Roger Pau Monné)
- remove inline.
(suggested by Roger Pau Monné)
- change 'libxc__psr_hw_info_to_libxl_psr_hw_info' to
'libxl__xc_hw_info_to_libxl_hw_info'.
(suggested by Roger Pau Monné)
- remove '_hw' from parameter names.
(suggested by Roger Pau Monné)
- change some 'LOGE' to 'LOG'.
(suggested by Roger Pau Monné)
- check returned 'xc_type' and remove redundant 'lvl' check.
(suggested by Roger Pau Monné)
v2:
- split this patch out from a big patch in v1.
(suggested by Wei Liu)
- change 'CAT_INFO'/'MBA_INFO' to 'CAT' and 'MBA. Also the libxl structure
name 'cat_info'/'mba_info' is changed to 'cat'/'mba'.
(suggested by Chao Peng)
- call 'libxl_psr_hw_info_list_free' in 'libxl_psr_cat_get_info' to free
allocated resources.
(suggested by Chao Peng)
---
tools/libxl/libxl_psr.c | 131 ++--
1 file changed, 93 insertions(+), 38 deletions(-)
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index e1cc250..b053abd 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -386,56 +386,41 @@ static xc_psr_feat_type
libxl__feat_type_to_libxc_feat_type(
return xc_type;
}
+static void libxl__hw_info_to_libxl_cat_info(
+libxl_psr_feat_type type, libxl_psr_hw_info *hw_info,
+libxl_psr_cat_info *cat_info)
+{
+assert(type == LIBXL_PSR_FEAT_TYPE_CAT);
+
+cat_info->id = hw_info->id;
+cat_info->cos_max = hw_info->u.cat.cos_max;
+cat_info->cbm_len = hw_info->u.cat.cbm_len;
+cat_info->cdp_enabled = hw_info->u.cat.cdp_enabled;
+}
+
int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
unsigned int *nr, unsigned int lvl)
{
GC_INIT(ctx);
int rc;
-int i = 0, socketid, nr_sockets;
-libxl_bitmap socketmap;
+unsigned int i;
+libxl_psr_hw_info *hw_info;
libxl_psr_cat_info *ptr;
-xc_psr_hw_info hw_info;
-xc_psr_feat_type xc_type;
-
-libxl_bitmap_init(&socketmap);
-
-rc = libxl__count_physical_sockets(gc, &nr_sockets);
-if (rc) {
-LOGE(ERROR, "failed to get system socket count");
-goto out;
-}
-libxl_socket_bitmap_alloc(ctx, &socketmap, nr_sockets);
-rc = libxl_get_online_socketmap(ctx, &socketmap);
-if (rc < 0) {
-LOGE(ERROR, "failed to get available sockets");
+rc = libxl_psr_get_hw_info(ctx, LIBXL_PSR_FEAT_TYPE_CAT, lvl, nr,
&hw_info);
+if (rc)
goto out;
-}
-
-xc_type = libxl__feat_type_to_libxc_feat_type(LIBXL_PSR_FEAT_TYPE_CAT,
lvl);
-
-ptr = libxl__malloc(NOGC, nr_sockets * sizeof(libxl_psr_cat_info));
-
-libxl_for_each_set_bit(socketid, socketmap) {
-ptr[i].id = socketid;
-if (xc_psr_get_hw_info(ctx->xch, socketid, xc_type, &hw_info)) {
-LOGE(ERROR, "failed to get hw info");
-rc = ERROR_FAIL;
-free(ptr);
-goto out;
-}
-ptr[i].cos_max = hw_info.cat.cos_max;
-ptr[i].cbm_len = hw_info.cat.cbm_len;
-ptr[i].cdp_enabled = hw_info.cat.cdp_enabled;
+ptr = libxl__malloc(NOGC, *nr * sizeof(libxl_psr_cat_info));
-i++;
-}
+for (i = 0; i < *nr; i++)
+libxl__hw_info_to_libxl_cat_info(LIBXL_PSR_FEAT_TYPE_CAT,
+ &hw_info[i],
+ &ptr[i])
[Xen-devel] [PATCH v9 12/16] tools: implement the new xl get hw info interface
This patch implements a new xl get HW info interface. A new argument
is added for psr-hwinfo command to get and show MBA HW info.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Wei Liu
---
CC: Wei Liu
CC: Ian Jackson
CC: Roger Pau Monné
CC: Chao Peng
v3:
- change the format string of printf in 'psr_mba_hwinfo'.
(suggested by Roger Pau Monné)
- add 'const' for 'opts[]' in 'main_psr_hwinfo'.
(suggested by Roger Pau Monné)
v2:
- split out this patch from a big patch in v1.
(suggested by Wei Liu)
- change 'MBA_INFO' to 'MBA'. Also, change 'mba_info' to 'mba'.
(suggested by Chao Peng)
---
tools/xl/xl_cmdtable.c | 1 +
tools/xl/xl_psr.c | 39 ---
2 files changed, 37 insertions(+), 3 deletions(-)
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 5546cf6..249eb92 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -550,6 +550,7 @@ struct cmd_spec cmd_table[] = {
"[options]",
"-m, --cmt Show Cache Monitoring Technology (CMT) hardware info\n"
"-a, --cat Show Cache Allocation Technology (CAT) hardware info\n"
+ "-b, --mba Show Memory Bandwidth Allocation (MBA) hardware info\n"
},
{ "psr-cmt-attach",
&main_psr_cmt_attach, 0, 1,
diff --git a/tools/xl/xl_psr.c b/tools/xl/xl_psr.c
index ef00048..ab47d96 100644
--- a/tools/xl/xl_psr.c
+++ b/tools/xl/xl_psr.c
@@ -475,6 +475,31 @@ static int psr_l2_cat_hwinfo(void)
return rc;
}
+static int psr_mba_hwinfo(void)
+{
+int rc;
+unsigned int i, nr;
+libxl_psr_hw_info *info;
+
+rc = libxl_psr_get_hw_info(ctx, LIBXL_PSR_FEAT_TYPE_MBA, 0, &nr, &info);
+if (rc)
+return rc;
+
+printf("Memory Bandwidth Allocation (MBA):\n");
+
+for (i = 0; i < nr; i++) {
+printf("Socket ID : %u\n", info[i].id);
+printf("Linear Mode : %s\n",
+ info[i].u.mba.linear ? "Enabled" : "Disabled");
+printf("Maximum COS : %u\n", info[i].u.mba.cos_max);
+printf("Maximum Throttling Value: %u\n", info[i].u.mba.thrtl_max);
+printf("Default Throttling Value: %u\n", 0);
+}
+
+libxl_psr_hw_info_list_free(info, nr);
+return rc;
+}
+
int main_psr_cat_cbm_set(int argc, char **argv)
{
uint32_t domid;
@@ -593,20 +618,24 @@ int main_psr_cat_show(int argc, char **argv)
int main_psr_hwinfo(int argc, char **argv)
{
int opt, ret = 0;
-bool all = true, cmt = false, cat = false;
-static struct option opts[] = {
+bool all = true, cmt = false, cat = false, mba = false;
+static const struct option opts[] = {
{"cmt", 0, 0, 'm'},
{"cat", 0, 0, 'a'},
+{"mba", 0, 0, 'b'},
COMMON_LONG_OPTS
};
-SWITCH_FOREACH_OPT(opt, "ma", opts, "psr-hwinfo", 0) {
+SWITCH_FOREACH_OPT(opt, "mab", opts, "psr-hwinfo", 0) {
case 'm':
all = false; cmt = true;
break;
case 'a':
all = false; cat = true;
break;
+case 'b':
+all = false; mba = true;
+break;
}
if (!ret && (all || cmt))
@@ -619,6 +648,10 @@ int main_psr_hwinfo(int argc, char **argv)
if (all || cat)
ret = psr_l2_cat_hwinfo();
+/* MBA is independent of CMT and CAT */
+if (all || mba)
+ret = psr_mba_hwinfo();
+
return ret;
}
--
1.9.1
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v9 09/16] tools: create general interfaces to support psr allocation features
This patch creates general interfaces in libxl to support all psr
allocation features.
Add 'LIBXL_HAVE_PSR_GENERIC' to indicate interface change.
Please note, the functionality cannot work until later patches
are applied.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Wei Liu
---
CC: Wei Liu
CC: Ian Jackson
CC: Roger Pau Monné
CC: Chao Peng
v5:
- adjust parameters position in 'libxl_psr_get_hw_info'.
(suggested by Roger Pau Monné)
v4:
- add description for LIBXL_HAVE_PSR_GENERIC to mention newly added
public functions.
(suggested by Roger Pau Monné)
v3:
- change 'LIBXL_HAVE_PSR_MBA' to 'LIBXL_HAVE_PSR_GENERIC'.
(suggested by Roger Pau Monné)
- 'typedef enum libxl_psr_cbm_type libxl_psr_type;' in libxl.h.
(suggested by Roger Pau Monné and Wei Liu)
- change 'libxl_psr_cbm_type' to 'libxl_psr_type' in newly defined
interfaces.
(suggested by Roger Pau Monné)
v2:
- remove '_INFO' in 'libxl_psr_feat_type' and make corresponding
changes in 'libxl_psr_hw_info'.
(suggested by Chao Peng)
---
tools/libxl/libxl.h | 37 +
tools/libxl/libxl_psr.c | 25 +
tools/libxl/libxl_types.idl | 22 ++
3 files changed, 84 insertions(+)
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 5e9aed7..da8af81 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -982,6 +982,17 @@ void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, const
libxl_mac *src);
#define LIBXL_HAVE_PSR_L2_CAT 1
/*
+ * LIBXL_HAVE_PSR_GENERIC
+ *
+ * If this is defined, the Memory Bandwidth Allocation feature is supported.
+ * The following public functions are available:
+ * libxl_psr_{set/get}_val
+ * libxl_psr_get_hw_info
+ * libxl_psr_hw_info_list_free
+ */
+#define LIBXL_HAVE_PSR_GENERIC 1
+
+/*
* LIBXL_HAVE_MCA_CAPS
*
* If this is defined, setting MCA capabilities for HVM domain is supported.
@@ -2302,6 +2313,32 @@ int libxl_psr_cat_get_info(libxl_ctx *ctx,
libxl_psr_cat_info **info,
int libxl_psr_cat_get_l3_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
int *nr);
void libxl_psr_cat_info_list_free(libxl_psr_cat_info *list, int nr);
+
+typedef enum libxl_psr_cbm_type libxl_psr_type;
+
+/*
+ * Function to set a domain's value. It operates on a single or multiple
+ * target(s) defined in 'target_map'. 'target_map' specifies all the sockets
+ * to be operated on.
+ */
+int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_type type, libxl_bitmap *target_map,
+ uint64_t val);
+/*
+ * Function to get a domain's cbm. It operates on a single 'target'.
+ * 'target' specifies which socket to be operated on.
+ */
+int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_type type, unsigned int target,
+ uint64_t *val);
+/*
+ * On success, the function returns an array of elements in 'info',
+ * and the length in 'nr'.
+ */
+int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_feat_type type,
+ unsigned int lvl, unsigned int *nr,
+ libxl_psr_hw_info **info);
+void libxl_psr_hw_info_list_free(libxl_psr_hw_info *list, unsigned int nr);
#endif
/* misc */
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index 197505a..d4f5f67 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -428,6 +428,31 @@ void libxl_psr_cat_info_list_free(libxl_psr_cat_info
*list, int nr)
free(list);
}
+int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_type type, libxl_bitmap *target_map,
+ uint64_t val)
+{
+return ERROR_FAIL;
+}
+
+int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_type type, unsigned int target,
+ uint64_t *val)
+{
+return ERROR_FAIL;
+}
+
+int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_feat_type type,
+ unsigned int lvl, unsigned int *nr,
+ libxl_psr_hw_info **info)
+{
+return ERROR_FAIL;
+}
+
+void libxl_psr_hw_info_list_free(libxl_psr_hw_info *list, unsigned int nr)
+{
+}
+
/*
* Local variables:
* mode: C
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index a239324..ff75a8f 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -1033,6 +1033,7 @@ libxl_psr_cbm_type = Enumeration("psr_cbm_type", [
(2, "L3_CBM_CODE"),
(3, "L3_CBM_DATA"),
(4, "L2_CBM"),
+(5, "MBA_THRTL"),
])
libxl_psr_cat_info = Struct("psr_cat_info", [
@@ -1041,3 +1042,24
[Xen-devel] [PATCH v9 08/16] x86: implement set value flow for MBA
This patch implements set value flow for MBA including its callback
function and domctl interface.
Signed-off-by: Yi Sun
Reviewed-by: Jan Beulich
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v9:
- adjust codes in 'mba_sanitize_thrtl'.
(suggested by Jan Beulich)
v8:
- restore some old codes in 'cat_check_cbm'.
(suggested by Jan Beulich)
- use 'fls()' but not 'flsl()'.
(suggested by Jan Beulich)
- use plain '=' to assign value for thrtl in 'mba_sanitize_thrtl'.
(suggested by Jan Beulich)
v7:
- change name of 'check_val' to 'sanitize'.
(suggested by Jan Beulich)
- fix comments.
(suggested by Jan Beulich)
- add parentheses and change '== 0' to '!'.
(suggested by Jan Beulich)
- remove unnecessary check of 'mba.thrtl_max'.
(suggested by Jan Beulich)
- remove unnecessary intermediate variable 'mod'.
(suggested by Jan Beulich)
- refine an assignement sentence to use '&='.
(suggested by Jan Beulich)
- change type of last parameter of 'sanitize' to 'uint32_t' and
apply same change to 'cat_check_cbm'.
(suggested by Jan Beulich)
v6:
- split co-exist features' values setting flow to a new patch.
(suggested by Jan Beulich)
- restore codes related to 'mba_check_thrtl' and 'check_value'.
(suggested by Jan Beulich)
v5:
- adjust position of 'cat_check_cbm' to not to make changes so big.
(suggested by Roger Pau Monné)
- remove 'props' from 'struct cos_write_info'.
(suggested by Roger Pau Monné)
- make a single return statement in 'mba_check_thrtl'.
(suggested by Jan Beulich)
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- join two checks into a single if.
(suggested by Roger Pau Monné)
- remove redundant local variable 'array_len'.
(suggested by Roger Pau Monné)
v3:
- modify commit message to make it clear.
(suggested by Roger Pau Monné)
- modify functionality of 'check_val' to make it simple to only check value.
Change the last parameter type from 'unsigned long *' to 'unsigned long'.
(suggested by Roger Pau Monné)
- call rdmsrl to get value just written into MSR for MBA. Because HW can
automatically change input value to what it wants.
(suggested by Roger Pau Monné)
- change type of 'write_msr' to 'uint32_t' to return the value actually
written into MSR. Then, change 'do_write_psr_msrs' to set the returned
value into 'cos_reg_val[]'
- move the declaration of 'j' into loop in 'do_write_psr_msrs'.
(suggested by Roger Pau Monné)
- change 'mba_info' to 'mba'.
(suggested by Roger Pau Monné)
- change 'cat_info' to 'cat'.
(suggested by Roger Pau Monné)
- rename 'psr_cat/PSR_CAT' to 'psr_alloc/PSR_ALLOC' and remove 'op/OP'
from name.
(suggested by Roger Pau Monné)
- change 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
v2:
- remove linear mode 'thrtl_max' check in 'mba_check_thrtl' because it has
been checked in 'mba_init_feature'.
(suggested by Chao Peng)
- for non-linear mode, check if '*thrtl' is not 0 in 'mba_check_thrtl'. If
it is 0, we do not need to change it.
(suggested by Chao Peng)
- move comments to explain changes of 'cos_write_info' from psr.c to commit
message.
(suggested by Chao Peng)
---
xen/arch/x86/domctl.c | 6 ++
xen/arch/x86/psr.c | 46 +
xen/include/public/domctl.h | 1 +
3 files changed, 49 insertions(+), 4 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index f7d299d..2ae436b 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1465,6 +1465,12 @@ long arch_do_domctl(
PSR_TYPE_L2_CBM);
break;
+case XEN_DOMCTL_PSR_SET_MBA_THRTL:
+ret = psr_set_val(d, domctl->u.psr_alloc.target,
+ domctl->u.psr_alloc.data,
+ PSR_TYPE_MBA_THRTL);
+break;
+
#define domctl_psr_get_val(d, domctl, type, copyback) ({\
uint32_t v_;\
int r_ = psr_get_val((d), (domctl)->u.psr_alloc.target, \
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index a4901d8..0ba8ef8 100644
--- a/xen/arch/x86/psr.c
[Xen-devel] [PATCH v9 00/16] Enable Memory Bandwidth Allocation in Xen
Hi, all, We plan to bring a new PSR (Platform Shared Resource) feature called Intel Memory Bandwidth Allocation (MBA) to Xen. Besides the MBA enabling, we change some interfaces to make them more general but not only for CAT. Any comments are welcome! You can find this series at: https://github.com/yisun-git/xen_mba mba_v9 This version has been rebased onto latest staging branch with commit 6ccf25. BRs, Sun Yi CC: Jan Beulich CC: Andrew Cooper CC: Wei Liu CC: Ian Jackson CC: Daniel De Graaf CC: Roger Pau Monné CC: Konrad Rzeszutek Wilk CC: Chao Peng CC: Julien Grall --- Acked and Reviewed list before V9: a - Acked-by r - Reviewed-by r patch 1 - docs: create Memory Bandwidth Allocation (MBA) feature document ar patch 2 - Rename PSR sysctl/domctl interfaces and xsm policy to make them be general ar patch 3 - x86: rename 'cbm_type' to 'psr_type' to make it general ar patch 4 - x86: a few optimizations to psr codes r patch 5 - x86: implement data structure and CPU init flow for MBA ar patch 6 - x86: implement get hw info flow for MBA ar patch 7 - x86: implement get value interface for MBA r patch 8 - x86: implement set value flow for MBA ar patch 9 - tools: create general interfaces to support psr allocation features ar patch 10 - tools: implement the new libxc get hw info interface ar patch 11 - tools: implement the new libxl get hw info interface ar patch 12 - tools: implement the new xl get hw info interface ar patch 13 - tools: rename 'xc_psr_cat_type' to 'xc_psr_type' ar patch 14 - tools: implement new generic get value interface and MBA get value command ar patch 15 - tools: implement new generic set value interface and MBA set value command ar patch 16 - docs: add MBA description in docs --- V9 change list: Patch 8: - adjust codes in 'mba_sanitize_thrtl'. (suggested by Jan Beulich) Yi Sun (16): docs: create Memory Bandwidth Allocation (MBA) feature document Rename PSR sysctl/domctl interfaces and xsm policy to make them be general x86: rename 'cbm_type' to 'psr_type' to make it general x86: a few optimizations to psr codes x86: implement data structure and CPU init flow for MBA x86: implement get hw info flow for MBA x86: implement get value interface for MBA x86: implement set value flow for MBA tools: create general interfaces to support psr allocation features tools: implement the new libxc get hw info interface tools: implement the new libxl get hw info interface tools: implement the new xl get hw info interface tools: rename 'xc_psr_cat_type' to 'xc_psr_type' tools: implement new generic get value interface and MBA get value command tools: implement new generic set value interface and MBA set value command docs: add MBA description in docs docs/features/intel_psr_mba.pandoc | 297 docs/man/xl.pod.1.in| 33 docs/misc/xl-psr.markdown | 62 tools/flask/policy/modules/dom0.te | 4 +- tools/libxc/include/xenctrl.h | 44 -- tools/libxc/xc_psr.c| 109 +++-- tools/libxl/libxl.h | 37 + tools/libxl/libxl_psr.c | 223 +-- tools/libxl/libxl_types.idl | 22 +++ tools/xl/xl.h | 2 + tools/xl/xl_cmdtable.c | 12 ++ tools/xl/xl_psr.c | 279 ++--- xen/arch/x86/domctl.c | 87 ++- xen/arch/x86/psr.c | 284 +- xen/arch/x86/sysctl.c | 57 --- xen/include/asm-x86/msr-index.h | 1 + xen/include/asm-x86/psr.h | 24 +-- xen/include/public/domctl.h | 26 ++-- xen/include/public/sysctl.h | 20 ++- xen/xsm/flask/hooks.c | 8 +- xen/xsm/flask/policy/access_vectors | 8 +- 21 files changed, 1314 insertions(+), 325 deletions(-) create mode 100644 docs/features/intel_psr_mba.pandoc -- 1.9.1 ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v8 00/16] Enable Memory Bandwidth Allocation in Xen
On 17-10-20 09:20:00, Yi Sun wrote: > On 17-10-19 16:08:09, Konrad Rzeszutek Wilk wrote: > > On Mon, Oct 16, 2017 at 11:04:05AM +0800, Yi Sun wrote: > > > a - Acked-by > > > r - Reviewed-by > > > > > > r patch 1 - docs: create Memory Bandwidth Allocation (MBA) feature > > > document > > > ar patch 2 - Rename PSR sysctl/domctl interfaces and xsm policy to > > > make them be general > > > ar patch 3 - x86: rename 'cbm_type' to 'psr_type' to make it general > > > ar patch 4 - x86: a few optimizations to psr codes > > > r patch 5 - x86: implement data structure and CPU init flow for MBA > > > ar patch 6 - x86: implement get hw info flow for MBA > > > ar patch 7 - x86: implement get value interface for MBA > > > > So 8 is missing and Ack/Review-edyb? > > > Jan provided Reviewed-by to patch 8 in this version. > > > > ar patch 9 - tools: create general interfaces to support psr > > > allocation features > > > ar patch 10 - tools: implement the new libxc get hw info interface > > > ar patch 11 - tools: implement the new libxl get hw info interface > > > ar patch 12 - tools: implement the new xl get hw info interface > > > ar patch 13 - tools: rename 'xc_psr_cat_type' to 'xc_psr_type' > > > ar patch 14 - tools: implement new generic get value interface and MBA > > > get value command > > > ar patch 15 - tools: implement new generic set value interface and MBA > > > set value command > > > ar patch 16 - docs: add MBA description in docs > > > > > > Also I tried to merge this on 'staging' and had a bit of issues. By any > > chance > > do you have an up-to-date branc? > > My codes base on commit 572a78. Let me try to apply patches onto staging. > Thanks for your attention to this series! > Andrew's below patch in staging branch causes the conflict. commit 5b42c82f5584ca8b0e169c6de1b6d81214ea07f2 Author: Andrew Cooper Date: Fri Oct 6 20:00:00 2017 +0100 tools/libxc: Fix domid parameter types I will rebase my patch onto it and submit version 9. > BRs, > Sun Yi > > ___ > Xen-devel mailing list > Xen-devel@lists.xen.org > https://lists.xen.org/xen-devel ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v8 00/16] Enable Memory Bandwidth Allocation in Xen
On 17-10-19 16:08:09, Konrad Rzeszutek Wilk wrote: > On Mon, Oct 16, 2017 at 11:04:05AM +0800, Yi Sun wrote: > > Hi, all, > > > > We plan to bring a new PSR (Platform Shared Resource) feature called > > Intel Memory Bandwidth Allocation (MBA) to Xen. > > > > Besides the MBA enabling, we change some interfaces to make them more > > general but not only for CAT. > > > > Any comments are welcome! > > > > You can find this series at: > > https://github.com/yisun-git/xen_mba mba_v8 > > > > This version bases on below pre-fix patch which has been merged into staging > > branch: > > "x86: psr: support co-exist features' values setting" > > https://lists.xen.org/archives/html/xen-devel/2017-10/msg00866.html > > > > CC: Jan Beulich > > CC: Andrew Cooper > > CC: Wei Liu > > CC: Ian Jackson > > CC: Daniel De Graaf > > CC: Roger Pau Monn? > > CC: Konrad Rzeszutek Wilk > > CC: Chao Peng > > CC: Julien Grall > > > > --- > > Acked and Reviewed list before V8: > > > > a - Acked-by > > r - Reviewed-by > > > > r patch 1 - docs: create Memory Bandwidth Allocation (MBA) feature > > document > > ar patch 2 - Rename PSR sysctl/domctl interfaces and xsm policy to make > > them be general > > ar patch 3 - x86: rename 'cbm_type' to 'psr_type' to make it general > > ar patch 4 - x86: a few optimizations to psr codes > > r patch 5 - x86: implement data structure and CPU init flow for MBA > > ar patch 6 - x86: implement get hw info flow for MBA > > ar patch 7 - x86: implement get value interface for MBA > > So 8 is missing and Ack/Review-edyb? > Jan provided Reviewed-by to patch 8 in this version. > > ar patch 9 - tools: create general interfaces to support psr allocation > > features > > ar patch 10 - tools: implement the new libxc get hw info interface > > ar patch 11 - tools: implement the new libxl get hw info interface > > ar patch 12 - tools: implement the new xl get hw info interface > > ar patch 13 - tools: rename 'xc_psr_cat_type' to 'xc_psr_type' > > ar patch 14 - tools: implement new generic get value interface and MBA > > get value command > > ar patch 15 - tools: implement new generic set value interface and MBA > > set value command > > ar patch 16 - docs: add MBA description in docs > > > Also I tried to merge this on 'staging' and had a bit of issues. By any chance > do you have an up-to-date branc? My codes base on commit 572a78. Let me try to apply patches onto staging. Thanks for your attention to this series! BRs, Sun Yi ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v9 02/16] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
On 17-10-19 05:36:37, Jan Beulich wrote: > >>> On 19.10.17 at 03:22, wrote: > > v9: > > - bump domctl/sysctl version number as this patch missed 4.10. > > And add info in commit message. > > (suggested by Jan Beulich) > > I don't understand this - I've specifically given reason why you > _don't_ need to increment them, and now you send a patch > incrementing them, indicating I would have suggested this? > Oh, very sorry, I misunderstood that. > Jan ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v9 02/16] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
This patch renames PSR sysctl/domctl interfaces and related xsm policy to
make them be general for all resource allocation features but not only
for CAT. Then, we can resuse the interfaces for all allocation features.
Basically, it changes 'psr_cat_op' to 'psr_alloc', and remove 'CAT_' from some
macros. E.g.:
1. psr_cat_op -> psr_alloc
2. XEN_DOMCTL_psr_cat_op -> XEN_DOMCTL_psr_alloc
3. XEN_SYSCTL_psr_cat_op -> XEN_SYSCTL_psr_alloc
4. XEN_DOMCTL_PSR_CAT_SET_L3_CBM -> XEN_DOMCTL_PSR_SET_L3_CBM
5. XEN_SYSCTL_PSR_CAT_get_l3_info -> XEN_SYSCTL_PSR_get_l3_info
Bump domctl/sysctl version number.
Signed-off-by: Yi Sun
Reviewed-by: Wei Liu
Reviewed-by: Roger Pau Monné
Acked-by: Jan Beulich
Acked-by: Daniel De Graaf
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Ian Jackson
CC: Daniel De Graaf
CC: Roger Pau Monné
CC: Chao Peng
v9:
- bump domctl/sysctl version number as this patch missed 4.10.
And add info in commit message.
(suggested by Jan Beulich)
v7:
- add single trailing underscore for internal variabled in macro.
(suggested by Jan Beulich)
- add parentheses for input parameters of marcro.
(suggested by Jan Beulich)
- adjust the postion of macro.
(suggested by Jan Beulich)
v6:
- move macro definition into the function and undefine it after use.
(suggested by Roger Pau Monné)
- do not bump sysctl version because it has been bumped for 4.10.
(suggested by Roger Pau Monné)
v5:
- remove domctl version number upgrade.
(suggested by Jan Beulich)
- restore 'XEN_SYSCTL_PSR_CAT_L3_CDP'.
(suggested by Jan Beulich)
- define a local macro to complete psr get value flow.
(suggested by Roger Pau Monné)
- remove 'Reviewed-by' and 'Acked-by'.
(suggested by Wei Liu)
v4:
- remove 'ALLOC_' from names.
(suggested by Roger Pau Monné)
- fix comments.
(suggested by Roger Pau Monné)
v3:
- remove 'op/OP' from names and modify some names from 'PSR_CAT' to
'PSR_ALLOC'.
(suggested by Roger Pau Monné)
v1:
- add description about what to be changed in commit message.
(suggested by Wei Liu)
- bump sysctl/domctl version numbers.
(suggested by Wei Liu)
---
tools/flask/policy/modules/dom0.te | 4 +--
tools/libxc/xc_psr.c| 50 +-
xen/arch/x86/domctl.c | 71 ++---
xen/arch/x86/sysctl.c | 28 +++
xen/include/public/domctl.h | 26 +++---
xen/include/public/sysctl.h | 14
xen/xsm/flask/hooks.c | 8 ++---
xen/xsm/flask/policy/access_vectors | 8 ++---
8 files changed, 104 insertions(+), 105 deletions(-)
diff --git a/tools/flask/policy/modules/dom0.te
b/tools/flask/policy/modules/dom0.te
index 1643b40..07de3d5 100644
--- a/tools/flask/policy/modules/dom0.te
+++ b/tools/flask/policy/modules/dom0.te
@@ -14,7 +14,7 @@ allow dom0_t xen_t:xen {
tmem_control getscheduler setscheduler
};
allow dom0_t xen_t:xen2 {
- resource_op psr_cmt_op psr_cat_op pmu_ctrl get_symbol
+ resource_op psr_cmt_op psr_alloc pmu_ctrl get_symbol
get_cpu_levelling_caps get_cpu_featureset livepatch_op
gcov_op set_parameter
};
@@ -39,7 +39,7 @@ allow dom0_t dom0_t:domain {
};
allow dom0_t dom0_t:domain2 {
set_cpuid gettsc settsc setscheduler set_max_evtchn set_vnumainfo
- get_vnumainfo psr_cmt_op psr_cat_op set_gnttab_limits
+ get_vnumainfo psr_cmt_op psr_alloc set_gnttab_limits
};
allow dom0_t dom0_t:resource { add remove };
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 039b920..5c54a35 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -258,27 +258,27 @@ int xc_psr_cat_set_domain_data(xc_interface *xch,
uint32_t domid,
switch ( type )
{
case XC_PSR_CAT_L3_CBM:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM;
+cmd = XEN_DOMCTL_PSR_SET_L3_CBM;
break;
case XC_PSR_CAT_L3_CBM_CODE:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_CODE;
+cmd = XEN_DOMCTL_PSR_SET_L3_CODE;
break;
case XC_PSR_CAT_L3_CBM_DATA:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_DATA;
+cmd = XEN_DOMCTL_PSR_SET_L3_DATA;
break;
case XC_PSR_CAT_L2_CBM:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM;
+cmd = XEN_DOMCTL_PSR_SET_L2_CBM;
break;
default:
errno = EINVAL;
return -1;
}
-domctl.cmd = XEN_DOMCTL_psr_cat_op;
+domctl.cmd = XEN_DOMCTL_psr_alloc;
domctl.domain = (domid_t)domid;
-domctl.u.psr_cat_op.cmd = cmd;
-domctl.u.psr_cat_op.target = target;
-domctl.u.psr_cat_op.data = data;
+domctl.u.psr_alloc.cmd = cmd;
+domctl.u.psr_alloc.target = target;
+domctl.u.psr_alloc.data = data;
return do_domctl(xch
Re: [Xen-devel] [PATCH v9 08/16] x86: implement set value flow for MBA
I forgot to add 'Reviewed-by' provided by Jan. So please ignore this but refer v9.1. Thanks! On 17-10-17 09:04:18, Yi Sun wrote: > This patch implements set value flow for MBA including its callback > function and domctl interface. > > Signed-off-by: Yi Sun ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v9.1 08/16] x86: implement set value flow for MBA
This patch implements set value flow for MBA including its callback
function and domctl interface.
Signed-off-by: Yi Sun
Reviewed-by: Jan Beulich
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v9:
- adjust codes in 'mba_sanitize_thrtl'.
(suggested by Jan Beulich)
v8:
- restore some old codes in 'cat_check_cbm'.
(suggested by Jan Beulich)
- use 'fls()' but not 'flsl()'.
(suggested by Jan Beulich)
- use plain '=' to assign value for thrtl in 'mba_sanitize_thrtl'.
(suggested by Jan Beulich)
v7:
- change name of 'check_val' to 'sanitize'.
(suggested by Jan Beulich)
- fix comments.
(suggested by Jan Beulich)
- add parentheses and change '== 0' to '!'.
(suggested by Jan Beulich)
- remove unnecessary check of 'mba.thrtl_max'.
(suggested by Jan Beulich)
- remove unnecessary intermediate variable 'mod'.
(suggested by Jan Beulich)
- refine an assignement sentence to use '&='.
(suggested by Jan Beulich)
- change type of last parameter of 'sanitize' to 'uint32_t' and
apply same change to 'cat_check_cbm'.
(suggested by Jan Beulich)
v6:
- split co-exist features' values setting flow to a new patch.
(suggested by Jan Beulich)
- restore codes related to 'mba_check_thrtl' and 'check_value'.
(suggested by Jan Beulich)
v5:
- adjust position of 'cat_check_cbm' to not to make changes so big.
(suggested by Roger Pau Monné)
- remove 'props' from 'struct cos_write_info'.
(suggested by Roger Pau Monné)
- make a single return statement in 'mba_check_thrtl'.
(suggested by Jan Beulich)
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- join two checks into a single if.
(suggested by Roger Pau Monné)
- remove redundant local variable 'array_len'.
(suggested by Roger Pau Monné)
v3:
- modify commit message to make it clear.
(suggested by Roger Pau Monné)
- modify functionality of 'check_val' to make it simple to only check value.
Change the last parameter type from 'unsigned long *' to 'unsigned long'.
(suggested by Roger Pau Monné)
- call rdmsrl to get value just written into MSR for MBA. Because HW can
automatically change input value to what it wants.
(suggested by Roger Pau Monné)
- change type of 'write_msr' to 'uint32_t' to return the value actually
written into MSR. Then, change 'do_write_psr_msrs' to set the returned
value into 'cos_reg_val[]'
- move the declaration of 'j' into loop in 'do_write_psr_msrs'.
(suggested by Roger Pau Monné)
- change 'mba_info' to 'mba'.
(suggested by Roger Pau Monné)
- change 'cat_info' to 'cat'.
(suggested by Roger Pau Monné)
- rename 'psr_cat/PSR_CAT' to 'psr_alloc/PSR_ALLOC' and remove 'op/OP'
from name.
(suggested by Roger Pau Monné)
- change 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
v2:
- remove linear mode 'thrtl_max' check in 'mba_check_thrtl' because it has
been checked in 'mba_init_feature'.
(suggested by Chao Peng)
- for non-linear mode, check if '*thrtl' is not 0 in 'mba_check_thrtl'. If
it is 0, we do not need to change it.
(suggested by Chao Peng)
- move comments to explain changes of 'cos_write_info' from psr.c to commit
message.
(suggested by Chao Peng)
---
xen/arch/x86/domctl.c | 6 ++
xen/arch/x86/psr.c | 46 +
xen/include/public/domctl.h | 1 +
3 files changed, 49 insertions(+), 4 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index b726eae..4bca15d 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1465,6 +1465,12 @@ long arch_do_domctl(
PSR_TYPE_L2_CBM);
break;
+case XEN_DOMCTL_PSR_SET_MBA_THRTL:
+ret = psr_set_val(d, domctl->u.psr_alloc.target,
+ domctl->u.psr_alloc.data,
+ PSR_TYPE_MBA_THRTL);
+break;
+
#define domctl_psr_get_val(d, domctl, type, copyback) ({\
uint32_t v_;\
int r_ = psr_get_val((d), (domctl)->u.psr_alloc.target, \
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 549f21b..e8f6dd8 100644
--- a/xen/arch/x86/psr.c
[Xen-devel] [PATCH v9 08/16] x86: implement set value flow for MBA
This patch implements set value flow for MBA including its callback
function and domctl interface.
Signed-off-by: Yi Sun
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v9:
- adjust codes in 'mba_sanitize_thrtl'.
(suggested by Jan Beulich)
v8:
- restore some old codes in 'cat_check_cbm'.
(suggested by Jan Beulich)
- use 'fls()' but not 'flsl()'.
(suggested by Jan Beulich)
- use plain '=' to assign value for thrtl in 'mba_sanitize_thrtl'.
(suggested by Jan Beulich)
v7:
- change name of 'check_val' to 'sanitize'.
(suggested by Jan Beulich)
- fix comments.
(suggested by Jan Beulich)
- add parentheses and change '== 0' to '!'.
(suggested by Jan Beulich)
- remove unnecessary check of 'mba.thrtl_max'.
(suggested by Jan Beulich)
- remove unnecessary intermediate variable 'mod'.
(suggested by Jan Beulich)
- refine an assignement sentence to use '&='.
(suggested by Jan Beulich)
- change type of last parameter of 'sanitize' to 'uint32_t' and
apply same change to 'cat_check_cbm'.
(suggested by Jan Beulich)
v6:
- split co-exist features' values setting flow to a new patch.
(suggested by Jan Beulich)
- restore codes related to 'mba_check_thrtl' and 'check_value'.
(suggested by Jan Beulich)
v5:
- adjust position of 'cat_check_cbm' to not to make changes so big.
(suggested by Roger Pau Monné)
- remove 'props' from 'struct cos_write_info'.
(suggested by Roger Pau Monné)
- make a single return statement in 'mba_check_thrtl'.
(suggested by Jan Beulich)
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- join two checks into a single if.
(suggested by Roger Pau Monné)
- remove redundant local variable 'array_len'.
(suggested by Roger Pau Monné)
v3:
- modify commit message to make it clear.
(suggested by Roger Pau Monné)
- modify functionality of 'check_val' to make it simple to only check value.
Change the last parameter type from 'unsigned long *' to 'unsigned long'.
(suggested by Roger Pau Monné)
- call rdmsrl to get value just written into MSR for MBA. Because HW can
automatically change input value to what it wants.
(suggested by Roger Pau Monné)
- change type of 'write_msr' to 'uint32_t' to return the value actually
written into MSR. Then, change 'do_write_psr_msrs' to set the returned
value into 'cos_reg_val[]'
- move the declaration of 'j' into loop in 'do_write_psr_msrs'.
(suggested by Roger Pau Monné)
- change 'mba_info' to 'mba'.
(suggested by Roger Pau Monné)
- change 'cat_info' to 'cat'.
(suggested by Roger Pau Monné)
- rename 'psr_cat/PSR_CAT' to 'psr_alloc/PSR_ALLOC' and remove 'op/OP'
from name.
(suggested by Roger Pau Monné)
- change 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
v2:
- remove linear mode 'thrtl_max' check in 'mba_check_thrtl' because it has
been checked in 'mba_init_feature'.
(suggested by Chao Peng)
- for non-linear mode, check if '*thrtl' is not 0 in 'mba_check_thrtl'. If
it is 0, we do not need to change it.
(suggested by Chao Peng)
- move comments to explain changes of 'cos_write_info' from psr.c to commit
message.
(suggested by Chao Peng)
---
xen/arch/x86/domctl.c | 6 ++
xen/arch/x86/psr.c | 46 +
xen/include/public/domctl.h | 1 +
3 files changed, 49 insertions(+), 4 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index b726eae..4bca15d 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1465,6 +1465,12 @@ long arch_do_domctl(
PSR_TYPE_L2_CBM);
break;
+case XEN_DOMCTL_PSR_SET_MBA_THRTL:
+ret = psr_set_val(d, domctl->u.psr_alloc.target,
+ domctl->u.psr_alloc.data,
+ PSR_TYPE_MBA_THRTL);
+break;
+
#define domctl_psr_get_val(d, domctl, type, copyback) ({\
uint32_t v_;\
int r_ = psr_get_val((d), (domctl)->u.psr_alloc.target, \
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 549f21b..e8f6dd8 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/
Re: [Xen-devel] [PATCH v8 08/16] x86: implement set value flow for MBA
On 17-10-16 06:49:49, Jan Beulich wrote:
> >>> On 16.10.17 at 05:04, wrote:
> > This patch implements set value flow for MBA including its callback
> > function and domctl interface.
> >
> > Signed-off-by: Yi Sun
>
> Reviewed-by: Jan Beulich
>
> > v8:
> > - restore some unnecessary changes in 'cat_check_cbm'.
> > (suggested by Jan Beulich)
>
> This reads as if you did exactly the opposite thing of what you
> actually did.
>
> > +static bool mba_sanitize_thrtl(const struct feat_node *feat, uint32_t
> > *thrtl)
> > +{
> > +if ( *thrtl > feat->mba.thrtl_max )
> > +return false;
>
> Wouldn't it be better to do this check after ...
>
> > +/*
> > + * Per SDM (chapter "Memory Bandwidth Allocation Configuration"):
> > + * 1. Linear mode: In the linear mode the input precision is defined
> > + *as 100-(MBA_MAX). For instance, if the MBA_MAX value is 90, the
> > + *input precision is 10%. Values not an even multiple of the
> > + *precision (e.g., 12%) will be rounded down (e.g., to 10% delay
> > + *applied).
> > + * 2. Non-linear mode: Input delay values are powers-of-two from zero
> > + *to the MBA_MAX value from CPUID. In this case any values not a
> > + *power of two will be rounded down the next nearest power of two.
> > + */
> > +if ( feat->mba.linear )
> > +*thrtl -= *thrtl % (100 - feat->mba.thrtl_max);
> > +else
> > +{
> > +/* Not power of 2. */
> > +if ( *thrtl & (*thrtl - 1) )
> > +*thrtl = 1 << (fls(*thrtl) - 1);
> > +}
>
> ... these adjustments? That way someone specifying e.g. a linear
> value of 95% would get 90% just like for 85% (s)he would get
> 80%.
>
> > +return true;
>
> If so, the return statement could simply become
>
> return *thrtl <= feat->mba.thrtl_max;
>
> My R-b also applies if you decide to make this change.
>
Thank you for the suggestion! It is not worthy to send whole patch set out.
I will just update this patch.
> Jan
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v8 11/16] tools: implement the new libxl get hw info interface
This patch implements the new libxl get hw info interface,
'libxl_psr_get_hw_info', which is suitable to all psr allocation
features. It also implements corresponding list free function,
'libxl_psr_hw_info_list_free' and makes 'libxl_psr_cat_get_info' call
'libxl_psr_get_hw_info' to avoid redundant code in libxl_psr.c.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Wei Liu
---
CC: Wei Liu
CC: Ian Jackson
CC: Roger Pau Monné
CC: Chao Peng
v5:
- change 'if (rc < 0)' to 'if (rc)'.
(suggested by Roger Pau Monné)
v4:
- remove 'xc_' from struct name.
(suggested by Roger Pau Monné)
- fix words in commit message.
(suggested by Roger Pau Monné)
- change type of 'libxl__hw_info_to_libxl_cat_info' to void and use
assert to check invalid type. Then, remove check for
'libxl__hw_info_to_libxl_cat_info'.
(suggested by Roger Pau Monné)
- change type of 'libxl__xc_hw_info_to_libxl_hw_info' to void and use
assert to check invalid type. Then, remove check for
'libxl__xc_hw_info_to_libxl_hw_info'.
(suggested by Roger Pau Monné)
v3:
- remove casting.
(suggested by Roger Pau Monné)
- remove inline.
(suggested by Roger Pau Monné)
- change 'libxc__psr_hw_info_to_libxl_psr_hw_info' to
'libxl__xc_hw_info_to_libxl_hw_info'.
(suggested by Roger Pau Monné)
- remove '_hw' from parameter names.
(suggested by Roger Pau Monné)
- change some 'LOGE' to 'LOG'.
(suggested by Roger Pau Monné)
- check returned 'xc_type' and remove redundant 'lvl' check.
(suggested by Roger Pau Monné)
v2:
- split this patch out from a big patch in v1.
(suggested by Wei Liu)
- change 'CAT_INFO'/'MBA_INFO' to 'CAT' and 'MBA. Also the libxl structure
name 'cat_info'/'mba_info' is changed to 'cat'/'mba'.
(suggested by Chao Peng)
- call 'libxl_psr_hw_info_list_free' in 'libxl_psr_cat_get_info' to free
allocated resources.
(suggested by Chao Peng)
---
tools/libxl/libxl_psr.c | 131 ++--
1 file changed, 93 insertions(+), 38 deletions(-)
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index e1cc250..b053abd 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -386,56 +386,41 @@ static xc_psr_feat_type
libxl__feat_type_to_libxc_feat_type(
return xc_type;
}
+static void libxl__hw_info_to_libxl_cat_info(
+libxl_psr_feat_type type, libxl_psr_hw_info *hw_info,
+libxl_psr_cat_info *cat_info)
+{
+assert(type == LIBXL_PSR_FEAT_TYPE_CAT);
+
+cat_info->id = hw_info->id;
+cat_info->cos_max = hw_info->u.cat.cos_max;
+cat_info->cbm_len = hw_info->u.cat.cbm_len;
+cat_info->cdp_enabled = hw_info->u.cat.cdp_enabled;
+}
+
int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
unsigned int *nr, unsigned int lvl)
{
GC_INIT(ctx);
int rc;
-int i = 0, socketid, nr_sockets;
-libxl_bitmap socketmap;
+unsigned int i;
+libxl_psr_hw_info *hw_info;
libxl_psr_cat_info *ptr;
-xc_psr_hw_info hw_info;
-xc_psr_feat_type xc_type;
-
-libxl_bitmap_init(&socketmap);
-
-rc = libxl__count_physical_sockets(gc, &nr_sockets);
-if (rc) {
-LOGE(ERROR, "failed to get system socket count");
-goto out;
-}
-libxl_socket_bitmap_alloc(ctx, &socketmap, nr_sockets);
-rc = libxl_get_online_socketmap(ctx, &socketmap);
-if (rc < 0) {
-LOGE(ERROR, "failed to get available sockets");
+rc = libxl_psr_get_hw_info(ctx, LIBXL_PSR_FEAT_TYPE_CAT, lvl, nr,
&hw_info);
+if (rc)
goto out;
-}
-
-xc_type = libxl__feat_type_to_libxc_feat_type(LIBXL_PSR_FEAT_TYPE_CAT,
lvl);
-
-ptr = libxl__malloc(NOGC, nr_sockets * sizeof(libxl_psr_cat_info));
-
-libxl_for_each_set_bit(socketid, socketmap) {
-ptr[i].id = socketid;
-if (xc_psr_get_hw_info(ctx->xch, socketid, xc_type, &hw_info)) {
-LOGE(ERROR, "failed to get hw info");
-rc = ERROR_FAIL;
-free(ptr);
-goto out;
-}
-ptr[i].cos_max = hw_info.cat.cos_max;
-ptr[i].cbm_len = hw_info.cat.cbm_len;
-ptr[i].cdp_enabled = hw_info.cat.cdp_enabled;
+ptr = libxl__malloc(NOGC, *nr * sizeof(libxl_psr_cat_info));
-i++;
-}
+for (i = 0; i < *nr; i++)
+libxl__hw_info_to_libxl_cat_info(LIBXL_PSR_FEAT_TYPE_CAT,
+ &hw_info[i],
+ &ptr[i])
[Xen-devel] [PATCH v8 07/16] x86: implement get value interface for MBA
This patch implements get value domctl interface for MBA.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Jan Beulich
---
CC: Andrew Cooper
CC: Jan Beulich
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v5:
- use newly defined macro to get MBA thrtl.
(suggested by Roger Pau Monné)
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
v3:
- change 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
---
xen/arch/x86/domctl.c | 4
xen/include/public/domctl.h | 1 +
2 files changed, 5 insertions(+)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index bc025ce..b726eae 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1491,6 +1491,10 @@ long arch_do_domctl(
ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L2_CBM, copyback);
break;
+case XEN_DOMCTL_PSR_GET_MBA_THRTL:
+ret = domctl_psr_get_val(d, domctl, PSR_TYPE_MBA_THRTL, copyback);
+break;
+
#undef domctl_psr_get_val
default:
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index 4a66be1..002a26f 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1070,6 +1070,7 @@ struct xen_domctl_psr_alloc {
#define XEN_DOMCTL_PSR_GET_L3_DATA5
#define XEN_DOMCTL_PSR_SET_L2_CBM 6
#define XEN_DOMCTL_PSR_GET_L2_CBM 7
+#define XEN_DOMCTL_PSR_GET_MBA_THRTL 9
uint32_t cmd; /* IN: XEN_DOMCTL_PSR_* */
uint32_t target;/* IN */
uint64_t data; /* IN/OUT */
--
1.9.1
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v8 16/16] docs: add MBA description in docs
This patch adds MBA description in related documents. Signed-off-by: Yi Sun Acked-by: Wei Liu Reviewed-by: Roger Pau Monné --- CC: Ian Jackson CC: Wei Liu CC: Roger Pau Monné CC: Chao Peng v5: - remove 'closed-loop' in 'xl-psr.markdown' (suggested by Roger Pau Monné) v4: - modify description of MBA in 'xl.pod.1.in' to be same as feature doc. (suggested by Roger Pau Monné) - fix words issue. (suggested by Roger Pau Monné) v2: - state the value type shown by 'psr-mba-show'. For linear mode, it shows decimal value. For non-linear mode, it shows hexadecimal value. (suggested by Chao Peng) --- docs/man/xl.pod.1.in | 33 + docs/misc/xl-psr.markdown | 62 +++ 2 files changed, 95 insertions(+) diff --git a/docs/man/xl.pod.1.in b/docs/man/xl.pod.1.in index cd8bb1c..324ef9e 100644 --- a/docs/man/xl.pod.1.in +++ b/docs/man/xl.pod.1.in @@ -1845,6 +1845,39 @@ processed. =back +=head2 Memory Bandwidth Allocation + +Intel Skylake and later server platforms offer capabilities to configure and +make use of the Memory Bandwidth Allocation (MBA) mechanisms, which provides +OS/VMMs the ability to slow misbehaving apps/VMs by using a credit-based +throttling mechanism. In the Xen implementation, MBA is used to control memory +bandwidth on VM basis. To enforce bandwidth on a specific domain, just set +throttling value (THRTL) for the domain. + +=over 4 + +=item B [I] I I + +Set throttling value (THRTL) for a domain. For how to specify I +please refer to L<http://xenbits.xen.org/docs/unstable/misc/xl-psr.html>. + +B + +=over 4 + +=item B<-s SOCKET>, B<--socket=SOCKET> + +Specify the socket to process, otherwise all sockets are processed. + +=back + +=item B [I] + +Show MBA settings for a certain domain or all domains. For linear mode, it +shows the decimal value. For non-linear mode, it shows hexadecimal value. + +=back + =head1 IGNORED FOR COMPATIBILITY WITH XM xl is mostly command-line compatible with the old xm utility used with diff --git a/docs/misc/xl-psr.markdown b/docs/misc/xl-psr.markdown index 04dd957..3d196ed 100644 --- a/docs/misc/xl-psr.markdown +++ b/docs/misc/xl-psr.markdown @@ -186,6 +186,68 @@ Setting data CBM for a domain: Setting the same code and data CBM for a domain: `xl psr-cat-set ` +## Memory Bandwidth Allocation (MBA) + +Memory Bandwidth Allocation (MBA) is a new feature available on Intel +Skylake and later server platforms that allows an OS or Hypervisor/VMM to +slow misbehaving apps/VMs by using a credit-based throttling mechanism. To +enforce bandwidth on a specific domain, just set throttling value (THRTL) +into Class of Service (COS). MBA provides two THRTL mode. One is linear mode +and the other is non-linear mode. + +In the linear mode the input precision is defined as 100-(THRTL_MAX). Values +not an even multiple of the precision (e.g., 12%) will be rounded down (e.g., +to 10% delay by the hardware). + +If linear values are not supported then input delay values are powers-of-two +from zero to the THRTL_MAX value from CPUID. In this case any values not a power +of two will be rounded down the next nearest power of two. + +For example, assuming a system with 2 domains: + + * A THRTL of 0x0 for every domain means each domain can access the whole cache + without any delay. This is the default. + + * Linear mode: Giving one domain a THRTL of 0xC and the other domain's 0 means + that the first domain gets 10% delay to access the cache and the other one + without any delay. + + * Non-linear mode: Giving one domain a THRTL of 0xC and the other domain's 0 + means that the first domain gets 8% delay to access the cache and the other + one without any delay. + +For more detailed information please refer to Intel SDM chapter +"Introduction to Memory Bandwidth Allocation". + +In Xen's implementation, THRTL can be configured with libxl/xl interfaces but +COS is maintained in hypervisor only. The cache partition granularity is per +domain, each domain has COS=0 assigned by default, the corresponding THRTL is +0, which means all the cache resource can be accessed without delay. + +### xl interfaces + +System MBA information such as maximum COS and maximum THRTL can be obtained by: + +`xl psr-hwinfo --mba` + +The simplest way to change a domain's THRTL from its default is running: + +`xl psr-mba-set [OPTIONS] ` + +In a multi-socket system, the same thrtl will be set on each socket by default. +Per socket thrtl can be specified with the `--socket SOCKET` option. + +Setting the THRTL may not be successful if insufficient COS is available. In +such case unused COS(es) may be freed by setting THRTL of all related domains to +its default value(0). + +Per domain THRTL settings can be shown by: + +`xl psr-mba-show [OPTIONS] ` + +For linear mode, it shows the decimal v
[Xen-devel] [PATCH v8 08/16] x86: implement set value flow for MBA
This patch implements set value flow for MBA including its callback
function and domctl interface.
Signed-off-by: Yi Sun
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v8:
- restore some unnecessary changes in 'cat_check_cbm'.
(suggested by Jan Beulich)
- use 'fls()' but not 'flsl()'.
(suggested by Jan Beulich)
- use plain '=' to assign value for thrtl in 'mba_sanitize_thrtl'.
(suggested by Jan Beulich)
v7:
- change name of 'check_val' to 'sanitize'.
(suggested by Jan Beulich)
- fix comments.
(suggested by Jan Beulich)
- add parentheses and change '== 0' to '!'.
(suggested by Jan Beulich)
- remove unnecessary check of 'mba.thrtl_max'.
(suggested by Jan Beulich)
- remove unnecessary intermediate variable 'mod'.
(suggested by Jan Beulich)
- refine an assignement sentence to use '&='.
(suggested by Jan Beulich)
- change type of last parameter of 'sanitize' to 'uint32_t' and
apply same change to 'cat_check_cbm'.
(suggested by Jan Beulich)
v6:
- split co-exist features' values setting flow to a new patch.
(suggested by Jan Beulich)
- restore codes related to 'mba_check_thrtl' and 'check_value'.
(suggested by Jan Beulich)
v5:
- adjust position of 'cat_check_cbm' to not to make changes so big.
(suggested by Roger Pau Monné)
- remove 'props' from 'struct cos_write_info'.
(suggested by Roger Pau Monné)
- make a single return statement in 'mba_check_thrtl'.
(suggested by Jan Beulich)
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- join two checks into a single if.
(suggested by Roger Pau Monné)
- remove redundant local variable 'array_len'.
(suggested by Roger Pau Monné)
v3:
- modify commit message to make it clear.
(suggested by Roger Pau Monné)
- modify functionality of 'check_val' to make it simple to only check value.
Change the last parameter type from 'unsigned long *' to 'unsigned long'.
(suggested by Roger Pau Monné)
- call rdmsrl to get value just written into MSR for MBA. Because HW can
automatically change input value to what it wants.
(suggested by Roger Pau Monné)
- change type of 'write_msr' to 'uint32_t' to return the value actually
written into MSR. Then, change 'do_write_psr_msrs' to set the returned
value into 'cos_reg_val[]'
- move the declaration of 'j' into loop in 'do_write_psr_msrs'.
(suggested by Roger Pau Monné)
- change 'mba_info' to 'mba'.
(suggested by Roger Pau Monné)
- change 'cat_info' to 'cat'.
(suggested by Roger Pau Monné)
- rename 'psr_cat/PSR_CAT' to 'psr_alloc/PSR_ALLOC' and remove 'op/OP'
from name.
(suggested by Roger Pau Monné)
- change 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
v2:
- remove linear mode 'thrtl_max' check in 'mba_check_thrtl' because it has
been checked in 'mba_init_feature'.
(suggested by Chao Peng)
- for non-linear mode, check if '*thrtl' is not 0 in 'mba_check_thrtl'. If
it is 0, we do not need to change it.
(suggested by Chao Peng)
- move comments to explain changes of 'cos_write_info' from psr.c to commit
message.
(suggested by Chao Peng)
---
xen/arch/x86/domctl.c | 6 ++
xen/arch/x86/psr.c | 49 +
xen/include/public/domctl.h | 1 +
3 files changed, 52 insertions(+), 4 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index b726eae..4bca15d 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1465,6 +1465,12 @@ long arch_do_domctl(
PSR_TYPE_L2_CBM);
break;
+case XEN_DOMCTL_PSR_SET_MBA_THRTL:
+ret = psr_set_val(d, domctl->u.psr_alloc.target,
+ domctl->u.psr_alloc.data,
+ PSR_TYPE_MBA_THRTL);
+break;
+
#define domctl_psr_get_val(d, domctl, type, copyback) ({\
uint32_t v_;\
int r_ = psr_get_val((d), (domctl)->u.psr_alloc.target, \
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 549f21b..04dd4a1 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -138,6 +138,12 @@ static const struct feat_props {
/* write_msr is used t
[Xen-devel] [PATCH v8 00/16] Enable Memory Bandwidth Allocation in Xen
Hi, all, We plan to bring a new PSR (Platform Shared Resource) feature called Intel Memory Bandwidth Allocation (MBA) to Xen. Besides the MBA enabling, we change some interfaces to make them more general but not only for CAT. Any comments are welcome! You can find this series at: https://github.com/yisun-git/xen_mba mba_v8 This version bases on below pre-fix patch which has been merged into staging branch: "x86: psr: support co-exist features' values setting" https://lists.xen.org/archives/html/xen-devel/2017-10/msg00866.html CC: Jan Beulich CC: Andrew Cooper CC: Wei Liu CC: Ian Jackson CC: Daniel De Graaf CC: Roger Pau Monné CC: Konrad Rzeszutek Wilk CC: Chao Peng CC: Julien Grall --- Acked and Reviewed list before V8: a - Acked-by r - Reviewed-by r patch 1 - docs: create Memory Bandwidth Allocation (MBA) feature document ar patch 2 - Rename PSR sysctl/domctl interfaces and xsm policy to make them be general ar patch 3 - x86: rename 'cbm_type' to 'psr_type' to make it general ar patch 4 - x86: a few optimizations to psr codes r patch 5 - x86: implement data structure and CPU init flow for MBA ar patch 6 - x86: implement get hw info flow for MBA ar patch 7 - x86: implement get value interface for MBA ar patch 9 - tools: create general interfaces to support psr allocation features ar patch 10 - tools: implement the new libxc get hw info interface ar patch 11 - tools: implement the new libxl get hw info interface ar patch 12 - tools: implement the new xl get hw info interface ar patch 13 - tools: rename 'xc_psr_cat_type' to 'xc_psr_type' ar patch 14 - tools: implement new generic get value interface and MBA get value command ar patch 15 - tools: implement new generic set value interface and MBA set value command ar patch 16 - docs: add MBA description in docs --- V8 change list: Patch 5: - remove unnecessary line split. (suggested by Jan Beulich) - use MASK_EXTR(). (suggested by Jan Beulich) Patch 8: - restore some unnecessary changes in 'cat_check_cbm'. (suggested by Jan Beulich) - use 'fls()' but not 'flsl()'. (suggested by Jan Beulich) - use plain '=' to assign value for thrtl in 'mba_sanitize_thrtl'. (suggested by Jan Beulich) Yi Sun (16): docs: create Memory Bandwidth Allocation (MBA) feature document Rename PSR sysctl/domctl interfaces and xsm policy to make them be general x86: rename 'cbm_type' to 'psr_type' to make it general x86: a few optimizations to psr codes x86: implement data structure and CPU init flow for MBA x86: implement get hw info flow for MBA x86: implement get value interface for MBA x86: implement set value flow for MBA tools: create general interfaces to support psr allocation features tools: implement the new libxc get hw info interface tools: implement the new libxl get hw info interface tools: implement the new xl get hw info interface tools: rename 'xc_psr_cat_type' to 'xc_psr_type' tools: implement new generic get value interface and MBA get value command tools: implement new generic set value interface and MBA set value command docs: add MBA description in docs docs/features/intel_psr_mba.pandoc | 297 docs/man/xl.pod.1.in| 33 docs/misc/xl-psr.markdown | 62 tools/flask/policy/modules/dom0.te | 4 +- tools/libxc/include/xenctrl.h | 44 -- tools/libxc/xc_psr.c| 109 +++-- tools/libxl/libxl.h | 37 + tools/libxl/libxl_psr.c | 223 +-- tools/libxl/libxl_types.idl | 22 +++ tools/xl/xl.h | 2 + tools/xl/xl_cmdtable.c | 12 ++ tools/xl/xl_psr.c | 279 ++--- xen/arch/x86/domctl.c | 87 ++- xen/arch/x86/psr.c | 287 +- xen/arch/x86/sysctl.c | 57 --- xen/include/asm-x86/msr-index.h | 1 + xen/include/asm-x86/psr.h | 24 +-- xen/include/public/domctl.h | 26 ++-- xen/include/public/sysctl.h | 20 ++- xen/xsm/flask/hooks.c | 8 +- xen/xsm/flask/policy/access_vectors | 8 +- 21 files changed, 1317 insertions(+), 325 deletions(-) create mode 100644 docs/features/intel_psr_mba.pandoc -- 1.9.1 ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v8 15/16] tools: implement new generic set value interface and MBA set value command
This patch implements new generic set value interfaces in libxc and libxl.
These interfaces are suitable for all allocation features. It also adds a
new MBA set value command in xl.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Wei Liu
---
CC: Wei Liu
CC: Ian Jackson
CC: Roger Pau Monné
CC: Chao Peng
v5:
- move xc_type definition and value get out of the loop.
(suggested by Roger Pau Monné)
v4:
- remove 'ALLOC_' from macro name.
(suggested by Roger Pau Monné)
- adjust place of argc check and return EXIT_FAILURE.
(suggested by Roger Pau Monné)
- fix indentation issue.
(suggested by Roger Pau Monné)
- move same type local variables declaration to a single line.
(suggested by Roger Pau Monné)
v3:
- add 'const' for 'opts[]' in 'main_psr_mba_set'.
(suggested by Roger Pau Monné)
- replace 'libxl_psr_cbm_type' to 'libxl_psr_type' for newly defined
interfaces.
(suggested by Roger Pau Monné)
---
tools/libxc/include/xenctrl.h | 6 ++---
tools/libxc/xc_psr.c | 9 ---
tools/libxl/libxl_psr.c | 52
tools/xl/xl.h | 1 +
tools/xl/xl_cmdtable.c| 6 +
tools/xl/xl_psr.c | 55 +++
6 files changed, 96 insertions(+), 33 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index d046e61..6e238a2 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2532,9 +2532,9 @@ int xc_psr_cmt_get_data(xc_interface *xch, uint32_t rmid,
uint32_t cpu,
uint64_t *tsc);
int xc_psr_cmt_enabled(xc_interface *xch);
-int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_type type, uint32_t target,
- uint64_t data);
+int xc_psr_set_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_type type, uint32_t target,
+ uint64_t data);
int xc_psr_get_domain_data(xc_interface *xch, uint32_t domid,
xc_psr_type type, uint32_t target,
uint64_t *data);
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 191de97..1609185 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -248,9 +248,9 @@ int xc_psr_cmt_enabled(xc_interface *xch)
return 0;
}
-int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_type type, uint32_t target,
- uint64_t data)
+int xc_psr_set_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_type type, uint32_t target,
+ uint64_t data)
{
DECLARE_DOMCTL;
uint32_t cmd;
@@ -269,6 +269,9 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t
domid,
case XC_PSR_CAT_L2_CBM:
cmd = XEN_DOMCTL_PSR_SET_L2_CBM;
break;
+case XC_PSR_MBA_THRTL:
+cmd = XEN_DOMCTL_PSR_SET_MBA_THRTL;
+break;
default:
errno = EINVAL;
return -1;
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index 7c560bc..9ced7d1 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -328,32 +328,7 @@ int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
libxl_psr_cbm_type type, libxl_bitmap *target_map,
uint64_t cbm)
{
-GC_INIT(ctx);
-int rc;
-int socketid, nr_sockets;
-
-rc = libxl__count_physical_sockets(gc, &nr_sockets);
-if (rc) {
-LOGED(ERROR, domid, "failed to get system socket count");
-goto out;
-}
-
-libxl_for_each_set_bit(socketid, *target_map) {
-xc_psr_type xc_type = libxl__psr_type_to_libxc_psr_type(type);
-
-if (socketid >= nr_sockets)
-break;
-
-if (xc_psr_cat_set_domain_data(ctx->xch, domid, xc_type,
- socketid, cbm)) {
-libxl__psr_alloc_log_err_msg(gc, errno, type);
-rc = ERROR_FAIL;
-}
-}
-
-out:
-GC_FREE;
-return rc;
+return libxl_psr_set_val(ctx, domid, type, target_map, cbm);
}
int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
@@ -453,7 +428,30 @@ int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
libxl_psr_type type, libxl_bitmap *target_map,
uint64_t val)
{
-return ERROR_FAIL;
+GC_INIT(ctx);
+int rc, socketid, nr_sockets;
+xc_psr_type xc_type = libxl__psr_type_to_libxc_psr_type(type);
+
+rc = libxl__count_physical_sockets(gc, &nr_sockets);
+if (rc) {
+LOG(ERROR, "failed to get system socket count");
+goto out;
+}
+
+libxl_for_each_set_bit(s
[Xen-devel] [PATCH v8 10/16] tools: implement the new libxc get hw info interface
This patch implements a new libxc get hw info interface and corresponding
data structures. It also changes libxl_psr.c to call this new interface.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Wei Liu
---
CC: Wei Liu
CC: Ian Jackson
CC: Roger Pau Monné
CC: Chao Peng
v6:
- remove unnecessary spaces in brackets.
(suggested by Wei Liu)
- use assert to check input lvl.
(suggested by Roger Pau Monné)
v5:
- directly define 'xc_psr_hw_info' as union type.
(suggested by Roger Pau Monné)
- converge L2 and L3 cases in 'xc_psr_get_hw_info'.
(suggested by Roger Pau Monné)
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- remove 'XC_PSR_FEAT_UNKNOWN' which is not necessary.
(suggested by Roger Pau Monné)
- remove 'FEAT_' from enum item names.
(suggested by Roger Pau Monné)
- remove 'xc_' from struct name.
(suggested by Roger Pau Monné)
- adjust codes to reduce indentation.
(suggested by Roger Pau Monné)
- assert for not happened case.
(suggested by Roger Pau Monné)
- add LOGE to show errno.
(suggested by Roger Pau Monné)
v3:
- rename 'psr_cat/PSR_CAT' to 'psr_alloc/PSR_ALLOC' and remove 'op/OP'
from name.
(suggested by Roger Pau Monné)
- remove 'info' from 'xc_cat_info' and 'xc_mba_info'.
(suggested by Roger Pau Monné)
- set errno in 'xc_psr_get_hw_info'.
(suggested by Roger Pau Monné)
- remove 'inline'.
(suggested by Roger Pau Monné)
- remove 'psr' from 'libxl__psr_feat_type_to_libxc_psr_feat_type' to make
function name shorter.
(suggested by Roger Pau Monné)
- check 'xc_type' in 'libxl_psr_cat_get_info'.
(suggested by Roger Pau Monné)
v2:
- split this patch out from a big patch in v1.
(suggested by Wei Liu)
- change 'CAT_INFO' and 'MBA_INFO' to 'CAT' and 'MBA'.
(suggested by Chao Peng)
---
tools/libxc/include/xenctrl.h | 27 ++---
tools/libxc/xc_psr.c | 55 +++
tools/libxl/libxl_psr.c | 38 --
3 files changed, 95 insertions(+), 25 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 3bcab3c..8527389 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2494,6 +2494,28 @@ enum xc_psr_cat_type {
};
typedef enum xc_psr_cat_type xc_psr_cat_type;
+enum xc_psr_feat_type {
+XC_PSR_CAT_L3,
+XC_PSR_CAT_L2,
+XC_PSR_MBA,
+};
+typedef enum xc_psr_feat_type xc_psr_feat_type;
+
+union xc_psr_hw_info {
+struct {
+uint32_t cos_max;
+uint32_t cbm_len;
+bool cdp_enabled;
+} cat;
+
+struct {
+uint32_t cos_max;
+uint32_t thrtl_max;
+bool linear;
+} mba;
+};
+typedef union xc_psr_hw_info xc_psr_hw_info;
+
int xc_psr_cmt_attach(xc_interface *xch, uint32_t domid);
int xc_psr_cmt_detach(xc_interface *xch, uint32_t domid);
int xc_psr_cmt_get_domain_rmid(xc_interface *xch, uint32_t domid,
@@ -2515,9 +2537,8 @@ int xc_psr_cat_set_domain_data(xc_interface *xch,
uint32_t domid,
int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
xc_psr_cat_type type, uint32_t target,
uint64_t *data);
-int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
-uint32_t *cos_max, uint32_t *cbm_len,
-bool *cdp_enabled);
+int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
+ xc_psr_feat_type type, xc_psr_hw_info *hw_info);
int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 5c54a35..2c605a7 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -323,37 +323,52 @@ int xc_psr_cat_get_domain_data(xc_interface *xch,
uint32_t domid,
return rc;
}
-int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
-uint32_t *cos_max, uint32_t *cbm_len, bool
*cdp_enabled)
+int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
+ xc_psr_feat_type type, xc_psr_hw_info *hw_info)
{
int rc = -1;
DECLARE_SYSCTL;
+if ( !hw_info )
+{
+errno = EINVAL;
+return rc;
+}
+
sysctl.cmd = XEN_SYSCTL_psr_alloc;
sysctl.u.psr_alloc.target = socket;
-switch ( lvl )
+switch ( type )
{
-case 2:
-sysctl.u.psr_alloc.cmd = XEN_SYSCTL_PSR_get_l2_info;
+case XC_PSR_CAT_L2:
+case XC_PSR_CAT_L3:
+
[Xen-devel] [PATCH v8 09/16] tools: create general interfaces to support psr allocation features
This patch creates general interfaces in libxl to support all psr
allocation features.
Add 'LIBXL_HAVE_PSR_GENERIC' to indicate interface change.
Please note, the functionality cannot work until later patches
are applied.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Wei Liu
---
CC: Wei Liu
CC: Ian Jackson
CC: Roger Pau Monné
CC: Chao Peng
v5:
- adjust parameters position in 'libxl_psr_get_hw_info'.
(suggested by Roger Pau Monné)
v4:
- add description for LIBXL_HAVE_PSR_GENERIC to mention newly added
public functions.
(suggested by Roger Pau Monné)
v3:
- change 'LIBXL_HAVE_PSR_MBA' to 'LIBXL_HAVE_PSR_GENERIC'.
(suggested by Roger Pau Monné)
- 'typedef enum libxl_psr_cbm_type libxl_psr_type;' in libxl.h.
(suggested by Roger Pau Monné and Wei Liu)
- change 'libxl_psr_cbm_type' to 'libxl_psr_type' in newly defined
interfaces.
(suggested by Roger Pau Monné)
v2:
- remove '_INFO' in 'libxl_psr_feat_type' and make corresponding
changes in 'libxl_psr_hw_info'.
(suggested by Chao Peng)
---
tools/libxl/libxl.h | 37 +
tools/libxl/libxl_psr.c | 25 +
tools/libxl/libxl_types.idl | 22 ++
3 files changed, 84 insertions(+)
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index f82b91e..11c87aa 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -976,6 +976,17 @@ void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, const
libxl_mac *src);
#define LIBXL_HAVE_PSR_L2_CAT 1
/*
+ * LIBXL_HAVE_PSR_GENERIC
+ *
+ * If this is defined, the Memory Bandwidth Allocation feature is supported.
+ * The following public functions are available:
+ * libxl_psr_{set/get}_val
+ * libxl_psr_get_hw_info
+ * libxl_psr_hw_info_list_free
+ */
+#define LIBXL_HAVE_PSR_GENERIC 1
+
+/*
* LIBXL_HAVE_MCA_CAPS
*
* If this is defined, setting MCA capabilities for HVM domain is supported.
@@ -2296,6 +2307,32 @@ int libxl_psr_cat_get_info(libxl_ctx *ctx,
libxl_psr_cat_info **info,
int libxl_psr_cat_get_l3_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
int *nr);
void libxl_psr_cat_info_list_free(libxl_psr_cat_info *list, int nr);
+
+typedef enum libxl_psr_cbm_type libxl_psr_type;
+
+/*
+ * Function to set a domain's value. It operates on a single or multiple
+ * target(s) defined in 'target_map'. 'target_map' specifies all the sockets
+ * to be operated on.
+ */
+int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_type type, libxl_bitmap *target_map,
+ uint64_t val);
+/*
+ * Function to get a domain's cbm. It operates on a single 'target'.
+ * 'target' specifies which socket to be operated on.
+ */
+int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_type type, unsigned int target,
+ uint64_t *val);
+/*
+ * On success, the function returns an array of elements in 'info',
+ * and the length in 'nr'.
+ */
+int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_feat_type type,
+ unsigned int lvl, unsigned int *nr,
+ libxl_psr_hw_info **info);
+void libxl_psr_hw_info_list_free(libxl_psr_hw_info *list, unsigned int nr);
#endif
/* misc */
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index 197505a..d4f5f67 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -428,6 +428,31 @@ void libxl_psr_cat_info_list_free(libxl_psr_cat_info
*list, int nr)
free(list);
}
+int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_type type, libxl_bitmap *target_map,
+ uint64_t val)
+{
+return ERROR_FAIL;
+}
+
+int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_type type, unsigned int target,
+ uint64_t *val)
+{
+return ERROR_FAIL;
+}
+
+int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_feat_type type,
+ unsigned int lvl, unsigned int *nr,
+ libxl_psr_hw_info **info)
+{
+return ERROR_FAIL;
+}
+
+void libxl_psr_hw_info_list_free(libxl_psr_hw_info *list, unsigned int nr)
+{
+}
+
/*
* Local variables:
* mode: C
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index 2d0bb8a..b05f10b 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -1032,6 +1032,7 @@ libxl_psr_cbm_type = Enumeration("psr_cbm_type", [
(2, "L3_CBM_CODE"),
(3, "L3_CBM_DATA"),
(4, "L2_CBM"),
+(5, "MBA_THRTL"),
])
libxl_psr_cat_info = Struct("psr_cat_info", [
@@ -1040,3 +1041,24
[Xen-devel] [PATCH v8 12/16] tools: implement the new xl get hw info interface
This patch implements a new xl get HW info interface. A new argument
is added for psr-hwinfo command to get and show MBA HW info.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Wei Liu
---
CC: Wei Liu
CC: Ian Jackson
CC: Roger Pau Monné
CC: Chao Peng
v3:
- change the format string of printf in 'psr_mba_hwinfo'.
(suggested by Roger Pau Monné)
- add 'const' for 'opts[]' in 'main_psr_hwinfo'.
(suggested by Roger Pau Monné)
v2:
- split out this patch from a big patch in v1.
(suggested by Wei Liu)
- change 'MBA_INFO' to 'MBA'. Also, change 'mba_info' to 'mba'.
(suggested by Chao Peng)
---
tools/xl/xl_cmdtable.c | 1 +
tools/xl/xl_psr.c | 39 ---
2 files changed, 37 insertions(+), 3 deletions(-)
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index 68a8a72..9e8f704 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -549,6 +549,7 @@ struct cmd_spec cmd_table[] = {
"[options]",
"-m, --cmt Show Cache Monitoring Technology (CMT) hardware info\n"
"-a, --cat Show Cache Allocation Technology (CAT) hardware info\n"
+ "-b, --mba Show Memory Bandwidth Allocation (MBA) hardware info\n"
},
{ "psr-cmt-attach",
&main_psr_cmt_attach, 0, 1,
diff --git a/tools/xl/xl_psr.c b/tools/xl/xl_psr.c
index ef00048..ab47d96 100644
--- a/tools/xl/xl_psr.c
+++ b/tools/xl/xl_psr.c
@@ -475,6 +475,31 @@ static int psr_l2_cat_hwinfo(void)
return rc;
}
+static int psr_mba_hwinfo(void)
+{
+int rc;
+unsigned int i, nr;
+libxl_psr_hw_info *info;
+
+rc = libxl_psr_get_hw_info(ctx, LIBXL_PSR_FEAT_TYPE_MBA, 0, &nr, &info);
+if (rc)
+return rc;
+
+printf("Memory Bandwidth Allocation (MBA):\n");
+
+for (i = 0; i < nr; i++) {
+printf("Socket ID : %u\n", info[i].id);
+printf("Linear Mode : %s\n",
+ info[i].u.mba.linear ? "Enabled" : "Disabled");
+printf("Maximum COS : %u\n", info[i].u.mba.cos_max);
+printf("Maximum Throttling Value: %u\n", info[i].u.mba.thrtl_max);
+printf("Default Throttling Value: %u\n", 0);
+}
+
+libxl_psr_hw_info_list_free(info, nr);
+return rc;
+}
+
int main_psr_cat_cbm_set(int argc, char **argv)
{
uint32_t domid;
@@ -593,20 +618,24 @@ int main_psr_cat_show(int argc, char **argv)
int main_psr_hwinfo(int argc, char **argv)
{
int opt, ret = 0;
-bool all = true, cmt = false, cat = false;
-static struct option opts[] = {
+bool all = true, cmt = false, cat = false, mba = false;
+static const struct option opts[] = {
{"cmt", 0, 0, 'm'},
{"cat", 0, 0, 'a'},
+{"mba", 0, 0, 'b'},
COMMON_LONG_OPTS
};
-SWITCH_FOREACH_OPT(opt, "ma", opts, "psr-hwinfo", 0) {
+SWITCH_FOREACH_OPT(opt, "mab", opts, "psr-hwinfo", 0) {
case 'm':
all = false; cmt = true;
break;
case 'a':
all = false; cat = true;
break;
+case 'b':
+all = false; mba = true;
+break;
}
if (!ret && (all || cmt))
@@ -619,6 +648,10 @@ int main_psr_hwinfo(int argc, char **argv)
if (all || cat)
ret = psr_l2_cat_hwinfo();
+/* MBA is independent of CMT and CAT */
+if (all || mba)
+ret = psr_mba_hwinfo();
+
return ret;
}
--
1.9.1
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v8 14/16] tools: implement new generic get value interface and MBA get value command
This patch implements generic get value interfaces in libxc and libxl.
It also refactors the get value flow in xl to make it be suitable for all
allocation features. Based on that, a new MBA get value command is added in xl.
Signed-off-by: Yi Sun
Acked-by: Wei Liu
Reviewed-by: Roger Pau Monné
---
CC: Ian Jackson
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v6:
- fix one coding style issue.
(suggested by Roger Pau Monné)
v5:
- start a newline for "CDP" because it exceeds 80 characters.
(suggested by Roger Pau Monné)
- remove a duplicated ';'.
(suggested by Roger Pau Monné)
- remove a extra newline.
(suggested by Roger Pau Monné)
- correct words in log message.
(suggested by Roger Pau Monné)
v4:
- use designated initializers for 'feat_name[]'.
(suggested by Roger Pau Monné)
- use LOG in 'libxl__psr_alloc_log_err_msg'.
(suggested by Roger Pau Monné)
v3:
- replace 'libxl_psr_cbm_type' to 'libxl_psr_type' in newly defined
interfaces.
(suggested by Roger Pau Monné)
v2:
- change 'CAT_INFO'/'MBA_INFO' to 'CAT'/'MBA'. The related structure names
are changed too.
(suggested by Chao Peng)
---
tools/libxc/include/xenctrl.h | 7 +-
tools/libxc/xc_psr.c | 9 +-
tools/libxl/libxl_psr.c | 58 -
tools/xl/xl.h | 1 +
tools/xl/xl_cmdtable.c| 5 ++
tools/xl/xl_psr.c | 185 ++
6 files changed, 183 insertions(+), 82 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 29fcb95..d046e61 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2491,6 +2491,7 @@ enum xc_psr_type {
XC_PSR_CAT_L3_CBM_CODE = 2,
XC_PSR_CAT_L3_CBM_DATA = 3,
XC_PSR_CAT_L2_CBM = 4,
+XC_PSR_MBA_THRTL = 5,
};
typedef enum xc_psr_type xc_psr_type;
@@ -2534,9 +2535,9 @@ int xc_psr_cmt_enabled(xc_interface *xch);
int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
xc_psr_type type, uint32_t target,
uint64_t data);
-int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_type type, uint32_t target,
- uint64_t *data);
+int xc_psr_get_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_type type, uint32_t target,
+ uint64_t *data);
int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
xc_psr_feat_type type, xc_psr_hw_info *hw_info);
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 01f4ba7..191de97 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -283,9 +283,9 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t
domid,
return do_domctl(xch, &domctl);
}
-int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_type type, uint32_t target,
- uint64_t *data)
+int xc_psr_get_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_type type, uint32_t target,
+ uint64_t *data)
{
int rc;
DECLARE_DOMCTL;
@@ -305,6 +305,9 @@ int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t
domid,
case XC_PSR_CAT_L2_CBM:
cmd = XEN_DOMCTL_PSR_GET_L2_CBM;
break;
+case XC_PSR_MBA_THRTL:
+cmd = XEN_DOMCTL_PSR_GET_MBA_THRTL;
+break;
default:
errno = EINVAL;
return -1;
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index c54cb6f..7c560bc 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -71,16 +71,30 @@ static void libxl__psr_cmt_log_err_msg(libxl__gc *gc, int
err)
LOGE(ERROR, "%s", msg);
}
-static void libxl__psr_cat_log_err_msg(libxl__gc *gc, int err)
+static void libxl__psr_alloc_log_err_msg(libxl__gc *gc,
+ int err,
+ libxl_psr_type type)
{
+/*
+ * Index is 'libxl_psr_type' so we set two 'CDP' to correspond to
+ * DATA and CODE.
+ */
+const char * const feat_name[] = {
+[LIBXL_PSR_CBM_TYPE_UNKNOWN] = "UNKNOWN",
+[LIBXL_PSR_CBM_TYPE_L3_CBM] = "L3 CAT",
+[LIBXL_PSR_CBM_TYPE_L3_CBM_CODE...LIBXL_PSR_CBM_TYPE_L3_CBM_DATA] =
+ "CDP",
+[LIBXL_PSR_CBM_TYPE_L2_CBM] = "L2 CAT",
+[LIBXL_PSR_CBM_TYPE_MBA_THRTL] = "MBA",
+};
char *msg;
switch (err) {
case ENODEV:
-msg = "CAT is not supported in this system";
+msg = "is not supported
[Xen-devel] [PATCH v8 13/16] tools: rename 'xc_psr_cat_type' to 'xc_psr_type'
This patch renames 'xc_psr_cat_type' to 'xc_psr_type' so that
the structure name is common for all allocation features.
Signed-off-by: Yi Sun
Acked-by: Wei Liu
Reviewed-by: Chao Peng
Reviewed-by: Roger Pau Monné
---
CC: Ian Jackson
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v5:
- remove a duplicated ';'.
(suggested by Roger Pau Monné)
v4:
- move assignment of xc_type to its declaration place.
(suggested by Roger Pau Monné)
v3:
- change 'xc_psr_val_type' to 'xc_psr_type'.
(suggested by Roger Pau Monné)
---
tools/libxc/include/xenctrl.h | 8
tools/libxc/xc_psr.c | 4 ++--
tools/libxl/libxl_psr.c | 11 +--
3 files changed, 11 insertions(+), 12 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 8527389..29fcb95 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2486,13 +2486,13 @@ enum xc_psr_cmt_type {
};
typedef enum xc_psr_cmt_type xc_psr_cmt_type;
-enum xc_psr_cat_type {
+enum xc_psr_type {
XC_PSR_CAT_L3_CBM = 1,
XC_PSR_CAT_L3_CBM_CODE = 2,
XC_PSR_CAT_L3_CBM_DATA = 3,
XC_PSR_CAT_L2_CBM = 4,
};
-typedef enum xc_psr_cat_type xc_psr_cat_type;
+typedef enum xc_psr_type xc_psr_type;
enum xc_psr_feat_type {
XC_PSR_CAT_L3,
@@ -2532,10 +2532,10 @@ int xc_psr_cmt_get_data(xc_interface *xch, uint32_t
rmid, uint32_t cpu,
int xc_psr_cmt_enabled(xc_interface *xch);
int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_type type, uint32_t target,
uint64_t data);
int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_type type, uint32_t target,
uint64_t *data);
int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
xc_psr_feat_type type, xc_psr_hw_info *hw_info);
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 2c605a7..01f4ba7 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -249,7 +249,7 @@ int xc_psr_cmt_enabled(xc_interface *xch)
return 0;
}
int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_type type, uint32_t target,
uint64_t data)
{
DECLARE_DOMCTL;
@@ -284,7 +284,7 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t
domid,
}
int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_type type, uint32_t target,
uint64_t *data)
{
int rc;
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index b053abd..c54cb6f 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -303,11 +303,11 @@ out:
return rc;
}
-static inline xc_psr_cat_type libxl__psr_cbm_type_to_libxc_psr_cat_type(
+static inline xc_psr_type libxl__psr_cbm_type_to_libxc_psr_type(
libxl_psr_cbm_type type)
{
-BUILD_BUG_ON(sizeof(libxl_psr_cbm_type) != sizeof(xc_psr_cat_type));
-return (xc_psr_cat_type)type;
+BUILD_BUG_ON(sizeof(libxl_psr_cbm_type) != sizeof(xc_psr_type));
+return (xc_psr_type)type;
}
int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
@@ -325,12 +325,11 @@ int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
}
libxl_for_each_set_bit(socketid, *target_map) {
-xc_psr_cat_type xc_type;
+xc_psr_type xc_type = libxl__psr_cbm_type_to_libxc_psr_type(type);
if (socketid >= nr_sockets)
break;
-xc_type = libxl__psr_cbm_type_to_libxc_psr_cat_type(type);
if (xc_psr_cat_set_domain_data(ctx->xch, domid, xc_type,
socketid, cbm)) {
libxl__psr_cat_log_err_msg(gc, errno);
@@ -349,7 +348,7 @@ int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
{
GC_INIT(ctx);
int rc = 0;
-xc_psr_cat_type xc_type = libxl__psr_cbm_type_to_libxc_psr_cat_type(type);
+xc_psr_type xc_type = libxl__psr_cbm_type_to_libxc_psr_type(type);
if (xc_psr_cat_get_domain_data(ctx->xch, domid, xc_type,
target, cbm_r)) {
--
1.9.1
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v8 06/16] x86: implement get hw info flow for MBA
This patch implements get HW info flow for MBA including its callback
function and sysctl interface.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Jan Beulich
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v7:
- change 'PSR_INFO_IDX_MBA_FLAG' to 'PSR_INFO_IDX_MBA_FLAGS'.
(suggested by Jan Beulich)
v5:
- use ASSERT in 'mba_get_feat_info'.
(suggested by Roger Pau Monné)
- correct initialization format of 'data[PSR_INFO_ARRAY_SIZE]'.
(suggested by Roger Pau Monné and Jan Beulich)
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- initialize 'data[PSR_INFO_ARRAY_SIZE]' to 0 to prevent to leak stack data.
(suggested by Roger Pau Monné)
v3:
- replace 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
v2:
- use 'XEN_SYSCTL_PSR_MBA_LINEAR' to set MBA feature HW info.
(suggested by Chao Peng)
v1:
- sort 'PSR_INFO_IDX_' macros as feature.
(suggested by Chao Peng)
- rename 'PSR_INFO_IDX_MBA_LINEAR' to 'PSR_INFO_IDX_MBA_FLAG'.
- rename 'linear' in 'struct mba_info' to 'flags' for future extension.
(suggested by Chao Peng)
---
xen/arch/x86/psr.c | 14 +-
xen/arch/x86/sysctl.c | 21 -
xen/include/asm-x86/psr.h | 2 ++
xen/include/public/sysctl.h | 8
4 files changed, 43 insertions(+), 2 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 3207988..549f21b 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -263,6 +263,10 @@ static enum psr_feat_type psr_type_to_feat_type(enum
psr_type type)
feat_type = FEAT_TYPE_L2_CAT;
break;
+case PSR_TYPE_MBA_THRTL:
+feat_type = FEAT_TYPE_MBA;
+break;
+
default:
ASSERT_UNREACHABLE();
}
@@ -481,7 +485,15 @@ static const struct feat_props l2_cat_props = {
static bool mba_get_feat_info(const struct feat_node *feat,
uint32_t data[], unsigned int array_len)
{
-return false;
+ASSERT(array_len == PSR_INFO_ARRAY_SIZE);
+
+data[PSR_INFO_IDX_COS_MAX] = feat->cos_max;
+data[PSR_INFO_IDX_MBA_THRTL_MAX] = feat->mba.thrtl_max;
+
+if ( feat->mba.linear )
+data[PSR_INFO_IDX_MBA_FLAGS] |= XEN_SYSCTL_PSR_MBA_LINEAR;
+
+return true;
}
static void mba_write_msr(unsigned int cos, uint32_t val,
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index 6d48cac..ffad585 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -174,7 +174,7 @@ long arch_do_sysctl(
case XEN_SYSCTL_psr_alloc:
switch ( sysctl->u.psr_alloc.cmd )
{
-uint32_t data[PSR_INFO_ARRAY_SIZE];
+uint32_t data[PSR_INFO_ARRAY_SIZE] = { };
case XEN_SYSCTL_PSR_get_l3_info:
{
@@ -214,6 +214,25 @@ long arch_do_sysctl(
break;
}
+case XEN_SYSCTL_PSR_get_mba_info:
+{
+ret = psr_get_info(sysctl->u.psr_alloc.target,
+ PSR_TYPE_MBA_THRTL, data, ARRAY_SIZE(data));
+if ( ret )
+break;
+
+sysctl->u.psr_alloc.u.mba_info.cos_max =
+ data[PSR_INFO_IDX_COS_MAX];
+sysctl->u.psr_alloc.u.mba_info.thrtl_max =
+ data[PSR_INFO_IDX_MBA_THRTL_MAX];
+sysctl->u.psr_alloc.u.mba_info.flags =
+ data[PSR_INFO_IDX_MBA_FLAGS];
+
+if ( __copy_field_to_guest(u_sysctl, sysctl, u.psr_alloc) )
+ret = -EFAULT;
+break;
+}
+
default:
ret = -EOPNOTSUPP;
break;
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index 3cf544a..c2257da 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -39,6 +39,8 @@
#define PSR_INFO_IDX_COS_MAX0
#define PSR_INFO_IDX_CAT_CBM_LEN1
#define PSR_INFO_IDX_CAT_FLAGS 2
+#define PSR_INFO_IDX_MBA_THRTL_MAX 1
+#define PSR_INFO_IDX_MBA_FLAGS 2
#define PSR_INFO_ARRAY_SIZE 3
struct psr_cmt_l3 {
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index a50e345..f7f26c3 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -698,6 +698,7 @@ struct xen_sysctl_pcitopoinfo {
#define XEN_SYSCTL_PSR_get_l3_info 0
#define XEN_SYSCTL_PSR_get_l2_info 1
+#define XEN_SYSCTL_PSR_get_mba_info 2
struct xen_sysctl_psr_alloc {
uint32_t cmd; /* IN: XEN_SYSCTL_PSR_* */
uint32_t target;/* IN */
@@ -708,6 +709,13 @@ struct xen_sysctl_psr_alloc {
#define XEN_SYSCTL_PSR_CAT_L
[Xen-devel] [PATCH v8 01/16] docs: create Memory Bandwidth Allocation (MBA) feature document
This patch creates MBA feature document in doc/features/. It describes key points to implement MBA which is described in details in Intel SDM "Introduction to Memory Bandwidth Allocation". Signed-off-by: Yi Sun Reviewed-by: Roger Pau Monné --- CC: Jan Beulich CC: Andrew Cooper CC: Wei Liu CC: Ian Jackson CC: Daniel De Graaf CC: Roger Pau Monné CC: Konrad Rzeszutek Wilk CC: Chao Peng CC: Julien Grall v6: - fix some words. (suggested by Roger Pau Monné) v5: - correct some words. (suggested by Roger Pau Monné) - change 'xl psr-mba-set 1 0xa' to 'xl psr-mba-set 1 10'. (suggested by Roger Pau Monné) v4: - add 'domain-name' as parameter of 'psr-mba-show/psr-mba-set'. (suggested by Roger Pau Monné) - fix some wordings. (suggested by Roger Pau Monné) - explain how user can know the MBA_MAX. (suggested by Roger Pau Monné) - move the description of 'Linear mode/Non-linear mode' into section of 'psr-mba-show'. (suggested by Roger Pau Monné) - change 'per-thread' to 'per-hyper-thread' to make it clearer. (suggested by Roger Pau Monné) - upgrade revision number. v3: - remove 'closed-loop' related description. (suggested by Roger Pau Monné) - explain 'linear' and 'non-linear' before mentioning them. (suggested by Roger Pau Monné) - adjust desription of 'psr-mba-set'. (suggested by Roger Pau Monné) - explain 'MBA_MAX'. (suggested by Roger Pau Monné) - remove 'n<64'. (suggested by Roger Pau Monné) - fix some wordings. (suggested by Roger Pau Monné) - add context in 'Testing' part to make things more clear. (suggested by Roger Pau Monné) v2: - declare 'HW' in Terminology. (suggested by Chao Peng) - replace 'COS ID of VCPU' to 'COS ID of domain'. (suggested by Chao Peng) - replace 'COS register' to 'Thrtl MSR'. (suggested by Chao Peng) - add description for 'psr-mba-show' to state that the decimal value is shown for linear mode but hexadecimal value is shown for non-linear mode. (suggested by Chao Peng) - remove content in 'Areas for improvement'. (suggested by Chao Peng) - use '<>' to specify mandatory argument to a command. (suggested by Wei Liu) v1: - remove a special character to avoid the error when building pandoc. --- docs/features/intel_psr_mba.pandoc | 297 + 1 file changed, 297 insertions(+) create mode 100644 docs/features/intel_psr_mba.pandoc diff --git a/docs/features/intel_psr_mba.pandoc b/docs/features/intel_psr_mba.pandoc new file mode 100644 index 000..86df661 --- /dev/null +++ b/docs/features/intel_psr_mba.pandoc @@ -0,0 +1,297 @@ +% Intel Memory Bandwidth Allocation (MBA) Feature +% Revision 1.8 + +\clearpage + +# Basics + + + Status: **Tech Preview** + +Architecture(s): Intel x86 + + Component(s): Hypervisor, toolstack + + Hardware: MBA is supported on Skylake Server and beyond + + +# Terminology + +* CAT Cache Allocation Technology +* CBM Capacity BitMasks +* CDP Code and Data Prioritization +* COS/CLOSClass of Service +* HW Hardware +* MBA Memory Bandwidth Allocation +* MSRsMachine Specific Registers +* PSR Intel Platform Shared Resource +* THRTL Throttle value or delay value + +# Overview + +The Memory Bandwidth Allocation (MBA) feature provides indirect and approximate +control over memory bandwidth available per-core. This feature provides OS/ +hypervisor the ability to slow misbehaving apps/domains by using a credit-based +throttling mechanism. + +# User details + +* Feature Enabling: + + Add "psr=mba" to boot line parameter to enable MBA feature. + +* xl interfaces: + + 1. `psr-mba-show [domain-id|domain-name]`: + + Show memory bandwidth throttling for domain. Under different modes, it + shows different type of data. + + There are two modes: + Linear mode: the input precision is defined as 100-(MBA_MAX). For instance, + if the MBA_MAX value is 90, the input precision is 10%. Values not an even + multiple of the precision (e.g., 12%) will be rounded down (e.g., to 10% + delay applied) by HW automatically. The response of throttling value is + linear. + + Non-linear mode: input delay values are powers-of-two from zero to the + MBA_MAX value from CPUID. In this case any values not a power of two will + be rounded down the next nearest power of two by HW automatically. The + response of throttli
[Xen-devel] [PATCH v8 02/16] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
This patch renames PSR sysctl/domctl interfaces and related xsm policy to
make them be general for all resource allocation features but not only
for CAT. Then, we can resuse the interfaces for all allocation features.
Basically, it changes 'psr_cat_op' to 'psr_alloc', and remove 'CAT_' from some
macros. E.g.:
1. psr_cat_op -> psr_alloc
2. XEN_DOMCTL_psr_cat_op -> XEN_DOMCTL_psr_alloc
3. XEN_SYSCTL_psr_cat_op -> XEN_SYSCTL_psr_alloc
4. XEN_DOMCTL_PSR_CAT_SET_L3_CBM -> XEN_DOMCTL_PSR_SET_L3_CBM
5. XEN_SYSCTL_PSR_CAT_get_l3_info -> XEN_SYSCTL_PSR_get_l3_info
Signed-off-by: Yi Sun
Reviewed-by: Wei Liu
Reviewed-by: Roger Pau Monné
Acked-by: Jan Beulich
Acked-by: Daniel De Graaf
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Ian Jackson
CC: Daniel De Graaf
CC: Roger Pau Monné
CC: Chao Peng
v7:
- add single trailing underscore for internal variabled in macro.
(suggested by Jan Beulich)
- add parentheses for input parameters of marcro.
(suggested by Jan Beulich)
- adjust the postion of macro.
(suggested by Jan Beulich)
v6:
- move macro definition into the function and undefine it after use.
(suggested by Roger Pau Monné)
- do not bump sysctl version because it has been bumped for 4.10.
(suggested by Roger Pau Monné)
v5:
- remove domctl version number upgrade.
(suggested by Jan Beulich)
- restore 'XEN_SYSCTL_PSR_CAT_L3_CDP'.
(suggested by Jan Beulich)
- define a local macro to complete psr get value flow.
(suggested by Roger Pau Monné)
- remove 'Reviewed-by' and 'Acked-by'.
(suggested by Wei Liu)
v4:
- remove 'ALLOC_' from names.
(suggested by Roger Pau Monné)
- fix comments.
(suggested by Roger Pau Monné)
v3:
- remove 'op/OP' from names and modify some names from 'PSR_CAT' to
'PSR_ALLOC'.
(suggested by Roger Pau Monné)
v1:
- add description about what to be changed in commit message.
(suggested by Wei Liu)
- bump sysctl/domctl version numbers.
(suggested by Wei Liu)
---
tools/flask/policy/modules/dom0.te | 4 +--
tools/libxc/xc_psr.c| 50 +-
xen/arch/x86/domctl.c | 71 ++---
xen/arch/x86/sysctl.c | 28 +++
xen/include/public/domctl.h | 24 ++---
xen/include/public/sysctl.h | 12 +++
xen/xsm/flask/hooks.c | 8 ++---
xen/xsm/flask/policy/access_vectors | 8 ++---
8 files changed, 102 insertions(+), 103 deletions(-)
diff --git a/tools/flask/policy/modules/dom0.te
b/tools/flask/policy/modules/dom0.te
index 1643b40..07de3d5 100644
--- a/tools/flask/policy/modules/dom0.te
+++ b/tools/flask/policy/modules/dom0.te
@@ -14,7 +14,7 @@ allow dom0_t xen_t:xen {
tmem_control getscheduler setscheduler
};
allow dom0_t xen_t:xen2 {
- resource_op psr_cmt_op psr_cat_op pmu_ctrl get_symbol
+ resource_op psr_cmt_op psr_alloc pmu_ctrl get_symbol
get_cpu_levelling_caps get_cpu_featureset livepatch_op
gcov_op set_parameter
};
@@ -39,7 +39,7 @@ allow dom0_t dom0_t:domain {
};
allow dom0_t dom0_t:domain2 {
set_cpuid gettsc settsc setscheduler set_max_evtchn set_vnumainfo
- get_vnumainfo psr_cmt_op psr_cat_op set_gnttab_limits
+ get_vnumainfo psr_cmt_op psr_alloc set_gnttab_limits
};
allow dom0_t dom0_t:resource { add remove };
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 039b920..5c54a35 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -258,27 +258,27 @@ int xc_psr_cat_set_domain_data(xc_interface *xch,
uint32_t domid,
switch ( type )
{
case XC_PSR_CAT_L3_CBM:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM;
+cmd = XEN_DOMCTL_PSR_SET_L3_CBM;
break;
case XC_PSR_CAT_L3_CBM_CODE:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_CODE;
+cmd = XEN_DOMCTL_PSR_SET_L3_CODE;
break;
case XC_PSR_CAT_L3_CBM_DATA:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_DATA;
+cmd = XEN_DOMCTL_PSR_SET_L3_DATA;
break;
case XC_PSR_CAT_L2_CBM:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM;
+cmd = XEN_DOMCTL_PSR_SET_L2_CBM;
break;
default:
errno = EINVAL;
return -1;
}
-domctl.cmd = XEN_DOMCTL_psr_cat_op;
+domctl.cmd = XEN_DOMCTL_psr_alloc;
domctl.domain = (domid_t)domid;
-domctl.u.psr_cat_op.cmd = cmd;
-domctl.u.psr_cat_op.target = target;
-domctl.u.psr_cat_op.data = data;
+domctl.u.psr_alloc.cmd = cmd;
+domctl.u.psr_alloc.target = target;
+domctl.u.psr_alloc.data = data;
return do_domctl(xch, &domctl);
}
@@ -294,31 +294,31 @@ int xc_psr_cat_get_domain_data(xc_interface *xch,
uint32_t domid,
switch ( type )
{
case XC_PSR_CAT_L3_CB
[Xen-devel] [PATCH v8 04/16] x86: a few optimizations to psr codes
This patch refines psr codes:
1. Change type of 'cat_init_feature' to 'bool' to remove the pointless
returning of error code.
2. Move printk in 'cat_init_feature' to reduce a return path.
3. Define a local variable 'feat_mask' in 'psr_cpu_init' to reduce calling of
'cpuid_count_leaf()'.
4. Change 'PSR_INFO_IDX_CAT_FLAG' to 'PSR_INFO_IDX_CAT_FLAGS'.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Jan Beulich
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v7:
- adjust the check to 'cat_init_feature' in 'psr_cpu_init'.
(suggested by Jan Beulich)
- change 'PSR_INFO_IDX_CAT_FLAG' to 'PSR_INFO_IDX_CAT_FLAGS'.
(suggested by Jan Beulich)
v6:
- restore 'write_msr()' type to 'void'.
(suggested by Jan Beulich and Roger Pau Monné)
- change 'ebx' in 'psr_cpu_init' to 'feat_mask'.
(suggested by Jan Beulich and Roger Pau Monné)
v5:
- create this patch to make codes clearer.
(suggested by Jan Beulich and Roger Pau Monné)
---
xen/arch/x86/psr.c| 45 ++---
xen/arch/x86/sysctl.c | 4 ++--
xen/include/asm-x86/psr.h | 2 +-
3 files changed, 25 insertions(+), 26 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 6dce823..50c5a98 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -273,10 +273,10 @@ static bool psr_check_cbm(unsigned int cbm_len, unsigned
long cbm)
}
/* CAT common functions implementation. */
-static int cat_init_feature(const struct cpuid_leaf *regs,
-struct feat_node *feat,
-struct psr_socket_info *info,
-enum psr_feat_type type)
+static bool cat_init_feature(const struct cpuid_leaf *regs,
+ struct feat_node *feat,
+ struct psr_socket_info *info,
+ enum psr_feat_type type)
{
const char *const cat_feat_name[FEAT_TYPE_NUM] = {
[FEAT_TYPE_L3_CAT] = "L3 CAT",
@@ -286,7 +286,7 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
/* No valid value so do not enable feature. */
if ( !regs->a || !regs->d )
-return -ENOENT;
+return false;
feat->cbm_len = (regs->a & CAT_CBM_LEN_MASK) + 1;
feat->cos_max = min(opt_cos_max, regs->d & CAT_COS_MAX_MASK);
@@ -296,7 +296,7 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
case FEAT_TYPE_L3_CAT:
case FEAT_TYPE_L2_CAT:
if ( feat->cos_max < 1 )
-return -ENOENT;
+return false;
/* We reserve cos=0 as default cbm (all bits within cbm_len are 1). */
feat->cos_reg_val[0] = cat_default_val(feat->cbm_len);
@@ -313,7 +313,7 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
uint64_t val;
if ( feat->cos_max < 3 )
-return -ENOENT;
+return false;
/* Cut half of cos_max when CDP is enabled. */
feat->cos_max = (feat->cos_max - 1) >> 1;
@@ -332,20 +332,18 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
}
default:
-return -ENOENT;
+return false;
}
/* Add this feature into array. */
info->features[type] = feat;
-if ( !opt_cpu_info )
-return 0;
-
-printk(XENLOG_INFO "%s: enabled on socket %u, cos_max:%u, cbm_len:%u\n",
- cat_feat_name[type], cpu_to_socket(smp_processor_id()),
- feat->cos_max, feat->cbm_len);
+if ( opt_cpu_info )
+printk(XENLOG_INFO "%s: enabled on socket %u, cos_max:%u,
cbm_len:%u\n",
+ cat_feat_name[type], cpu_to_socket(smp_processor_id()),
+ feat->cos_max, feat->cbm_len);
-return 0;
+return true;
}
static bool cat_get_feat_info(const struct feat_node *feat,
@@ -356,7 +354,7 @@ static bool cat_get_feat_info(const struct feat_node *feat,
data[PSR_INFO_IDX_COS_MAX] = feat->cos_max;
data[PSR_INFO_IDX_CAT_CBM_LEN] = feat->cbm_len;
-data[PSR_INFO_IDX_CAT_FLAG] = 0;
+data[PSR_INFO_IDX_CAT_FLAGS] = 0;
return true;
}
@@ -383,7 +381,7 @@ static bool l3_cdp_get_feat_info(const struct feat_node
*feat,
if ( !cat_get_feat_info(feat, data, array_len) )
return false;
-data[PSR_INFO_IDX_CAT_FLAG] |= XEN_SYSCTL_PSR_CAT_L3_CDP;
+data[PSR_INFO_IDX_CAT_FLAGS] |= XEN_SYSCTL_PSR_CAT_L3_CDP;
return true;
}
@@ -1413,6 +1411,7 @@ static void psr_cpu_init(void)
unsigned int socket, cpu = smp_processor_id();
struct feat_node *feat;
struct cpuid_leaf regs;
+uint32_t feat_mask;
if ( !psr_alloc_feat_enabled() || !boot_cpu_ha
[Xen-devel] [PATCH v8 05/16] x86: implement data structure and CPU init flow for MBA
This patch implements main data structures of MBA. Like CAT features, MBA HW info has cos_max which means the max thrtl register number, and thrtl_max which means the max throttle value (delay value). It also has a flag to represent if the throttle value is linear or non-linear. One thrtl register of MBA stores a throttle value for one or more domains. The throttle value means the delay applied to traffic between L2 cache and next cache level. This patch also implements init flow for MBA and register stub callback functions. Signed-off-by: Yi Sun Reviewed-by: Roger Pau Monné Reviewed-by: Jan Beulich --- CC: Jan Beulich CC: Andrew Cooper CC: Wei Liu CC: Roger Pau Monné CC: Chao Peng v8: - remove unnecessary line split. (suggested by Jan Beulich) - use MASK_EXTR(). (suggested by Jan Beulich) v7: - modify commit message. (suggested by Jan Beulich) - move the changes about check of 'cat_init_feature' in 'psr_cpu_init' into previous patch. (suggested by Jan Beulich) v6: - restore type of 'mba_write_msr' to 'void'. v5: - move out some CAT codes optimization to a new patch. (suggested by Jan Beulich) - modify commit message. (suggested by Jan Beulich) - change print type of 'linear' to be %d. (suggested by Jan Beulich) - change type of 'mba_write_msr' to uint32_t. - move printk in 'mba_init_feature' to reduce one return path. (suggested by Roger Pau Monné) - move the MBA format string in printk to a new line. (suggested by Roger Pau Monné) v4: - modify commit message. (suggested by Roger Pau Monné) - fix a comment. (suggested by Roger Pau Monné) - join two checks in a single if. (suggested by Roger Pau Monné) - remove redundant initialization of 'feat->cos_reg_val[0]'. (suggested by Roger Pau Monné) - change 'reg_b' to 'ebx'. (suggested by Jan Beulich) - change type of 'mba_init_feature' from 'int' to 'bool'. (suggested by Roger Pau Monné) - change type of 'cat_init_feature' from 'int' to 'bool'. v3: - replace 'psr_val_type' to 'psr_type'. Also, change 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'. (suggested by Roger Pau Monné) - replace 'MBA_LINEAR' to 'MBA_LINEAR_MASK' to make the name more clear. (suggested by Roger Pau Monné) - replase 'cat_info'/'mba_info' to 'cat'/'mba' to make the names shorter. (suggested by Roger Pau Monné) - change type of 'linear' to 'bool'. (suggested by Roger Pau Monné) - make format string of printf in one line. (suggested by Roger Pau Monné) v2: - modify commit message to replace 'cos register' to 'thrtl register' to make it accurate. (suggested by Chao Peng) - restore the place of the sentence to assign value to 'feat->cbm_len' because the MBA init flow is splitted out as a separate function in v1. (suggested by Chao Peng) - add comment to explain what the MBA thrtl defaul value '0' stands for. (suggested by Chao Peng) - check 'thrtl_max' under linear mode. It could not be euqal or larger than 100. (suggested by Chao Peng) v1: - rebase codes onto L2 CAT v15. - move comment to appropriate place. (suggested by Chao Peng) - implement 'mba_init_feature' and keep 'cat_init_feature'. (suggested by Chao Peng) - keep 'regs.b' into a local variable to avoid reading CPUID every time. (suggested by Chao Peng) --- xen/arch/x86/psr.c | 127 +++- xen/include/asm-x86/msr-index.h | 1 + xen/include/asm-x86/psr.h | 2 + 3 files changed, 114 insertions(+), 16 deletions(-) diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c index 50c5a98..3207988 100644 --- a/xen/arch/x86/psr.c +++ b/xen/arch/x86/psr.c @@ -27,13 +27,16 @@ * - CMT Cache Monitoring Technology * - COS/CLOSClass of Service. Also mean COS registers. * - COS_MAX Max number of COS for the feature (minus 1) + * - MBA Memory Bandwidth Allocation * - MSRsMachine Specific Registers * - PSR Intel Platform Shared Resource + * - THRTL_MAX Max throttle value (delay value) of MBA */ #define PSR_CMT(1u << 0) #define PSR_CAT(1u << 1) #define PSR_CDP(1u << 2) +#define PSR_MBA(1u << 3) #define CAT_CBM_LEN_MASK 0x1f #define CAT_COS_MAX_MASK 0x @@ -60,10 +63,14 @@ */ #define MAX_COS_NUM 2 +#define MBA_LINEAR_MASK(1u << 2) +#define MBA_THRTL_MAX_MASK 0xfff + enum psr_feat_type
[Xen-devel] [PATCH v8 03/16] x86: rename 'cbm_type' to 'psr_type' to make it general
This patch renames 'cbm_type' to 'psr_type' to generalize it.
Then, we can reuse this for all psr allocation features.
Signed-off-by: Yi Sun
Reviewed-by: Wei Liu
Reviewed-by: Roger Pau Monné
Acked-by: Jan Beulich
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v5:
- correct character of reviewer's name.
(suggested by Jan Beulich)
v4:
- fix words in commit message.
(suggested by Roger Pau Monné)
v3:
- replace 'psr_val_type' to 'psr_type' and remove '_VAL' from the enum
items.
(suggested by Roger Pau Monné)
v2:
- replace 'PSR_VAL_TYPE_{L3, L2}' to 'PSR_VAL_TYPE_{L3, L2}_CBM'.
(suggested by Chao Peng)
---
xen/arch/x86/domctl.c | 16 ++--
xen/arch/x86/psr.c| 62 +--
xen/arch/x86/sysctl.c | 4 +--
xen/include/asm-x86/psr.h | 18 +++---
4 files changed, 52 insertions(+), 48 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 1cffe93..bc025ce 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1444,25 +1444,25 @@ long arch_do_domctl(
case XEN_DOMCTL_PSR_SET_L3_CBM:
ret = psr_set_val(d, domctl->u.psr_alloc.target,
domctl->u.psr_alloc.data,
- PSR_CBM_TYPE_L3);
+ PSR_TYPE_L3_CBM);
break;
case XEN_DOMCTL_PSR_SET_L3_CODE:
ret = psr_set_val(d, domctl->u.psr_alloc.target,
domctl->u.psr_alloc.data,
- PSR_CBM_TYPE_L3_CODE);
+ PSR_TYPE_L3_CODE);
break;
case XEN_DOMCTL_PSR_SET_L3_DATA:
ret = psr_set_val(d, domctl->u.psr_alloc.target,
domctl->u.psr_alloc.data,
- PSR_CBM_TYPE_L3_DATA);
+ PSR_TYPE_L3_DATA);
break;
case XEN_DOMCTL_PSR_SET_L2_CBM:
ret = psr_set_val(d, domctl->u.psr_alloc.target,
domctl->u.psr_alloc.data,
- PSR_CBM_TYPE_L2);
+ PSR_TYPE_L2_CBM);
break;
#define domctl_psr_get_val(d, domctl, type, copyback) ({\
@@ -1476,19 +1476,19 @@ long arch_do_domctl(
})
case XEN_DOMCTL_PSR_GET_L3_CBM:
-ret = domctl_psr_get_val(d, domctl, PSR_CBM_TYPE_L3, copyback);
+ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L3_CBM, copyback);
break;
case XEN_DOMCTL_PSR_GET_L3_CODE:
-ret = domctl_psr_get_val(d, domctl, PSR_CBM_TYPE_L3_CODE,
copyback);
+ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L3_CODE, copyback);
break;
case XEN_DOMCTL_PSR_GET_L3_DATA:
-ret = domctl_psr_get_val(d, domctl, PSR_CBM_TYPE_L3_DATA,
copyback);
+ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L3_DATA, copyback);
break;
case XEN_DOMCTL_PSR_GET_L2_CBM:
-ret = domctl_psr_get_val(d, domctl, PSR_CBM_TYPE_L2, copyback);
+ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L2_CBM, copyback);
break;
#undef domctl_psr_get_val
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 11c204e..6dce823 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -100,24 +100,24 @@ static const struct feat_props {
unsigned int cos_num;
/*
- * An array to save all 'enum cbm_type' values of the feature. It is
+ * An array to save all 'enum psr_type' values of the feature. It is
* used with cos_num together to get/write a feature's COS registers
* values one by one.
*/
-enum cbm_type type[MAX_COS_NUM];
+enum psr_type type[MAX_COS_NUM];
/*
* alt_type is 'alternative type'. When this 'alt_type' is input, the
* feature does some special operations.
*/
-enum cbm_type alt_type;
+enum psr_type alt_type;
/* get_feat_info is used to return feature HW info through sysctl. */
bool (*get_feat_info)(const struct feat_node *feat,
uint32_t data[], unsigned int array_len);
/* write_msr is used to write out feature MSR register. */
-void (*write_msr)(unsigned int cos, uint32_t val, enum cbm_type type);
+void (*write_msr)(unsigned int cos, uint32_t val, enum psr_type type);
} *feat_props[FEAT_TYPE_NUM];
/*
@@ -215,13 +215,13 @@ static void free_socket_resources(unsigned int socket)
bitmap_zero(info->dom_set, DOMID_IDLE + 1);
}
-static enum psr_feat_type psr_cbm_type_to_feat_type(enum cbm_type type)
+static enum psr_feat_type psr_type_to_feat_type(enum psr_type type)
{
enum psr_feat_type feat_
Re: [Xen-devel] [PATCH v7 08/16] x86: implement set value flow for MBA
On 17-10-13 09:56:14, Jan Beulich wrote:
> >>> On 13.10.17 at 10:41, wrote:
> > @@ -274,16 +280,18 @@ static enum psr_feat_type psr_type_to_feat_type(enum
> > psr_type type)
> > return feat_type;
> > }
> >
> > -static bool psr_check_cbm(unsigned int cbm_len, unsigned long cbm)
> > +/* Implementation of allocation features' functions. */
> > +static bool cat_check_cbm(const struct feat_node *feat, uint32_t *val)
> > {
> > unsigned int first_bit, zero_bit;
> > +unsigned int cbm_len = feat->cat.cbm_len;
> > +unsigned long cbm = *val;
>
> These are necessary changes.
>
> > -/* Set bits should only in the range of [0, cbm_len]. */
> > -if ( cbm & (~0ul << cbm_len) )
> > -return false;
> > -
> > -/* At least one bit need to be set. */
> > -if ( cbm == 0 )
> > +/*
> > + * Set bits should be only in the range of [0, cbm_len).
> > + * And, at least one bit need to be set.
> > + */
> > +if ( (cbm & (~0ul << cbm_len)) || !cbm )
>
> But all of this doesn't really belong here. I don't outright object to
> you leaving it the way it is, but I'd prefer if you dropped these
> changes, or moved them to a separate patch if you think this is
> worthwhile.
>
Then, I would prefer to drop these changes.
> > @@ -501,6 +511,35 @@ static bool mba_get_feat_info(const struct feat_node
> > *feat,
> > static void mba_write_msr(unsigned int cos, uint32_t val,
> >enum psr_type type)
> > {
> > +wrmsrl(MSR_IA32_PSR_MBA_MASK(cos), val);
> > +}
> > +
> > +static bool mba_sanitize_thrtl(const struct feat_node *feat, uint32_t
> > *thrtl)
> > +{
> > +if ( *thrtl > feat->mba.thrtl_max )
> > +return false;
> > +
> > +/*
> > + * Per SDM (chapter "Memory Bandwidth Allocation Configuration"):
> > + * 1. Linear mode: In the linear mode the input precision is defined
> > + *as 100-(MBA_MAX). For instance, if the MBA_MAX value is 90, the
> > + *input precision is 10%. Values not an even multiple of the
> > + *precision (e.g., 12%) will be rounded down (e.g., to 10% delay
> > + *applied).
> > + * 2. Non-linear mode: Input delay values are powers-of-two from zero
> > + *to the MBA_MAX value from CPUID. In this case any values not a
> > + *power of two will be rounded down the next nearest power of two.
> > + */
> > +if ( feat->mba.linear )
> > +*thrtl -= *thrtl % (100 - feat->mba.thrtl_max);
> > +else
> > +{
> > +/* Not power of 2. */
> > +if ( *thrtl & (*thrtl - 1) )
> > +*thrtl &= 1 << (flsl(*thrtl) - 1);
>
> fls() will do now that the parameter type is uint32_t.
>
Yes, you are right. Sorry for missing it.
> Also why do you think &= is better than plain = here?
Not better. Will change it to '='.
>
> Jan
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v7 16/16] docs: add MBA description in docs
This patch adds MBA description in related documents. Signed-off-by: Yi Sun Acked-by: Wei Liu Reviewed-by: Roger Pau Monné --- CC: Ian Jackson CC: Wei Liu CC: Roger Pau Monné CC: Chao Peng v5: - remove 'closed-loop' in 'xl-psr.markdown' (suggested by Roger Pau Monné) v4: - modify description of MBA in 'xl.pod.1.in' to be same as feature doc. (suggested by Roger Pau Monné) - fix words issue. (suggested by Roger Pau Monné) v2: - state the value type shown by 'psr-mba-show'. For linear mode, it shows decimal value. For non-linear mode, it shows hexadecimal value. (suggested by Chao Peng) --- docs/man/xl.pod.1.in | 33 + docs/misc/xl-psr.markdown | 62 +++ 2 files changed, 95 insertions(+) diff --git a/docs/man/xl.pod.1.in b/docs/man/xl.pod.1.in index cd8bb1c..324ef9e 100644 --- a/docs/man/xl.pod.1.in +++ b/docs/man/xl.pod.1.in @@ -1845,6 +1845,39 @@ processed. =back +=head2 Memory Bandwidth Allocation + +Intel Skylake and later server platforms offer capabilities to configure and +make use of the Memory Bandwidth Allocation (MBA) mechanisms, which provides +OS/VMMs the ability to slow misbehaving apps/VMs by using a credit-based +throttling mechanism. In the Xen implementation, MBA is used to control memory +bandwidth on VM basis. To enforce bandwidth on a specific domain, just set +throttling value (THRTL) for the domain. + +=over 4 + +=item B [I] I I + +Set throttling value (THRTL) for a domain. For how to specify I +please refer to L<http://xenbits.xen.org/docs/unstable/misc/xl-psr.html>. + +B + +=over 4 + +=item B<-s SOCKET>, B<--socket=SOCKET> + +Specify the socket to process, otherwise all sockets are processed. + +=back + +=item B [I] + +Show MBA settings for a certain domain or all domains. For linear mode, it +shows the decimal value. For non-linear mode, it shows hexadecimal value. + +=back + =head1 IGNORED FOR COMPATIBILITY WITH XM xl is mostly command-line compatible with the old xm utility used with diff --git a/docs/misc/xl-psr.markdown b/docs/misc/xl-psr.markdown index 04dd957..3d196ed 100644 --- a/docs/misc/xl-psr.markdown +++ b/docs/misc/xl-psr.markdown @@ -186,6 +186,68 @@ Setting data CBM for a domain: Setting the same code and data CBM for a domain: `xl psr-cat-set ` +## Memory Bandwidth Allocation (MBA) + +Memory Bandwidth Allocation (MBA) is a new feature available on Intel +Skylake and later server platforms that allows an OS or Hypervisor/VMM to +slow misbehaving apps/VMs by using a credit-based throttling mechanism. To +enforce bandwidth on a specific domain, just set throttling value (THRTL) +into Class of Service (COS). MBA provides two THRTL mode. One is linear mode +and the other is non-linear mode. + +In the linear mode the input precision is defined as 100-(THRTL_MAX). Values +not an even multiple of the precision (e.g., 12%) will be rounded down (e.g., +to 10% delay by the hardware). + +If linear values are not supported then input delay values are powers-of-two +from zero to the THRTL_MAX value from CPUID. In this case any values not a power +of two will be rounded down the next nearest power of two. + +For example, assuming a system with 2 domains: + + * A THRTL of 0x0 for every domain means each domain can access the whole cache + without any delay. This is the default. + + * Linear mode: Giving one domain a THRTL of 0xC and the other domain's 0 means + that the first domain gets 10% delay to access the cache and the other one + without any delay. + + * Non-linear mode: Giving one domain a THRTL of 0xC and the other domain's 0 + means that the first domain gets 8% delay to access the cache and the other + one without any delay. + +For more detailed information please refer to Intel SDM chapter +"Introduction to Memory Bandwidth Allocation". + +In Xen's implementation, THRTL can be configured with libxl/xl interfaces but +COS is maintained in hypervisor only. The cache partition granularity is per +domain, each domain has COS=0 assigned by default, the corresponding THRTL is +0, which means all the cache resource can be accessed without delay. + +### xl interfaces + +System MBA information such as maximum COS and maximum THRTL can be obtained by: + +`xl psr-hwinfo --mba` + +The simplest way to change a domain's THRTL from its default is running: + +`xl psr-mba-set [OPTIONS] ` + +In a multi-socket system, the same thrtl will be set on each socket by default. +Per socket thrtl can be specified with the `--socket SOCKET` option. + +Setting the THRTL may not be successful if insufficient COS is available. In +such case unused COS(es) may be freed by setting THRTL of all related domains to +its default value(0). + +Per domain THRTL settings can be shown by: + +`xl psr-mba-show [OPTIONS] ` + +For linear mode, it shows the decimal v
[Xen-devel] [PATCH v7 06/16] x86: implement get hw info flow for MBA
This patch implements get HW info flow for MBA including its callback
function and sysctl interface.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Jan Beulich
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v7:
- change 'PSR_INFO_IDX_MBA_FLAG' to 'PSR_INFO_IDX_MBA_FLAGS'.
(suggested by Jan Beulich)
v5:
- use ASSERT in 'mba_get_feat_info'.
(suggested by Roger Pau Monné)
- correct initialization format of 'data[PSR_INFO_ARRAY_SIZE]'.
(suggested by Roger Pau Monné and Jan Beulich)
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- initialize 'data[PSR_INFO_ARRAY_SIZE]' to 0 to prevent to leak stack data.
(suggested by Roger Pau Monné)
v3:
- replace 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
v2:
- use 'XEN_SYSCTL_PSR_MBA_LINEAR' to set MBA feature HW info.
(suggested by Chao Peng)
v1:
- sort 'PSR_INFO_IDX_' macros as feature.
(suggested by Chao Peng)
- rename 'PSR_INFO_IDX_MBA_LINEAR' to 'PSR_INFO_IDX_MBA_FLAG'.
- rename 'linear' in 'struct mba_info' to 'flags' for future extension.
(suggested by Chao Peng)
---
xen/arch/x86/psr.c | 14 +-
xen/arch/x86/sysctl.c | 21 -
xen/include/asm-x86/psr.h | 2 ++
xen/include/public/sysctl.h | 8
4 files changed, 43 insertions(+), 2 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 64d30b9..f5b395b 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -263,6 +263,10 @@ static enum psr_feat_type psr_type_to_feat_type(enum
psr_type type)
feat_type = FEAT_TYPE_L2_CAT;
break;
+case PSR_TYPE_MBA_THRTL:
+feat_type = FEAT_TYPE_MBA;
+break;
+
default:
ASSERT_UNREACHABLE();
}
@@ -483,7 +487,15 @@ static const struct feat_props l2_cat_props = {
static bool mba_get_feat_info(const struct feat_node *feat,
uint32_t data[], unsigned int array_len)
{
-return false;
+ASSERT(array_len == PSR_INFO_ARRAY_SIZE);
+
+data[PSR_INFO_IDX_COS_MAX] = feat->cos_max;
+data[PSR_INFO_IDX_MBA_THRTL_MAX] = feat->mba.thrtl_max;
+
+if ( feat->mba.linear )
+data[PSR_INFO_IDX_MBA_FLAGS] |= XEN_SYSCTL_PSR_MBA_LINEAR;
+
+return true;
}
static void mba_write_msr(unsigned int cos, uint32_t val,
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index 6d48cac..ffad585 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -174,7 +174,7 @@ long arch_do_sysctl(
case XEN_SYSCTL_psr_alloc:
switch ( sysctl->u.psr_alloc.cmd )
{
-uint32_t data[PSR_INFO_ARRAY_SIZE];
+uint32_t data[PSR_INFO_ARRAY_SIZE] = { };
case XEN_SYSCTL_PSR_get_l3_info:
{
@@ -214,6 +214,25 @@ long arch_do_sysctl(
break;
}
+case XEN_SYSCTL_PSR_get_mba_info:
+{
+ret = psr_get_info(sysctl->u.psr_alloc.target,
+ PSR_TYPE_MBA_THRTL, data, ARRAY_SIZE(data));
+if ( ret )
+break;
+
+sysctl->u.psr_alloc.u.mba_info.cos_max =
+ data[PSR_INFO_IDX_COS_MAX];
+sysctl->u.psr_alloc.u.mba_info.thrtl_max =
+ data[PSR_INFO_IDX_MBA_THRTL_MAX];
+sysctl->u.psr_alloc.u.mba_info.flags =
+ data[PSR_INFO_IDX_MBA_FLAGS];
+
+if ( __copy_field_to_guest(u_sysctl, sysctl, u.psr_alloc) )
+ret = -EFAULT;
+break;
+}
+
default:
ret = -EOPNOTSUPP;
break;
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index 3cf544a..c2257da 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -39,6 +39,8 @@
#define PSR_INFO_IDX_COS_MAX0
#define PSR_INFO_IDX_CAT_CBM_LEN1
#define PSR_INFO_IDX_CAT_FLAGS 2
+#define PSR_INFO_IDX_MBA_THRTL_MAX 1
+#define PSR_INFO_IDX_MBA_FLAGS 2
#define PSR_INFO_ARRAY_SIZE 3
struct psr_cmt_l3 {
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index a50e345..f7f26c3 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -698,6 +698,7 @@ struct xen_sysctl_pcitopoinfo {
#define XEN_SYSCTL_PSR_get_l3_info 0
#define XEN_SYSCTL_PSR_get_l2_info 1
+#define XEN_SYSCTL_PSR_get_mba_info 2
struct xen_sysctl_psr_alloc {
uint32_t cmd; /* IN: XEN_SYSCTL_PSR_* */
uint32_t target;/* IN */
@@ -708,6 +709,13 @@ struct xen_sysctl_psr_alloc {
#define XEN_SYSCTL_PSR_CAT_L
[Xen-devel] [PATCH v7 07/16] x86: implement get value interface for MBA
This patch implements get value domctl interface for MBA.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Jan Beulich
---
CC: Andrew Cooper
CC: Jan Beulich
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v5:
- use newly defined macro to get MBA thrtl.
(suggested by Roger Pau Monné)
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
v3:
- change 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
---
xen/arch/x86/domctl.c | 4
xen/include/public/domctl.h | 1 +
2 files changed, 5 insertions(+)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index ae06627..ffb038c 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1491,6 +1491,10 @@ long arch_do_domctl(
ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L2_CBM, copyback);
break;
+case XEN_DOMCTL_PSR_GET_MBA_THRTL:
+ret = domctl_psr_get_val(d, domctl, PSR_TYPE_MBA_THRTL, copyback);
+break;
+
#undef domctl_psr_get_val
default:
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index c099334..e8f4c4c 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1069,6 +1069,7 @@ struct xen_domctl_psr_alloc {
#define XEN_DOMCTL_PSR_GET_L3_DATA5
#define XEN_DOMCTL_PSR_SET_L2_CBM 6
#define XEN_DOMCTL_PSR_GET_L2_CBM 7
+#define XEN_DOMCTL_PSR_GET_MBA_THRTL 9
uint32_t cmd; /* IN: XEN_DOMCTL_PSR_* */
uint32_t target;/* IN */
uint64_t data; /* IN/OUT */
--
1.9.1
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v7 08/16] x86: implement set value flow for MBA
This patch implements set value flow for MBA including its callback
function and domctl interface.
Signed-off-by: Yi Sun
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v7:
- change name of 'check_val' to 'sanitize'.
(suggested by Jan Beulich)
- fix comments.
(suggested by Jan Beulich)
- add parentheses and change '== 0' to '!'.
(suggested by Jan Beulich)
- remove unnecessary check of 'mba.thrtl_max'.
(suggested by Jan Beulich)
- remove unnecessary intermediate variable 'mod'.
(suggested by Jan Beulich)
- refine an assignement sentence to use '&='.
(suggested by Jan Beulich)
- change type of last parameter of 'sanitize' to 'uint32_t' and
apply same change to 'cat_check_cbm'.
(suggested by Jan Beulich)
v6:
- split co-exist features' values setting flow to a new patch.
(suggested by Jan Beulich)
- restore codes related to 'mba_check_thrtl' and 'check_value'.
(suggested by Jan Beulich)
v5:
- adjust position of 'cat_check_cbm' to not to make changes so big.
(suggested by Roger Pau Monné)
- remove 'props' from 'struct cos_write_info'.
(suggested by Roger Pau Monné)
- make a single return statement in 'mba_check_thrtl'.
(suggested by Jan Beulich)
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- join two checks into a single if.
(suggested by Roger Pau Monné)
- remove redundant local variable 'array_len'.
(suggested by Roger Pau Monné)
v3:
- modify commit message to make it clear.
(suggested by Roger Pau Monné)
- modify functionality of 'check_val' to make it simple to only check value.
Change the last parameter type from 'unsigned long *' to 'unsigned long'.
(suggested by Roger Pau Monné)
- call rdmsrl to get value just written into MSR for MBA. Because HW can
automatically change input value to what it wants.
(suggested by Roger Pau Monné)
- change type of 'write_msr' to 'uint32_t' to return the value actually
written into MSR. Then, change 'do_write_psr_msrs' to set the returned
value into 'cos_reg_val[]'
- move the declaration of 'j' into loop in 'do_write_psr_msrs'.
(suggested by Roger Pau Monné)
- change 'mba_info' to 'mba'.
(suggested by Roger Pau Monné)
- change 'cat_info' to 'cat'.
(suggested by Roger Pau Monné)
- rename 'psr_cat/PSR_CAT' to 'psr_alloc/PSR_ALLOC' and remove 'op/OP'
from name.
(suggested by Roger Pau Monné)
- change 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
v2:
- remove linear mode 'thrtl_max' check in 'mba_check_thrtl' because it has
been checked in 'mba_init_feature'.
(suggested by Chao Peng)
- for non-linear mode, check if '*thrtl' is not 0 in 'mba_check_thrtl'. If
it is 0, we do not need to change it.
(suggested by Chao Peng)
- move comments to explain changes of 'cos_write_info' from psr.c to commit
message.
(suggested by Chao Peng)
---
xen/arch/x86/domctl.c | 6 +
xen/arch/x86/psr.c | 58 ++---
xen/include/public/domctl.h | 1 +
3 files changed, 56 insertions(+), 9 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index ffb038c..e95004b 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1465,6 +1465,12 @@ long arch_do_domctl(
PSR_TYPE_L2_CBM);
break;
+case XEN_DOMCTL_PSR_SET_MBA_THRTL:
+ret = psr_set_val(d, domctl->u.psr_alloc.target,
+ domctl->u.psr_alloc.data,
+ PSR_TYPE_MBA_THRTL);
+break;
+
#define domctl_psr_get_val(d, domctl, type, copyback) ({\
uint32_t v_;\
int r_ = psr_get_val((d), (domctl)->u.psr_alloc.target, \
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index f5b395b..c80ba25 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -138,6 +138,12 @@ static const struct feat_props {
/* write_msr is used to write out feature MSR register. */
void (*write_msr)(unsigned int cos, uint32_t val, enum psr_type type);
+
+/*
+ * sanitize is used to check if input val fulfills SDM requirement.
+ * And change it to valid value if SDM allows.
+ */
+bool (*sanitize)(const struct feat_node *feat, uint3
[Xen-devel] [PATCH v7 01/16] docs: create Memory Bandwidth Allocation (MBA) feature document
This patch creates MBA feature document in doc/features/. It describes key points to implement MBA which is described in details in Intel SDM "Introduction to Memory Bandwidth Allocation". Signed-off-by: Yi Sun Reviewed-by: Roger Pau Monné --- CC: Jan Beulich CC: Andrew Cooper CC: Wei Liu CC: Ian Jackson CC: Daniel De Graaf CC: Roger Pau Monné CC: Konrad Rzeszutek Wilk CC: Chao Peng CC: Julien Grall v6: - fix some words. (suggested by Roger Pau Monné) v5: - correct some words. (suggested by Roger Pau Monné) - change 'xl psr-mba-set 1 0xa' to 'xl psr-mba-set 1 10'. (suggested by Roger Pau Monné) v4: - add 'domain-name' as parameter of 'psr-mba-show/psr-mba-set'. (suggested by Roger Pau Monné) - fix some wordings. (suggested by Roger Pau Monné) - explain how user can know the MBA_MAX. (suggested by Roger Pau Monné) - move the description of 'Linear mode/Non-linear mode' into section of 'psr-mba-show'. (suggested by Roger Pau Monné) - change 'per-thread' to 'per-hyper-thread' to make it clearer. (suggested by Roger Pau Monné) - upgrade revision number. v3: - remove 'closed-loop' related description. (suggested by Roger Pau Monné) - explain 'linear' and 'non-linear' before mentioning them. (suggested by Roger Pau Monné) - adjust desription of 'psr-mba-set'. (suggested by Roger Pau Monné) - explain 'MBA_MAX'. (suggested by Roger Pau Monné) - remove 'n<64'. (suggested by Roger Pau Monné) - fix some wordings. (suggested by Roger Pau Monné) - add context in 'Testing' part to make things more clear. (suggested by Roger Pau Monné) v2: - declare 'HW' in Terminology. (suggested by Chao Peng) - replace 'COS ID of VCPU' to 'COS ID of domain'. (suggested by Chao Peng) - replace 'COS register' to 'Thrtl MSR'. (suggested by Chao Peng) - add description for 'psr-mba-show' to state that the decimal value is shown for linear mode but hexadecimal value is shown for non-linear mode. (suggested by Chao Peng) - remove content in 'Areas for improvement'. (suggested by Chao Peng) - use '<>' to specify mandatory argument to a command. (suggested by Wei Liu) v1: - remove a special character to avoid the error when building pandoc. --- docs/features/intel_psr_mba.pandoc | 297 + 1 file changed, 297 insertions(+) create mode 100644 docs/features/intel_psr_mba.pandoc diff --git a/docs/features/intel_psr_mba.pandoc b/docs/features/intel_psr_mba.pandoc new file mode 100644 index 000..86df661 --- /dev/null +++ b/docs/features/intel_psr_mba.pandoc @@ -0,0 +1,297 @@ +% Intel Memory Bandwidth Allocation (MBA) Feature +% Revision 1.8 + +\clearpage + +# Basics + + + Status: **Tech Preview** + +Architecture(s): Intel x86 + + Component(s): Hypervisor, toolstack + + Hardware: MBA is supported on Skylake Server and beyond + + +# Terminology + +* CAT Cache Allocation Technology +* CBM Capacity BitMasks +* CDP Code and Data Prioritization +* COS/CLOSClass of Service +* HW Hardware +* MBA Memory Bandwidth Allocation +* MSRsMachine Specific Registers +* PSR Intel Platform Shared Resource +* THRTL Throttle value or delay value + +# Overview + +The Memory Bandwidth Allocation (MBA) feature provides indirect and approximate +control over memory bandwidth available per-core. This feature provides OS/ +hypervisor the ability to slow misbehaving apps/domains by using a credit-based +throttling mechanism. + +# User details + +* Feature Enabling: + + Add "psr=mba" to boot line parameter to enable MBA feature. + +* xl interfaces: + + 1. `psr-mba-show [domain-id|domain-name]`: + + Show memory bandwidth throttling for domain. Under different modes, it + shows different type of data. + + There are two modes: + Linear mode: the input precision is defined as 100-(MBA_MAX). For instance, + if the MBA_MAX value is 90, the input precision is 10%. Values not an even + multiple of the precision (e.g., 12%) will be rounded down (e.g., to 10% + delay applied) by HW automatically. The response of throttling value is + linear. + + Non-linear mode: input delay values are powers-of-two from zero to the + MBA_MAX value from CPUID. In this case any values not a power of two will + be rounded down the next nearest power of two by HW automatically. The + response of throttli
[Xen-devel] [PATCH v7 13/16] tools: rename 'xc_psr_cat_type' to 'xc_psr_type'
This patch renames 'xc_psr_cat_type' to 'xc_psr_type' so that
the structure name is common for all allocation features.
Signed-off-by: Yi Sun
Acked-by: Wei Liu
Reviewed-by: Chao Peng
Reviewed-by: Roger Pau Monné
---
CC: Ian Jackson
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v5:
- remove a duplicated ';'.
(suggested by Roger Pau Monné)
v4:
- move assignment of xc_type to its declaration place.
(suggested by Roger Pau Monné)
v3:
- change 'xc_psr_val_type' to 'xc_psr_type'.
(suggested by Roger Pau Monné)
---
tools/libxc/include/xenctrl.h | 8
tools/libxc/xc_psr.c | 4 ++--
tools/libxl/libxl_psr.c | 11 +--
3 files changed, 11 insertions(+), 12 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 2d977c8..2736bc5 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2466,13 +2466,13 @@ enum xc_psr_cmt_type {
};
typedef enum xc_psr_cmt_type xc_psr_cmt_type;
-enum xc_psr_cat_type {
+enum xc_psr_type {
XC_PSR_CAT_L3_CBM = 1,
XC_PSR_CAT_L3_CBM_CODE = 2,
XC_PSR_CAT_L3_CBM_DATA = 3,
XC_PSR_CAT_L2_CBM = 4,
};
-typedef enum xc_psr_cat_type xc_psr_cat_type;
+typedef enum xc_psr_type xc_psr_type;
enum xc_psr_feat_type {
XC_PSR_CAT_L3,
@@ -2512,10 +2512,10 @@ int xc_psr_cmt_get_data(xc_interface *xch, uint32_t
rmid, uint32_t cpu,
int xc_psr_cmt_enabled(xc_interface *xch);
int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_type type, uint32_t target,
uint64_t data);
int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_type type, uint32_t target,
uint64_t *data);
int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
xc_psr_feat_type type, xc_psr_hw_info *hw_info);
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 2c605a7..01f4ba7 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -249,7 +249,7 @@ int xc_psr_cmt_enabled(xc_interface *xch)
return 0;
}
int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_type type, uint32_t target,
uint64_t data)
{
DECLARE_DOMCTL;
@@ -284,7 +284,7 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t
domid,
}
int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_type type, uint32_t target,
uint64_t *data)
{
int rc;
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index b053abd..c54cb6f 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -303,11 +303,11 @@ out:
return rc;
}
-static inline xc_psr_cat_type libxl__psr_cbm_type_to_libxc_psr_cat_type(
+static inline xc_psr_type libxl__psr_cbm_type_to_libxc_psr_type(
libxl_psr_cbm_type type)
{
-BUILD_BUG_ON(sizeof(libxl_psr_cbm_type) != sizeof(xc_psr_cat_type));
-return (xc_psr_cat_type)type;
+BUILD_BUG_ON(sizeof(libxl_psr_cbm_type) != sizeof(xc_psr_type));
+return (xc_psr_type)type;
}
int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
@@ -325,12 +325,11 @@ int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
}
libxl_for_each_set_bit(socketid, *target_map) {
-xc_psr_cat_type xc_type;
+xc_psr_type xc_type = libxl__psr_cbm_type_to_libxc_psr_type(type);
if (socketid >= nr_sockets)
break;
-xc_type = libxl__psr_cbm_type_to_libxc_psr_cat_type(type);
if (xc_psr_cat_set_domain_data(ctx->xch, domid, xc_type,
socketid, cbm)) {
libxl__psr_cat_log_err_msg(gc, errno);
@@ -349,7 +348,7 @@ int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
{
GC_INIT(ctx);
int rc = 0;
-xc_psr_cat_type xc_type = libxl__psr_cbm_type_to_libxc_psr_cat_type(type);
+xc_psr_type xc_type = libxl__psr_cbm_type_to_libxc_psr_type(type);
if (xc_psr_cat_get_domain_data(ctx->xch, domid, xc_type,
target, cbm_r)) {
--
1.9.1
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v7 15/16] tools: implement new generic set value interface and MBA set value command
This patch implements new generic set value interfaces in libxc and libxl.
These interfaces are suitable for all allocation features. It also adds a
new MBA set value command in xl.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Wei Liu
---
CC: Wei Liu
CC: Ian Jackson
CC: Roger Pau Monné
CC: Chao Peng
v5:
- move xc_type definition and value get out of the loop.
(suggested by Roger Pau Monné)
v4:
- remove 'ALLOC_' from macro name.
(suggested by Roger Pau Monné)
- adjust place of argc check and return EXIT_FAILURE.
(suggested by Roger Pau Monné)
- fix indentation issue.
(suggested by Roger Pau Monné)
- move same type local variables declaration to a single line.
(suggested by Roger Pau Monné)
v3:
- add 'const' for 'opts[]' in 'main_psr_mba_set'.
(suggested by Roger Pau Monné)
- replace 'libxl_psr_cbm_type' to 'libxl_psr_type' for newly defined
interfaces.
(suggested by Roger Pau Monné)
---
tools/libxc/include/xenctrl.h | 6 ++---
tools/libxc/xc_psr.c | 9 ---
tools/libxl/libxl_psr.c | 52
tools/xl/xl.h | 1 +
tools/xl/xl_cmdtable.c| 6 +
tools/xl/xl_psr.c | 55 +++
6 files changed, 96 insertions(+), 33 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index dcea09e..f2463f9 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2512,9 +2512,9 @@ int xc_psr_cmt_get_data(xc_interface *xch, uint32_t rmid,
uint32_t cpu,
uint64_t *tsc);
int xc_psr_cmt_enabled(xc_interface *xch);
-int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_type type, uint32_t target,
- uint64_t data);
+int xc_psr_set_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_type type, uint32_t target,
+ uint64_t data);
int xc_psr_get_domain_data(xc_interface *xch, uint32_t domid,
xc_psr_type type, uint32_t target,
uint64_t *data);
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 191de97..1609185 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -248,9 +248,9 @@ int xc_psr_cmt_enabled(xc_interface *xch)
return 0;
}
-int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_type type, uint32_t target,
- uint64_t data)
+int xc_psr_set_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_type type, uint32_t target,
+ uint64_t data)
{
DECLARE_DOMCTL;
uint32_t cmd;
@@ -269,6 +269,9 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t
domid,
case XC_PSR_CAT_L2_CBM:
cmd = XEN_DOMCTL_PSR_SET_L2_CBM;
break;
+case XC_PSR_MBA_THRTL:
+cmd = XEN_DOMCTL_PSR_SET_MBA_THRTL;
+break;
default:
errno = EINVAL;
return -1;
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index 7c560bc..9ced7d1 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -328,32 +328,7 @@ int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
libxl_psr_cbm_type type, libxl_bitmap *target_map,
uint64_t cbm)
{
-GC_INIT(ctx);
-int rc;
-int socketid, nr_sockets;
-
-rc = libxl__count_physical_sockets(gc, &nr_sockets);
-if (rc) {
-LOGED(ERROR, domid, "failed to get system socket count");
-goto out;
-}
-
-libxl_for_each_set_bit(socketid, *target_map) {
-xc_psr_type xc_type = libxl__psr_type_to_libxc_psr_type(type);
-
-if (socketid >= nr_sockets)
-break;
-
-if (xc_psr_cat_set_domain_data(ctx->xch, domid, xc_type,
- socketid, cbm)) {
-libxl__psr_alloc_log_err_msg(gc, errno, type);
-rc = ERROR_FAIL;
-}
-}
-
-out:
-GC_FREE;
-return rc;
+return libxl_psr_set_val(ctx, domid, type, target_map, cbm);
}
int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
@@ -453,7 +428,30 @@ int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
libxl_psr_type type, libxl_bitmap *target_map,
uint64_t val)
{
-return ERROR_FAIL;
+GC_INIT(ctx);
+int rc, socketid, nr_sockets;
+xc_psr_type xc_type = libxl__psr_type_to_libxc_psr_type(type);
+
+rc = libxl__count_physical_sockets(gc, &nr_sockets);
+if (rc) {
+LOG(ERROR, "failed to get system socket count");
+goto out;
+}
+
+libxl_for_each_set_bit(s
[Xen-devel] [PATCH v7 14/16] tools: implement new generic get value interface and MBA get value command
This patch implements generic get value interfaces in libxc and libxl.
It also refactors the get value flow in xl to make it be suitable for all
allocation features. Based on that, a new MBA get value command is added in xl.
Signed-off-by: Yi Sun
Acked-by: Wei Liu
Reviewed-by: Roger Pau Monné
---
CC: Ian Jackson
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v6:
- fix one coding style issue.
(suggested by Roger Pau Monné)
v5:
- start a newline for "CDP" because it exceeds 80 characters.
(suggested by Roger Pau Monné)
- remove a duplicated ';'.
(suggested by Roger Pau Monné)
- remove a extra newline.
(suggested by Roger Pau Monné)
- correct words in log message.
(suggested by Roger Pau Monné)
v4:
- use designated initializers for 'feat_name[]'.
(suggested by Roger Pau Monné)
- use LOG in 'libxl__psr_alloc_log_err_msg'.
(suggested by Roger Pau Monné)
v3:
- replace 'libxl_psr_cbm_type' to 'libxl_psr_type' in newly defined
interfaces.
(suggested by Roger Pau Monné)
v2:
- change 'CAT_INFO'/'MBA_INFO' to 'CAT'/'MBA'. The related structure names
are changed too.
(suggested by Chao Peng)
---
tools/libxc/include/xenctrl.h | 7 +-
tools/libxc/xc_psr.c | 9 +-
tools/libxl/libxl_psr.c | 58 -
tools/xl/xl.h | 1 +
tools/xl/xl_cmdtable.c| 5 ++
tools/xl/xl_psr.c | 185 ++
6 files changed, 183 insertions(+), 82 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 2736bc5..dcea09e 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2471,6 +2471,7 @@ enum xc_psr_type {
XC_PSR_CAT_L3_CBM_CODE = 2,
XC_PSR_CAT_L3_CBM_DATA = 3,
XC_PSR_CAT_L2_CBM = 4,
+XC_PSR_MBA_THRTL = 5,
};
typedef enum xc_psr_type xc_psr_type;
@@ -2514,9 +2515,9 @@ int xc_psr_cmt_enabled(xc_interface *xch);
int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
xc_psr_type type, uint32_t target,
uint64_t data);
-int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_type type, uint32_t target,
- uint64_t *data);
+int xc_psr_get_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_type type, uint32_t target,
+ uint64_t *data);
int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
xc_psr_feat_type type, xc_psr_hw_info *hw_info);
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 01f4ba7..191de97 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -283,9 +283,9 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t
domid,
return do_domctl(xch, &domctl);
}
-int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_type type, uint32_t target,
- uint64_t *data)
+int xc_psr_get_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_type type, uint32_t target,
+ uint64_t *data)
{
int rc;
DECLARE_DOMCTL;
@@ -305,6 +305,9 @@ int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t
domid,
case XC_PSR_CAT_L2_CBM:
cmd = XEN_DOMCTL_PSR_GET_L2_CBM;
break;
+case XC_PSR_MBA_THRTL:
+cmd = XEN_DOMCTL_PSR_GET_MBA_THRTL;
+break;
default:
errno = EINVAL;
return -1;
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index c54cb6f..7c560bc 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -71,16 +71,30 @@ static void libxl__psr_cmt_log_err_msg(libxl__gc *gc, int
err)
LOGE(ERROR, "%s", msg);
}
-static void libxl__psr_cat_log_err_msg(libxl__gc *gc, int err)
+static void libxl__psr_alloc_log_err_msg(libxl__gc *gc,
+ int err,
+ libxl_psr_type type)
{
+/*
+ * Index is 'libxl_psr_type' so we set two 'CDP' to correspond to
+ * DATA and CODE.
+ */
+const char * const feat_name[] = {
+[LIBXL_PSR_CBM_TYPE_UNKNOWN] = "UNKNOWN",
+[LIBXL_PSR_CBM_TYPE_L3_CBM] = "L3 CAT",
+[LIBXL_PSR_CBM_TYPE_L3_CBM_CODE...LIBXL_PSR_CBM_TYPE_L3_CBM_DATA] =
+ "CDP",
+[LIBXL_PSR_CBM_TYPE_L2_CBM] = "L2 CAT",
+[LIBXL_PSR_CBM_TYPE_MBA_THRTL] = "MBA",
+};
char *msg;
switch (err) {
case ENODEV:
-msg = "CAT is not supported in this system";
+msg = "is not supported
[Xen-devel] [PATCH v7 09/16] tools: create general interfaces to support psr allocation features
This patch creates general interfaces in libxl to support all psr
allocation features.
Add 'LIBXL_HAVE_PSR_GENERIC' to indicate interface change.
Please note, the functionality cannot work until later patches
are applied.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Wei Liu
---
CC: Wei Liu
CC: Ian Jackson
CC: Roger Pau Monné
CC: Chao Peng
v5:
- adjust parameters position in 'libxl_psr_get_hw_info'.
(suggested by Roger Pau Monné)
v4:
- add description for LIBXL_HAVE_PSR_GENERIC to mention newly added
public functions.
(suggested by Roger Pau Monné)
v3:
- change 'LIBXL_HAVE_PSR_MBA' to 'LIBXL_HAVE_PSR_GENERIC'.
(suggested by Roger Pau Monné)
- 'typedef enum libxl_psr_cbm_type libxl_psr_type;' in libxl.h.
(suggested by Roger Pau Monné and Wei Liu)
- change 'libxl_psr_cbm_type' to 'libxl_psr_type' in newly defined
interfaces.
(suggested by Roger Pau Monné)
v2:
- remove '_INFO' in 'libxl_psr_feat_type' and make corresponding
changes in 'libxl_psr_hw_info'.
(suggested by Chao Peng)
---
tools/libxl/libxl.h | 37 +
tools/libxl/libxl_psr.c | 25 +
tools/libxl/libxl_types.idl | 22 ++
3 files changed, 84 insertions(+)
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 827272e..0d2dee8 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -967,6 +967,17 @@ void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, const
libxl_mac *src);
#define LIBXL_HAVE_PSR_L2_CAT 1
/*
+ * LIBXL_HAVE_PSR_GENERIC
+ *
+ * If this is defined, the Memory Bandwidth Allocation feature is supported.
+ * The following public functions are available:
+ * libxl_psr_{set/get}_val
+ * libxl_psr_get_hw_info
+ * libxl_psr_hw_info_list_free
+ */
+#define LIBXL_HAVE_PSR_GENERIC 1
+
+/*
* LIBXL_HAVE_MCA_CAPS
*
* If this is defined, setting MCA capabilities for HVM domain is supported.
@@ -2287,6 +2298,32 @@ int libxl_psr_cat_get_info(libxl_ctx *ctx,
libxl_psr_cat_info **info,
int libxl_psr_cat_get_l3_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
int *nr);
void libxl_psr_cat_info_list_free(libxl_psr_cat_info *list, int nr);
+
+typedef enum libxl_psr_cbm_type libxl_psr_type;
+
+/*
+ * Function to set a domain's value. It operates on a single or multiple
+ * target(s) defined in 'target_map'. 'target_map' specifies all the sockets
+ * to be operated on.
+ */
+int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_type type, libxl_bitmap *target_map,
+ uint64_t val);
+/*
+ * Function to get a domain's cbm. It operates on a single 'target'.
+ * 'target' specifies which socket to be operated on.
+ */
+int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_type type, unsigned int target,
+ uint64_t *val);
+/*
+ * On success, the function returns an array of elements in 'info',
+ * and the length in 'nr'.
+ */
+int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_feat_type type,
+ unsigned int lvl, unsigned int *nr,
+ libxl_psr_hw_info **info);
+void libxl_psr_hw_info_list_free(libxl_psr_hw_info *list, unsigned int nr);
#endif
/* misc */
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index 197505a..d4f5f67 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -428,6 +428,31 @@ void libxl_psr_cat_info_list_free(libxl_psr_cat_info
*list, int nr)
free(list);
}
+int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_type type, libxl_bitmap *target_map,
+ uint64_t val)
+{
+return ERROR_FAIL;
+}
+
+int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_type type, unsigned int target,
+ uint64_t *val)
+{
+return ERROR_FAIL;
+}
+
+int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_feat_type type,
+ unsigned int lvl, unsigned int *nr,
+ libxl_psr_hw_info **info)
+{
+return ERROR_FAIL;
+}
+
+void libxl_psr_hw_info_list_free(libxl_psr_hw_info *list, unsigned int nr)
+{
+}
+
/*
* Local variables:
* mode: C
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index c2a1141..6f53b2d 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -1025,6 +1025,7 @@ libxl_psr_cbm_type = Enumeration("psr_cbm_type", [
(2, "L3_CBM_CODE"),
(3, "L3_CBM_DATA"),
(4, "L2_CBM"),
+(5, "MBA_THRTL"),
])
libxl_psr_cat_info = Struct("psr_cat_info", [
@@ -1033,3 +1034,24
[Xen-devel] [PATCH v7 10/16] tools: implement the new libxc get hw info interface
This patch implements a new libxc get hw info interface and corresponding
data structures. It also changes libxl_psr.c to call this new interface.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Wei Liu
---
CC: Wei Liu
CC: Ian Jackson
CC: Roger Pau Monné
CC: Chao Peng
v6:
- remove unnecessary spaces in brackets.
(suggested by Wei Liu)
- use assert to check input lvl.
(suggested by Roger Pau Monné)
v5:
- directly define 'xc_psr_hw_info' as union type.
(suggested by Roger Pau Monné)
- converge L2 and L3 cases in 'xc_psr_get_hw_info'.
(suggested by Roger Pau Monné)
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- remove 'XC_PSR_FEAT_UNKNOWN' which is not necessary.
(suggested by Roger Pau Monné)
- remove 'FEAT_' from enum item names.
(suggested by Roger Pau Monné)
- remove 'xc_' from struct name.
(suggested by Roger Pau Monné)
- adjust codes to reduce indentation.
(suggested by Roger Pau Monné)
- assert for not happened case.
(suggested by Roger Pau Monné)
- add LOGE to show errno.
(suggested by Roger Pau Monné)
v3:
- rename 'psr_cat/PSR_CAT' to 'psr_alloc/PSR_ALLOC' and remove 'op/OP'
from name.
(suggested by Roger Pau Monné)
- remove 'info' from 'xc_cat_info' and 'xc_mba_info'.
(suggested by Roger Pau Monné)
- set errno in 'xc_psr_get_hw_info'.
(suggested by Roger Pau Monné)
- remove 'inline'.
(suggested by Roger Pau Monné)
- remove 'psr' from 'libxl__psr_feat_type_to_libxc_psr_feat_type' to make
function name shorter.
(suggested by Roger Pau Monné)
- check 'xc_type' in 'libxl_psr_cat_get_info'.
(suggested by Roger Pau Monné)
v2:
- split this patch out from a big patch in v1.
(suggested by Wei Liu)
- change 'CAT_INFO' and 'MBA_INFO' to 'CAT' and 'MBA'.
(suggested by Chao Peng)
---
tools/libxc/include/xenctrl.h | 27 ++---
tools/libxc/xc_psr.c | 55 +++
tools/libxl/libxl_psr.c | 38 --
3 files changed, 95 insertions(+), 25 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index b970905..2d977c8 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2474,6 +2474,28 @@ enum xc_psr_cat_type {
};
typedef enum xc_psr_cat_type xc_psr_cat_type;
+enum xc_psr_feat_type {
+XC_PSR_CAT_L3,
+XC_PSR_CAT_L2,
+XC_PSR_MBA,
+};
+typedef enum xc_psr_feat_type xc_psr_feat_type;
+
+union xc_psr_hw_info {
+struct {
+uint32_t cos_max;
+uint32_t cbm_len;
+bool cdp_enabled;
+} cat;
+
+struct {
+uint32_t cos_max;
+uint32_t thrtl_max;
+bool linear;
+} mba;
+};
+typedef union xc_psr_hw_info xc_psr_hw_info;
+
int xc_psr_cmt_attach(xc_interface *xch, uint32_t domid);
int xc_psr_cmt_detach(xc_interface *xch, uint32_t domid);
int xc_psr_cmt_get_domain_rmid(xc_interface *xch, uint32_t domid,
@@ -2495,9 +2517,8 @@ int xc_psr_cat_set_domain_data(xc_interface *xch,
uint32_t domid,
int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
xc_psr_cat_type type, uint32_t target,
uint64_t *data);
-int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
-uint32_t *cos_max, uint32_t *cbm_len,
-bool *cdp_enabled);
+int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
+ xc_psr_feat_type type, xc_psr_hw_info *hw_info);
int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 5c54a35..2c605a7 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -323,37 +323,52 @@ int xc_psr_cat_get_domain_data(xc_interface *xch,
uint32_t domid,
return rc;
}
-int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
-uint32_t *cos_max, uint32_t *cbm_len, bool
*cdp_enabled)
+int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
+ xc_psr_feat_type type, xc_psr_hw_info *hw_info)
{
int rc = -1;
DECLARE_SYSCTL;
+if ( !hw_info )
+{
+errno = EINVAL;
+return rc;
+}
+
sysctl.cmd = XEN_SYSCTL_psr_alloc;
sysctl.u.psr_alloc.target = socket;
-switch ( lvl )
+switch ( type )
{
-case 2:
-sysctl.u.psr_alloc.cmd = XEN_SYSCTL_PSR_get_l2_info;
+case XC_PSR_CAT_L2:
+case XC_PSR_CAT_L3:
+
[Xen-devel] [PATCH v7 04/16] x86: a few optimizations to psr codes
This patch refines psr codes:
1. Change type of 'cat_init_feature' to 'bool' to remove the pointless
returning of error code.
2. Move printk in 'cat_init_feature' to reduce a return path.
3. Define a local variable 'feat_mask' in 'psr_cpu_init' to reduce calling of
'cpuid_count_leaf()'.
4. Change 'PSR_INFO_IDX_CAT_FLAG' to 'PSR_INFO_IDX_CAT_FLAGS'.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Jan Beulich
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v7:
- adjust the check to 'cat_init_feature' in 'psr_cpu_init'.
(suggested by Jan Beulich)
- change 'PSR_INFO_IDX_CAT_FLAG' to 'PSR_INFO_IDX_CAT_FLAGS'.
(suggested by Jan Beulich)
v6:
- restore 'write_msr()' type to 'void'.
(suggested by Jan Beulich and Roger Pau Monné)
- change 'ebx' in 'psr_cpu_init' to 'feat_mask'.
(suggested by Jan Beulich and Roger Pau Monné)
v5:
- create this patch to make codes clearer.
(suggested by Jan Beulich and Roger Pau Monné)
---
xen/arch/x86/psr.c| 45 ++---
xen/arch/x86/sysctl.c | 4 ++--
xen/include/asm-x86/psr.h | 2 +-
3 files changed, 25 insertions(+), 26 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 6dce823..50c5a98 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -273,10 +273,10 @@ static bool psr_check_cbm(unsigned int cbm_len, unsigned
long cbm)
}
/* CAT common functions implementation. */
-static int cat_init_feature(const struct cpuid_leaf *regs,
-struct feat_node *feat,
-struct psr_socket_info *info,
-enum psr_feat_type type)
+static bool cat_init_feature(const struct cpuid_leaf *regs,
+ struct feat_node *feat,
+ struct psr_socket_info *info,
+ enum psr_feat_type type)
{
const char *const cat_feat_name[FEAT_TYPE_NUM] = {
[FEAT_TYPE_L3_CAT] = "L3 CAT",
@@ -286,7 +286,7 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
/* No valid value so do not enable feature. */
if ( !regs->a || !regs->d )
-return -ENOENT;
+return false;
feat->cbm_len = (regs->a & CAT_CBM_LEN_MASK) + 1;
feat->cos_max = min(opt_cos_max, regs->d & CAT_COS_MAX_MASK);
@@ -296,7 +296,7 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
case FEAT_TYPE_L3_CAT:
case FEAT_TYPE_L2_CAT:
if ( feat->cos_max < 1 )
-return -ENOENT;
+return false;
/* We reserve cos=0 as default cbm (all bits within cbm_len are 1). */
feat->cos_reg_val[0] = cat_default_val(feat->cbm_len);
@@ -313,7 +313,7 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
uint64_t val;
if ( feat->cos_max < 3 )
-return -ENOENT;
+return false;
/* Cut half of cos_max when CDP is enabled. */
feat->cos_max = (feat->cos_max - 1) >> 1;
@@ -332,20 +332,18 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
}
default:
-return -ENOENT;
+return false;
}
/* Add this feature into array. */
info->features[type] = feat;
-if ( !opt_cpu_info )
-return 0;
-
-printk(XENLOG_INFO "%s: enabled on socket %u, cos_max:%u, cbm_len:%u\n",
- cat_feat_name[type], cpu_to_socket(smp_processor_id()),
- feat->cos_max, feat->cbm_len);
+if ( opt_cpu_info )
+printk(XENLOG_INFO "%s: enabled on socket %u, cos_max:%u,
cbm_len:%u\n",
+ cat_feat_name[type], cpu_to_socket(smp_processor_id()),
+ feat->cos_max, feat->cbm_len);
-return 0;
+return true;
}
static bool cat_get_feat_info(const struct feat_node *feat,
@@ -356,7 +354,7 @@ static bool cat_get_feat_info(const struct feat_node *feat,
data[PSR_INFO_IDX_COS_MAX] = feat->cos_max;
data[PSR_INFO_IDX_CAT_CBM_LEN] = feat->cbm_len;
-data[PSR_INFO_IDX_CAT_FLAG] = 0;
+data[PSR_INFO_IDX_CAT_FLAGS] = 0;
return true;
}
@@ -383,7 +381,7 @@ static bool l3_cdp_get_feat_info(const struct feat_node
*feat,
if ( !cat_get_feat_info(feat, data, array_len) )
return false;
-data[PSR_INFO_IDX_CAT_FLAG] |= XEN_SYSCTL_PSR_CAT_L3_CDP;
+data[PSR_INFO_IDX_CAT_FLAGS] |= XEN_SYSCTL_PSR_CAT_L3_CDP;
return true;
}
@@ -1413,6 +1411,7 @@ static void psr_cpu_init(void)
unsigned int socket, cpu = smp_processor_id();
struct feat_node *feat;
struct cpuid_leaf regs;
+uint32_t feat_mask;
if ( !psr_alloc_feat_enabled() || !boot_cpu_ha
[Xen-devel] [PATCH v7 11/16] tools: implement the new libxl get hw info interface
This patch implements the new libxl get hw info interface,
'libxl_psr_get_hw_info', which is suitable to all psr allocation
features. It also implements corresponding list free function,
'libxl_psr_hw_info_list_free' and makes 'libxl_psr_cat_get_info' call
'libxl_psr_get_hw_info' to avoid redundant code in libxl_psr.c.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Wei Liu
---
CC: Wei Liu
CC: Ian Jackson
CC: Roger Pau Monné
CC: Chao Peng
v5:
- change 'if (rc < 0)' to 'if (rc)'.
(suggested by Roger Pau Monné)
v4:
- remove 'xc_' from struct name.
(suggested by Roger Pau Monné)
- fix words in commit message.
(suggested by Roger Pau Monné)
- change type of 'libxl__hw_info_to_libxl_cat_info' to void and use
assert to check invalid type. Then, remove check for
'libxl__hw_info_to_libxl_cat_info'.
(suggested by Roger Pau Monné)
- change type of 'libxl__xc_hw_info_to_libxl_hw_info' to void and use
assert to check invalid type. Then, remove check for
'libxl__xc_hw_info_to_libxl_hw_info'.
(suggested by Roger Pau Monné)
v3:
- remove casting.
(suggested by Roger Pau Monné)
- remove inline.
(suggested by Roger Pau Monné)
- change 'libxc__psr_hw_info_to_libxl_psr_hw_info' to
'libxl__xc_hw_info_to_libxl_hw_info'.
(suggested by Roger Pau Monné)
- remove '_hw' from parameter names.
(suggested by Roger Pau Monné)
- change some 'LOGE' to 'LOG'.
(suggested by Roger Pau Monné)
- check returned 'xc_type' and remove redundant 'lvl' check.
(suggested by Roger Pau Monné)
v2:
- split this patch out from a big patch in v1.
(suggested by Wei Liu)
- change 'CAT_INFO'/'MBA_INFO' to 'CAT' and 'MBA. Also the libxl structure
name 'cat_info'/'mba_info' is changed to 'cat'/'mba'.
(suggested by Chao Peng)
- call 'libxl_psr_hw_info_list_free' in 'libxl_psr_cat_get_info' to free
allocated resources.
(suggested by Chao Peng)
---
tools/libxl/libxl_psr.c | 131 ++--
1 file changed, 93 insertions(+), 38 deletions(-)
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index e1cc250..b053abd 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -386,56 +386,41 @@ static xc_psr_feat_type
libxl__feat_type_to_libxc_feat_type(
return xc_type;
}
+static void libxl__hw_info_to_libxl_cat_info(
+libxl_psr_feat_type type, libxl_psr_hw_info *hw_info,
+libxl_psr_cat_info *cat_info)
+{
+assert(type == LIBXL_PSR_FEAT_TYPE_CAT);
+
+cat_info->id = hw_info->id;
+cat_info->cos_max = hw_info->u.cat.cos_max;
+cat_info->cbm_len = hw_info->u.cat.cbm_len;
+cat_info->cdp_enabled = hw_info->u.cat.cdp_enabled;
+}
+
int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
unsigned int *nr, unsigned int lvl)
{
GC_INIT(ctx);
int rc;
-int i = 0, socketid, nr_sockets;
-libxl_bitmap socketmap;
+unsigned int i;
+libxl_psr_hw_info *hw_info;
libxl_psr_cat_info *ptr;
-xc_psr_hw_info hw_info;
-xc_psr_feat_type xc_type;
-
-libxl_bitmap_init(&socketmap);
-
-rc = libxl__count_physical_sockets(gc, &nr_sockets);
-if (rc) {
-LOGE(ERROR, "failed to get system socket count");
-goto out;
-}
-libxl_socket_bitmap_alloc(ctx, &socketmap, nr_sockets);
-rc = libxl_get_online_socketmap(ctx, &socketmap);
-if (rc < 0) {
-LOGE(ERROR, "failed to get available sockets");
+rc = libxl_psr_get_hw_info(ctx, LIBXL_PSR_FEAT_TYPE_CAT, lvl, nr,
&hw_info);
+if (rc)
goto out;
-}
-
-xc_type = libxl__feat_type_to_libxc_feat_type(LIBXL_PSR_FEAT_TYPE_CAT,
lvl);
-
-ptr = libxl__malloc(NOGC, nr_sockets * sizeof(libxl_psr_cat_info));
-
-libxl_for_each_set_bit(socketid, socketmap) {
-ptr[i].id = socketid;
-if (xc_psr_get_hw_info(ctx->xch, socketid, xc_type, &hw_info)) {
-LOGE(ERROR, "failed to get hw info");
-rc = ERROR_FAIL;
-free(ptr);
-goto out;
-}
-ptr[i].cos_max = hw_info.cat.cos_max;
-ptr[i].cbm_len = hw_info.cat.cbm_len;
-ptr[i].cdp_enabled = hw_info.cat.cdp_enabled;
+ptr = libxl__malloc(NOGC, *nr * sizeof(libxl_psr_cat_info));
-i++;
-}
+for (i = 0; i < *nr; i++)
+libxl__hw_info_to_libxl_cat_info(LIBXL_PSR_FEAT_TYPE_CAT,
+ &hw_info[i],
+ &ptr[i])
[Xen-devel] [PATCH v7 05/16] x86: implement data structure and CPU init flow for MBA
This patch implements main data structures of MBA.
Like CAT features, MBA HW info has cos_max which means the max thrtl
register number, and thrtl_max which means the max throttle value
(delay value). It also has a flag to represent if the throttle
value is linear or non-linear.
One thrtl register of MBA stores a throttle value for one or more
domains. The throttle value means the delay applied to traffic between
L2 cache and next cache level.
This patch also implements init flow for MBA and register stub
callback functions.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v7:
- modify commit message.
(suggested by Jan Beulich)
- move the changes about check of 'cat_init_feature' in 'psr_cpu_init'
into previous patch.
(suggested by Jan Beulich)
v6:
- restore type of 'mba_write_msr' to 'void'.
v5:
- move out some CAT codes optimization to a new patch.
(suggested by Jan Beulich)
- modify commit message.
(suggested by Jan Beulich)
- change print type of 'linear' to be %d.
(suggested by Jan Beulich)
- change type of 'mba_write_msr' to uint32_t.
- move printk in 'mba_init_feature' to reduce one return path.
(suggested by Roger Pau Monné)
- move the MBA format string in printk to a new line.
(suggested by Roger Pau Monné)
v4:
- modify commit message.
(suggested by Roger Pau Monné)
- fix a comment.
(suggested by Roger Pau Monné)
- join two checks in a single if.
(suggested by Roger Pau Monné)
- remove redundant initialization of 'feat->cos_reg_val[0]'.
(suggested by Roger Pau Monné)
- change 'reg_b' to 'ebx'.
(suggested by Jan Beulich)
- change type of 'mba_init_feature' from 'int' to 'bool'.
(suggested by Roger Pau Monné)
- change type of 'cat_init_feature' from 'int' to 'bool'.
v3:
- replace 'psr_val_type' to 'psr_type'. Also, change 'PSR_VAL_TYPE_MBA' to
'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
- replace 'MBA_LINEAR' to 'MBA_LINEAR_MASK' to make the name more clear.
(suggested by Roger Pau Monné)
- replase 'cat_info'/'mba_info' to 'cat'/'mba' to make the names shorter.
(suggested by Roger Pau Monné)
- change type of 'linear' to 'bool'.
(suggested by Roger Pau Monné)
- make format string of printf in one line.
(suggested by Roger Pau Monné)
v2:
- modify commit message to replace 'cos register' to 'thrtl register' to
make it accurate.
(suggested by Chao Peng)
- restore the place of the sentence to assign value to 'feat->cbm_len'
because the MBA init flow is splitted out as a separate function in v1.
(suggested by Chao Peng)
- add comment to explain what the MBA thrtl defaul value '0' stands for.
(suggested by Chao Peng)
- check 'thrtl_max' under linear mode. It could not be euqal or larger than
100.
(suggested by Chao Peng)
v1:
- rebase codes onto L2 CAT v15.
- move comment to appropriate place.
(suggested by Chao Peng)
- implement 'mba_init_feature' and keep 'cat_init_feature'.
(suggested by Chao Peng)
- keep 'regs.b' into a local variable to avoid reading CPUID every time.
(suggested by Chao Peng)
---
xen/arch/x86/psr.c | 129 +++-
xen/include/asm-x86/msr-index.h | 1 +
xen/include/asm-x86/psr.h | 2 +
3 files changed, 116 insertions(+), 16 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 50c5a98..64d30b9 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -27,13 +27,16 @@
* - CMT Cache Monitoring Technology
* - COS/CLOSClass of Service. Also mean COS registers.
* - COS_MAX Max number of COS for the feature (minus 1)
+ * - MBA Memory Bandwidth Allocation
* - MSRsMachine Specific Registers
* - PSR Intel Platform Shared Resource
+ * - THRTL_MAX Max throttle value (delay value) of MBA
*/
#define PSR_CMT(1u << 0)
#define PSR_CAT(1u << 1)
#define PSR_CDP(1u << 2)
+#define PSR_MBA(1u << 3)
#define CAT_CBM_LEN_MASK 0x1f
#define CAT_COS_MAX_MASK 0x
@@ -60,10 +63,14 @@
*/
#define MAX_COS_NUM 2
+#define MBA_LINEAR_MASK(1u << 2)
+#define MBA_THRTL_MAX_MASK 0xfff
+
enum psr_feat_type {
FEAT_TYPE_L3_CAT,
FEAT_TYPE_L3_CDP,
FEAT_TYPE_L2_CAT,
+FEAT_TYPE_MBA,
FEAT_TYPE_NUM,
FEAT_TYPE_UNKNOWN,
};
@@ -71,7 +78,6
[Xen-devel] [PATCH v7 12/16] tools: implement the new xl get hw info interface
This patch implements a new xl get HW info interface. A new argument
is added for psr-hwinfo command to get and show MBA HW info.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Wei Liu
---
CC: Wei Liu
CC: Ian Jackson
CC: Roger Pau Monné
CC: Chao Peng
v3:
- change the format string of printf in 'psr_mba_hwinfo'.
(suggested by Roger Pau Monné)
- add 'const' for 'opts[]' in 'main_psr_hwinfo'.
(suggested by Roger Pau Monné)
v2:
- split out this patch from a big patch in v1.
(suggested by Wei Liu)
- change 'MBA_INFO' to 'MBA'. Also, change 'mba_info' to 'mba'.
(suggested by Chao Peng)
---
tools/xl/xl_cmdtable.c | 1 +
tools/xl/xl_psr.c | 39 ---
2 files changed, 37 insertions(+), 3 deletions(-)
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index c304a85..dbbfc02 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -549,6 +549,7 @@ struct cmd_spec cmd_table[] = {
"[options]",
"-m, --cmt Show Cache Monitoring Technology (CMT) hardware info\n"
"-a, --cat Show Cache Allocation Technology (CAT) hardware info\n"
+ "-b, --mba Show Memory Bandwidth Allocation (MBA) hardware info\n"
},
{ "psr-cmt-attach",
&main_psr_cmt_attach, 0, 1,
diff --git a/tools/xl/xl_psr.c b/tools/xl/xl_psr.c
index ef00048..ab47d96 100644
--- a/tools/xl/xl_psr.c
+++ b/tools/xl/xl_psr.c
@@ -475,6 +475,31 @@ static int psr_l2_cat_hwinfo(void)
return rc;
}
+static int psr_mba_hwinfo(void)
+{
+int rc;
+unsigned int i, nr;
+libxl_psr_hw_info *info;
+
+rc = libxl_psr_get_hw_info(ctx, LIBXL_PSR_FEAT_TYPE_MBA, 0, &nr, &info);
+if (rc)
+return rc;
+
+printf("Memory Bandwidth Allocation (MBA):\n");
+
+for (i = 0; i < nr; i++) {
+printf("Socket ID : %u\n", info[i].id);
+printf("Linear Mode : %s\n",
+ info[i].u.mba.linear ? "Enabled" : "Disabled");
+printf("Maximum COS : %u\n", info[i].u.mba.cos_max);
+printf("Maximum Throttling Value: %u\n", info[i].u.mba.thrtl_max);
+printf("Default Throttling Value: %u\n", 0);
+}
+
+libxl_psr_hw_info_list_free(info, nr);
+return rc;
+}
+
int main_psr_cat_cbm_set(int argc, char **argv)
{
uint32_t domid;
@@ -593,20 +618,24 @@ int main_psr_cat_show(int argc, char **argv)
int main_psr_hwinfo(int argc, char **argv)
{
int opt, ret = 0;
-bool all = true, cmt = false, cat = false;
-static struct option opts[] = {
+bool all = true, cmt = false, cat = false, mba = false;
+static const struct option opts[] = {
{"cmt", 0, 0, 'm'},
{"cat", 0, 0, 'a'},
+{"mba", 0, 0, 'b'},
COMMON_LONG_OPTS
};
-SWITCH_FOREACH_OPT(opt, "ma", opts, "psr-hwinfo", 0) {
+SWITCH_FOREACH_OPT(opt, "mab", opts, "psr-hwinfo", 0) {
case 'm':
all = false; cmt = true;
break;
case 'a':
all = false; cat = true;
break;
+case 'b':
+all = false; mba = true;
+break;
}
if (!ret && (all || cmt))
@@ -619,6 +648,10 @@ int main_psr_hwinfo(int argc, char **argv)
if (all || cat)
ret = psr_l2_cat_hwinfo();
+/* MBA is independent of CMT and CAT */
+if (all || mba)
+ret = psr_mba_hwinfo();
+
return ret;
}
--
1.9.1
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v7 03/16] x86: rename 'cbm_type' to 'psr_type' to make it general
This patch renames 'cbm_type' to 'psr_type' to generalize it.
Then, we can reuse this for all psr allocation features.
Signed-off-by: Yi Sun
Reviewed-by: Wei Liu
Reviewed-by: Roger Pau Monné
Acked-by: Jan Beulich
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v5:
- correct character of reviewer's name.
(suggested by Jan Beulich)
v4:
- fix words in commit message.
(suggested by Roger Pau Monné)
v3:
- replace 'psr_val_type' to 'psr_type' and remove '_VAL' from the enum
items.
(suggested by Roger Pau Monné)
v2:
- replace 'PSR_VAL_TYPE_{L3, L2}' to 'PSR_VAL_TYPE_{L3, L2}_CBM'.
(suggested by Chao Peng)
---
xen/arch/x86/domctl.c | 16 ++--
xen/arch/x86/psr.c| 62 +--
xen/arch/x86/sysctl.c | 4 +--
xen/include/asm-x86/psr.h | 18 +++---
4 files changed, 52 insertions(+), 48 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 6a9376f..ae06627 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1444,25 +1444,25 @@ long arch_do_domctl(
case XEN_DOMCTL_PSR_SET_L3_CBM:
ret = psr_set_val(d, domctl->u.psr_alloc.target,
domctl->u.psr_alloc.data,
- PSR_CBM_TYPE_L3);
+ PSR_TYPE_L3_CBM);
break;
case XEN_DOMCTL_PSR_SET_L3_CODE:
ret = psr_set_val(d, domctl->u.psr_alloc.target,
domctl->u.psr_alloc.data,
- PSR_CBM_TYPE_L3_CODE);
+ PSR_TYPE_L3_CODE);
break;
case XEN_DOMCTL_PSR_SET_L3_DATA:
ret = psr_set_val(d, domctl->u.psr_alloc.target,
domctl->u.psr_alloc.data,
- PSR_CBM_TYPE_L3_DATA);
+ PSR_TYPE_L3_DATA);
break;
case XEN_DOMCTL_PSR_SET_L2_CBM:
ret = psr_set_val(d, domctl->u.psr_alloc.target,
domctl->u.psr_alloc.data,
- PSR_CBM_TYPE_L2);
+ PSR_TYPE_L2_CBM);
break;
#define domctl_psr_get_val(d, domctl, type, copyback) ({\
@@ -1476,19 +1476,19 @@ long arch_do_domctl(
})
case XEN_DOMCTL_PSR_GET_L3_CBM:
-ret = domctl_psr_get_val(d, domctl, PSR_CBM_TYPE_L3, copyback);
+ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L3_CBM, copyback);
break;
case XEN_DOMCTL_PSR_GET_L3_CODE:
-ret = domctl_psr_get_val(d, domctl, PSR_CBM_TYPE_L3_CODE,
copyback);
+ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L3_CODE, copyback);
break;
case XEN_DOMCTL_PSR_GET_L3_DATA:
-ret = domctl_psr_get_val(d, domctl, PSR_CBM_TYPE_L3_DATA,
copyback);
+ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L3_DATA, copyback);
break;
case XEN_DOMCTL_PSR_GET_L2_CBM:
-ret = domctl_psr_get_val(d, domctl, PSR_CBM_TYPE_L2, copyback);
+ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L2_CBM, copyback);
break;
#undef domctl_psr_get_val
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 11c204e..6dce823 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -100,24 +100,24 @@ static const struct feat_props {
unsigned int cos_num;
/*
- * An array to save all 'enum cbm_type' values of the feature. It is
+ * An array to save all 'enum psr_type' values of the feature. It is
* used with cos_num together to get/write a feature's COS registers
* values one by one.
*/
-enum cbm_type type[MAX_COS_NUM];
+enum psr_type type[MAX_COS_NUM];
/*
* alt_type is 'alternative type'. When this 'alt_type' is input, the
* feature does some special operations.
*/
-enum cbm_type alt_type;
+enum psr_type alt_type;
/* get_feat_info is used to return feature HW info through sysctl. */
bool (*get_feat_info)(const struct feat_node *feat,
uint32_t data[], unsigned int array_len);
/* write_msr is used to write out feature MSR register. */
-void (*write_msr)(unsigned int cos, uint32_t val, enum cbm_type type);
+void (*write_msr)(unsigned int cos, uint32_t val, enum psr_type type);
} *feat_props[FEAT_TYPE_NUM];
/*
@@ -215,13 +215,13 @@ static void free_socket_resources(unsigned int socket)
bitmap_zero(info->dom_set, DOMID_IDLE + 1);
}
-static enum psr_feat_type psr_cbm_type_to_feat_type(enum cbm_type type)
+static enum psr_feat_type psr_type_to_feat_type(enum psr_type type)
{
enum psr_feat_type feat_
[Xen-devel] [PATCH v7 00/16] Enable Memory Bandwidth Allocation in Xen
Hi, all, We plan to bring a new PSR (Platform Shared Resource) feature called Intel Memory Bandwidth Allocation (MBA) to Xen. Besides the MBA enabling, we change some interfaces to make them more general but not only for CAT. Any comments are welcome! You can find this series at: https://github.com/yisun-git/xen_mba mba_v7 This version bases on below pre-fix patch which has been merged into staging branch: "x86: psr: support co-exist features' values setting" https://lists.xen.org/archives/html/xen-devel/2017-10/msg00866.html CC: Jan Beulich CC: Andrew Cooper CC: Wei Liu CC: Ian Jackson CC: Daniel De Graaf CC: Roger Pau Monné CC: Konrad Rzeszutek Wilk CC: Chao Peng CC: Julien Grall --- Acked and Reviewed list before V7: a - Acked-by r - Reviewed-by r patch 1 - docs: create Memory Bandwidth Allocation (MBA) feature document ar patch 2 - Rename PSR sysctl/domctl interfaces and xsm policy to make them be general ar patch 3 - x86: rename 'cbm_type' to 'psr_type' to make it general ar patch 4 - x86: a few optimizations to psr codes r patch 5 - x86: implement data structure and CPU init flow for MBA ar patch 6 - x86: implement get hw info flow for MBA ar patch 7 - x86: implement get value interface for MBA ar patch 9 - tools: create general interfaces to support psr allocation features ar patch 10 - tools: implement the new libxc get hw info interface ar patch 11 - tools: implement the new libxl get hw info interface ar patch 12 - tools: implement the new xl get hw info interface ar patch 13 - tools: rename 'xc_psr_cat_type' to 'xc_psr_type' ar patch 14 - tools: implement new generic get value interface and MBA get value command ar patch 15 - tools: implement new generic set value interface and MBA set value command ar patch 16 - docs: add MBA description in docs --- V7 change list: Patch 2: - add single trailing underscore for internal variabled in macro. (suggested by Jan Beulich) - add parentheses for input parameters of marcro. (suggested by Jan Beulich) - adjust the postion of macro. (suggested by Jan Beulich) - rebase to latest codes. Patch 4: - adjust the check to 'cat_init_feature' in 'psr_cpu_init'. (suggested by Jan Beulich) - change 'PSR_INFO_IDX_CAT_FLAG' to 'PSR_INFO_IDX_CAT_FLAGS'. (suggested by Jan Beulich) Patch 5: - modify commit message. (suggested by Jan Beulich) - move the changes about check of 'cat_init_feature' in 'psr_cpu_init' into previous patch. (suggested by Jan Beulich) Patch 6: - change 'PSR_INFO_IDX_MBA_FLAG' to 'PSR_INFO_IDX_MBA_FLAGS'. (suggested by Jan Beulich) Patch 8: - change name of 'check_val' to 'sanitize'. (suggested by Jan Beulich) - fix comments. (suggested by Jan Beulich) - add parentheses and change '== 0' to '!'. (suggested by Jan Beulich) - remove unnecessary check of 'mba.thrtl_max'. (suggested by Jan Beulich) - remove unnecessary intermediate variable 'mod'. (suggested by Jan Beulich) - refine an assignement sentence to use '&='. (suggested by Jan Beulich) - change type of last parameter of 'sanitize' to 'uint32_t' and apply same change to 'cat_check_cbm'. (suggested by Jan Beulich) Yi Sun (16): docs: create Memory Bandwidth Allocation (MBA) feature document Rename PSR sysctl/domctl interfaces and xsm policy to make them be general x86: rename 'cbm_type' to 'psr_type' to make it general x86: a few optimizations to psr codes x86: implement data structure and CPU init flow for MBA x86: implement get hw info flow for MBA x86: implement get value interface for MBA x86: implement set value flow for MBA tools: create general interfaces to support psr allocation features tools: implement the new libxc get hw info interface tools: implement the new libxl get hw info interface tools: implement the new xl get hw info interface tools: rename 'xc_psr_cat_type' to 'xc_psr_type' tools: implement new generic get value interface and MBA get value command tools: implement new generic set value interface and MBA set value command docs: add MBA description in docs docs/features/intel_psr_mba.pandoc | 297 +++ docs/man/xl.pod.1.in| 33 docs/misc/xl-psr.markdown | 62 tools/flask/policy/modules/dom0.te | 4 +- tools/libxc/include/xenctrl.h | 44 -- tools/libxc/xc_psr.c| 109 +++-- tools/libxl/libxl.h | 37 + tools/libxl/libxl_psr.c | 223 +-- tools/lib
[Xen-devel] [PATCH v7 02/16] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
This patch renames PSR sysctl/domctl interfaces and related xsm policy to
make them be general for all resource allocation features but not only
for CAT. Then, we can resuse the interfaces for all allocation features.
Basically, it changes 'psr_cat_op' to 'psr_alloc', and remove 'CAT_' from some
macros. E.g.:
1. psr_cat_op -> psr_alloc
2. XEN_DOMCTL_psr_cat_op -> XEN_DOMCTL_psr_alloc
3. XEN_SYSCTL_psr_cat_op -> XEN_SYSCTL_psr_alloc
4. XEN_DOMCTL_PSR_CAT_SET_L3_CBM -> XEN_DOMCTL_PSR_SET_L3_CBM
5. XEN_SYSCTL_PSR_CAT_get_l3_info -> XEN_SYSCTL_PSR_get_l3_info
Signed-off-by: Yi Sun
Reviewed-by: Wei Liu
Reviewed-by: Roger Pau Monné
Acked-by: Jan Beulich
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Ian Jackson
CC: Daniel De Graaf
CC: Roger Pau Monné
CC: Chao Peng
v7:
- add single trailing underscore for internal variabled in macro.
(suggested by Jan Beulich)
- add parentheses for input parameters of marcro.
(suggested by Jan Beulich)
- adjust the postion of macro.
(suggested by Jan Beulich)
- rebase to latest codes.
v6:
- move macro definition into the function and undefine it after use.
(suggested by Roger Pau Monné)
- do not bump sysctl version because it has been bumped for 4.10.
(suggested by Roger Pau Monné)
v5:
- remove domctl version number upgrade.
(suggested by Jan Beulich)
- restore 'XEN_SYSCTL_PSR_CAT_L3_CDP'.
(suggested by Jan Beulich)
- define a local macro to complete psr get value flow.
(suggested by Roger Pau Monné)
- remove 'Reviewed-by' and 'Acked-by'.
(suggested by Wei Liu)
v4:
- remove 'ALLOC_' from names.
(suggested by Roger Pau Monné)
- fix comments.
(suggested by Roger Pau Monné)
v3:
- remove 'op/OP' from names and modify some names from 'PSR_CAT' to
'PSR_ALLOC'.
(suggested by Roger Pau Monné)
v1:
- add description about what to be changed in commit message.
(suggested by Wei Liu)
- bump sysctl/domctl version numbers.
(suggested by Wei Liu)
---
tools/flask/policy/modules/dom0.te | 4 +--
tools/libxc/xc_psr.c| 50 +-
xen/arch/x86/domctl.c | 71 ++---
xen/arch/x86/sysctl.c | 28 +++
xen/include/public/domctl.h | 24 ++---
xen/include/public/sysctl.h | 12 +++
xen/xsm/flask/hooks.c | 8 ++---
xen/xsm/flask/policy/access_vectors | 8 ++---
8 files changed, 102 insertions(+), 103 deletions(-)
diff --git a/tools/flask/policy/modules/dom0.te
b/tools/flask/policy/modules/dom0.te
index 1643b40..07de3d5 100644
--- a/tools/flask/policy/modules/dom0.te
+++ b/tools/flask/policy/modules/dom0.te
@@ -14,7 +14,7 @@ allow dom0_t xen_t:xen {
tmem_control getscheduler setscheduler
};
allow dom0_t xen_t:xen2 {
- resource_op psr_cmt_op psr_cat_op pmu_ctrl get_symbol
+ resource_op psr_cmt_op psr_alloc pmu_ctrl get_symbol
get_cpu_levelling_caps get_cpu_featureset livepatch_op
gcov_op set_parameter
};
@@ -39,7 +39,7 @@ allow dom0_t dom0_t:domain {
};
allow dom0_t dom0_t:domain2 {
set_cpuid gettsc settsc setscheduler set_max_evtchn set_vnumainfo
- get_vnumainfo psr_cmt_op psr_cat_op set_gnttab_limits
+ get_vnumainfo psr_cmt_op psr_alloc set_gnttab_limits
};
allow dom0_t dom0_t:resource { add remove };
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 039b920..5c54a35 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -258,27 +258,27 @@ int xc_psr_cat_set_domain_data(xc_interface *xch,
uint32_t domid,
switch ( type )
{
case XC_PSR_CAT_L3_CBM:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM;
+cmd = XEN_DOMCTL_PSR_SET_L3_CBM;
break;
case XC_PSR_CAT_L3_CBM_CODE:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_CODE;
+cmd = XEN_DOMCTL_PSR_SET_L3_CODE;
break;
case XC_PSR_CAT_L3_CBM_DATA:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_DATA;
+cmd = XEN_DOMCTL_PSR_SET_L3_DATA;
break;
case XC_PSR_CAT_L2_CBM:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM;
+cmd = XEN_DOMCTL_PSR_SET_L2_CBM;
break;
default:
errno = EINVAL;
return -1;
}
-domctl.cmd = XEN_DOMCTL_psr_cat_op;
+domctl.cmd = XEN_DOMCTL_psr_alloc;
domctl.domain = (domid_t)domid;
-domctl.u.psr_cat_op.cmd = cmd;
-domctl.u.psr_cat_op.target = target;
-domctl.u.psr_cat_op.data = data;
+domctl.u.psr_alloc.cmd = cmd;
+domctl.u.psr_alloc.target = target;
+domctl.u.psr_alloc.data = data;
return do_domctl(xch, &domctl);
}
@@ -294,31 +294,31 @@ int xc_psr_cat_get_domain_data(xc_interface *xch,
uint32_t domid,
switch ( type )
{
case XC_PSR_CAT_L3_CB
Re: [Xen-devel] [PATCH v6 08/16] x86: implement set value flow for MBA
On 17-10-12 03:43:26, Jan Beulich wrote:
> >>> On 12.10.17 at 06:33, wrote:
> > On 17-10-11 07:38:52, Jan Beulich wrote:
> >> >>> On 08.10.17 at 09:23, wrote:
> >> > --- a/xen/arch/x86/psr.c
> >> > +++ b/xen/arch/x86/psr.c
> >> > @@ -138,6 +138,12 @@ static const struct feat_props {
> >> >
> >> > /* write_msr is used to write out feature MSR register. */
> >> > void (*write_msr)(unsigned int cos, uint32_t val, enum psr_type
> >> > type);
> >> > +
> >> > +/*
> >> > + * check_val is used to check if input val fulfills SDM requirement.
> >> > + * Change it to valid value if SDM allows.
> >> > + */
> >> > +bool (*check_val)(const struct feat_node *feat, unsigned long *val);
> >>
> >> I'm pretty sure I've said so before - "check" to me implies all r/o
> >> inputs. Perhaps sanitize_val() or even just sanitize()?
> >>
> >> And why unsigned long when the only caller has a uint32_t in its
> >> hands?
> >>
> > To be compatible with cat_check_cbm (old name is 'psr_check_cbm' in L2
> > series),
> > the last parameter type is 'unsigned long'. We have discussed it in L2
> > patch set
> > v9, patch 10.
>
> Iirc (without checking the old thread) this was for calculations to
> be done as unsigned long ones. If that's the only aspect here,
> then imo this is not a valid reason for the hook's parameter type
> to be unsigned long *.
>
Because below macros used in cat_check_cbm require the input addr to be unsigned
long, we define the last parameter of cat_check_cbm to be unsigned long.
find_first_bit
find_next_zero_bit
find_next_bit
If you think the unsigned long is not appropriate for 'check_val', I think I
have to define a local variable in cat_check_cbm to do the convertion.
> Jan
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v6 08/16] x86: implement set value flow for MBA
On 17-10-11 07:38:52, Jan Beulich wrote:
> >>> On 08.10.17 at 09:23, wrote:
> > --- a/xen/arch/x86/psr.c
> > +++ b/xen/arch/x86/psr.c
> > @@ -138,6 +138,12 @@ static const struct feat_props {
> >
> > /* write_msr is used to write out feature MSR register. */
> > void (*write_msr)(unsigned int cos, uint32_t val, enum psr_type type);
> > +
> > +/*
> > + * check_val is used to check if input val fulfills SDM requirement.
> > + * Change it to valid value if SDM allows.
> > + */
> > +bool (*check_val)(const struct feat_node *feat, unsigned long *val);
>
> I'm pretty sure I've said so before - "check" to me implies all r/o
> inputs. Perhaps sanitize_val() or even just sanitize()?
>
> And why unsigned long when the only caller has a uint32_t in its
> hands?
>
To be compatible with cat_check_cbm (old name is 'psr_check_cbm' in L2 series),
the last parameter type is 'unsigned long'. We have discussed it in L2 patch set
v9, patch 10.
> > @@ -274,30 +280,30 @@ static enum psr_feat_type psr_type_to_feat_type(enum
> > psr_type type)
> > return feat_type;
> > }
> >
> > -static bool psr_check_cbm(unsigned int cbm_len, unsigned long cbm)
> > +/* Implementation of allocation features' functions. */
> > +static bool cat_check_cbm(const struct feat_node *feat, unsigned long *cbm)
> > {
> > unsigned int first_bit, zero_bit;
> > +unsigned int cbm_len = feat->cat.cbm_len;
> >
> > -/* Set bits should only in the range of [0, cbm_len]. */
> > -if ( cbm & (~0ul << cbm_len) )
> > -return false;
> > -
> > -/* At least one bit need to be set. */
> > -if ( cbm == 0 )
> > +/*
> > + * Set bits should only in the range of [0, cbm_len].
>
> As you alter the comment anyway, please also add the missing "be".
> Also - isn't the upper bound of the range exclusive, i.e. shouldn't
> this be [0, cbm_len)?
>
Thanks!
> > + * And, at least one bit need to be set.
> > + */
> > +if ( *cbm & (~0ul << cbm_len) || *cbm == 0 )
>
> Parentheses missing for the left side operand of || and if you omit
> != 0 on the left part (which I appreciate) please also use ! instead
> of == 0 on the right side.
>
Got it.
> > +static bool mba_check_thrtl(const struct feat_node *feat, unsigned long
> > *thrtl)
> > +{
> > +if ( *thrtl > feat->mba.thrtl_max )
> > +return false;
> > +
> > +/*
> > + * Per SDM (chapter "Memory Bandwidth Allocation Configuration"):
> > + * 1. Linear mode: In the linear mode the input precision is defined
> > + *as 100-(MBA_MAX). For instance, if the MBA_MAX value is 90, the
> > + *input precision is 10%. Values not an even multiple of the
> > + *precision (e.g., 12%) will be rounded down (e.g., to 10% delay
> > + *applied).
> > + * 2. Non-linear mode: Input delay values are powers-of-two from zero
> > + *to the MBA_MAX value from CPUID. In this case any values not a
> > + *power of two will be rounded down the next nearest power of two.
> > + */
> > +if ( feat->mba.linear )
> > +{
> > +unsigned int mod;
> > +
> > +if ( feat->mba.thrtl_max >= 100 )
> > +return false;
>
> Don't you check this right after collecting CPUID output? If so,
> this should be at most an ASSERT().
>
Yes, I have checked this in mba_init_feature. Will remove this check.
> > +mod = *thrtl % (100 - feat->mba.thrtl_max);
> > +*thrtl -= mod;
>
> Do you really need the intermediate variable?
>
Will remove 'mod'.
> > +}
> > +else
> > +{
> > +/* Not power of 2. */
> > +if ( *thrtl & (*thrtl - 1) )
> > +*thrtl = *thrtl & (1 << (flsl(*thrtl) - 1));
>
> &= or even simply =.
>
Ok, thanks!
> > @@ -950,6 +997,7 @@ static int insert_val_into_array(uint32_t val[],
> > const struct feat_node *feat;
> > const struct feat_props *props;
> > unsigned int i;
> > +unsigned long check_val = new_val;
> > int ret;
> >
> > ASSERT(feat_type < FEAT_TYPE_NUM);
> > @@ -974,9 +1022,11 @@ static int insert_val_into_array(uint32_t val[],
> > if ( array_len < props->cos_num )
> > return -ENOSPC;
> >
> > -if ( !psr_check_cbm(feat->cat.cbm_len, new_val) )
> > +if ( !props->check_val(feat, &check_val) )
> > return -EINVAL;
> >
> > +new_val = check_val;
>
> When the function pointer's parameter changes to uint32_t *
> you won't need the intermediate variable anymore afaict.
>
> Jan
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v5] x86: psr: support co-exist features' values setting
Many thanks for the changes! The changes look good to me and pass the test.
On 17-10-11 06:06:49, Jan Beulich wrote:
> >>> On 11.10.17 at 09:20, wrote:
> > --- a/xen/arch/x86/psr.c
> > +++ b/xen/arch/x86/psr.c
> > @@ -,25 +,43 @@ static unsigned int get_socket_cpu(unsigned int
> > socket)
> > struct cos_write_info
> > {
> > unsigned int cos;
> > -struct feat_node *feature;
> > const uint32_t *val;
> > -const struct feat_props *props;
> > +unsigned int array_len;
> > };
>
> The addition wants to go into the hole after "cos".
>
> > static void do_write_psr_msrs(void *data)
> > {
> > -const struct cos_write_info *info = data;
> > -struct feat_node *feat = info->feature;
> > -const struct feat_props *props = info->props;
> > -unsigned int i, cos = info->cos, cos_num = props->cos_num;
> > +struct cos_write_info *info = data;
>
> const
>
> > +unsigned int i, index = 0, cos = info->cos;
> > +struct psr_socket_info *socket_info =
>
> const
>
> > +
> > get_socket_info(cpu_to_socket(smp_processor_id()));
> >
> > -for ( i = 0; i < cos_num; i++ )
> > +/*
> > + * Iterate all featuers to write different value (not same as MSR) for
> > + * each feature.
> > + */
> > +for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
> > {
> > -if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
> > +struct feat_node *feat = socket_info->features[i];
> > +const struct feat_props *props = feat_props[i];
> > +unsigned int cos_num, j;
> > +
> > +if ( !feat || !props )
> > +continue;
> > +
> > +cos_num = props->cos_num;
> > +ASSERT(info->array_len >= index + cos_num);
>
> While this transformation from the original -ENOSPC return looks to
> be correct, but not obviously so, it would have been a good idea
> to mention this in the commit message. After all the above can be
> correct only if the original -ENOSPC return path could have been
> an ASSERT() as well.
>
> > +for ( j = 0; j < cos_num; j++ )
> > {
> > -feat->cos_reg_val[cos * cos_num + i] = info->val[i];
> > -props->write_msr(cos, info->val[i], props->type[i]);
> > +if ( feat->cos_reg_val[cos * cos_num + j] != info->val[index +
> > j] )
> > +{
> > +feat->cos_reg_val[cos * cos_num + j] = info->val[index +
> > j];
> > +props->write_msr(cos, info->val[index + j],
> > props->type[j]);
> > +}
> > }
> > +
> > +index += cos_num;
>
> Looks like I only meant to comment on the uses of index above:
> If you incremented it alongside j, you could use just index in the
> respective array accesses, and you'd avoid the last statement
> above altogether.
>
> In the interest of getting the patch in I'll see to make the
> adjustments myself. Please double check the result in case I end
> up committing what I've come up with.
>
> Jan
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v5] x86: psr: support co-exist features' values setting
The whole value array is transferred into 'do_write_psr_msrs'. Then, we can
write all features values on the cos id into MSRs.
Because multiple features may co-exist, we need handle all features to write
values of them into a COS register with new COS ID. E.g:
1. L3 CAT and L2 CAT co-exist.
2. Dom1 and Dom2 share the same COS ID (2). The L3 CAT CBM of Dom1 is 0x1ff,
the L2 CAT CBM of Dom1 is 0x1f.
3. User wants to change L2 CBM of Dom1 to be 0xf. Because COS ID 2 is
used by Dom2 too, we have to pick a new COS ID 3. The values of Dom1 on
COS ID 3 are all default values as below:
-
| COS 3 |
-
L3 CAT | 0x7ff |
-
L2 CAT | 0xff |
-
4. After setting, the L3 CAT CBM value of Dom1 should be kept and the new L2
CAT CBM is set. So, the values on COS ID 3 should be below.
-
| COS 3 |
-
L3 CAT | 0x1ff |
-
L2 CAT | 0xf |
-
Signed-off-by: Yi Sun
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Julien Grall
v5:
- remove 'result' and use an ASSERT to handle error case.
(suggested by Chao Peng)
v4:
- remove init of 'result'.
(suggested by Roger Pau Monné)
- remove 'features' in 'cos_write_info' and get socket info in
'do_write_psr_msrs' to get features array.
(suggested by Jan Beulich)
- fix a typo in commit message.
(suggested by Kent R. Spillner)
v3:
- add 'result' in 'cos_write_info' to return error code.
(suggested by Roger Pau Monné)
v2:
- fix issues in commit message.
(suggested by Roger Pau Monné)
- remove unnecessary local variable 'val_array'.
(suggested by Roger Pau Monné)
---
xen/arch/x86/psr.c | 55 +-
1 file changed, 30 insertions(+), 25 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index daa2aeb..8936cf7 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -,25 +,43 @@ static unsigned int get_socket_cpu(unsigned int socket)
struct cos_write_info
{
unsigned int cos;
-struct feat_node *feature;
const uint32_t *val;
-const struct feat_props *props;
+unsigned int array_len;
};
static void do_write_psr_msrs(void *data)
{
-const struct cos_write_info *info = data;
-struct feat_node *feat = info->feature;
-const struct feat_props *props = info->props;
-unsigned int i, cos = info->cos, cos_num = props->cos_num;
+struct cos_write_info *info = data;
+unsigned int i, index = 0, cos = info->cos;
+struct psr_socket_info *socket_info =
+get_socket_info(cpu_to_socket(smp_processor_id()));
-for ( i = 0; i < cos_num; i++ )
+/*
+ * Iterate all featuers to write different value (not same as MSR) for
+ * each feature.
+ */
+for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
{
-if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
+struct feat_node *feat = socket_info->features[i];
+const struct feat_props *props = feat_props[i];
+unsigned int cos_num, j;
+
+if ( !feat || !props )
+continue;
+
+cos_num = props->cos_num;
+ASSERT(info->array_len >= index + cos_num);
+
+for ( j = 0; j < cos_num; j++ )
{
-feat->cos_reg_val[cos * cos_num + i] = info->val[i];
-props->write_msr(cos, info->val[i], props->type[i]);
+if ( feat->cos_reg_val[cos * cos_num + j] != info->val[index + j] )
+{
+feat->cos_reg_val[cos * cos_num + j] = info->val[index + j];
+props->write_msr(cos, info->val[index + j], props->type[j]);
+}
}
+
+index += cos_num;
}
}
@@ -1137,30 +1155,17 @@ static int write_psr_msrs(unsigned int socket, unsigned
int cos,
const uint32_t val[], unsigned int array_len,
enum psr_feat_type feat_type)
{
-int ret;
struct psr_socket_info *info = get_socket_info(socket);
struct cos_write_info data =
{
.cos = cos,
-.feature = info->features[feat_type],
-.props = feat_props[feat_type],
+.val = val,
+.array_len = array_len,
};
if ( cos > info->features[feat_type]->cos_max )
return -EINVAL;
-/* Skip to the feature's value head. */
-ret = skip_prior_features(&array_len, feat_type);
-if ( ret < 0 )
-return ret;
-
-val += ret;
-
-if ( array_len < feat_props[feat_type]->cos_num )
-return -ENOSPC;
-
-data.val = val;
-
if ( socket == cpu_to_socket(smp_processor_id()) )
do_write_psr_msrs(&data);
else
--
1.9.1
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v4] x86: psr: support co-exist features' values setting
On 17-10-11 14:59:23, Chao Peng wrote:
> On Wed, 2017-10-11 at 09:55 +0800, Yi Sun wrote:
> > static void do_write_psr_msrs(void *data)
> > {
> > -const struct cos_write_info *info = data;
> > -struct feat_node *feat = info->feature;
> > -const struct feat_props *props = info->props;
> > -unsigned int i, cos = info->cos, cos_num = props->cos_num;
> > +struct cos_write_info *info = data;
> > +unsigned int i, index = 0, cos = info->cos;
> > +struct psr_socket_info *socket_info =
> > +get_socket_info(cpu_to_socket(smp_process
> > or_id()));
> >
> > -for ( i = 0; i < cos_num; i++ )
> > +/*
> > + * Iterate all featuers to write different value (not same as
> > MSR) for
> > + * each feature.
> > + */
> > +for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
> > {
> > -if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
> > +struct feat_node *feat = socket_info->features[i];
> > +const struct feat_props *props = feat_props[i];
> > +unsigned int cos_num, j;
> > +
> > +if ( !feat || !props )
> > +continue;
> > +
> > +cos_num = props->cos_num;
> > +if ( info->array_len < index + cos_num )
> > +{
> > +info->result = -ENOSPC;
> > +return;
>
> This will have side effect (You may have run write_msr for some features
> already) when you return the error. It probably will not cause logic
> error here, there is performance penalty however (writing MSR and
> sending IPI).
>
> Another thing is this error is a real error that we want to propagate to
> user? E.g, I don't quite understand in which case the condition can
> happen? If this is only a program error then ASSERT can be used.
>
Thanks! If error happens, this error is a program error. So, an ASSERT here
is better.
> Chao
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v4] x86: psr: support co-exist features' values setting
The whole value array is transferred into 'do_write_psr_msrs'. Then, we can
write all features values on the cos id into MSRs.
Because multiple features may co-exist, we need handle all features to write
values of them into a COS register with new COS ID. E.g:
1. L3 CAT and L2 CAT co-exist.
2. Dom1 and Dom2 share the same COS ID (2). The L3 CAT CBM of Dom1 is 0x1ff,
the L2 CAT CBM of Dom1 is 0x1f.
3. User wants to change L2 CBM of Dom1 to be 0xf. Because COS ID 2 is
used by Dom2 too, we have to pick a new COS ID 3. The values of Dom1 on
COS ID 3 are all default values as below:
-
| COS 3 |
-
L3 CAT | 0x7ff |
-
L2 CAT | 0xff |
-
4. After setting, the L3 CAT CBM value of Dom1 should be kept and the new L2
CAT CBM is set. So, the values on COS ID 3 should be below.
-
| COS 3 |
-
L3 CAT | 0x1ff |
-
L2 CAT | 0xf |
-
Signed-off-by: Yi Sun
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Julien Grall
v4:
- remove init of 'result'.
(suggested by Roger Pau Monné)
- remove 'features' in 'cos_write_info' and get socket info in
'do_write_psr_msrs' to get features array.
(suggested by Jan Beulich)
- fix a typo in commit message.
(suggested by Kent R. Spillner)
v3:
- add 'result' in 'cos_write_info' to return error code.
(suggested by Roger Pau Monné)
v2:
- fix issues in commit message.
(suggested by Roger Pau Monné)
- remove unnecessary local variable 'val_array'.
(suggested by Roger Pau Monné)
---
xen/arch/x86/psr.c | 62 +++---
1 file changed, 36 insertions(+), 26 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index daa2aeb..a812124 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -,25 +,48 @@ static unsigned int get_socket_cpu(unsigned int socket)
struct cos_write_info
{
unsigned int cos;
-struct feat_node *feature;
const uint32_t *val;
-const struct feat_props *props;
+unsigned int array_len;
+int result;
};
static void do_write_psr_msrs(void *data)
{
-const struct cos_write_info *info = data;
-struct feat_node *feat = info->feature;
-const struct feat_props *props = info->props;
-unsigned int i, cos = info->cos, cos_num = props->cos_num;
+struct cos_write_info *info = data;
+unsigned int i, index = 0, cos = info->cos;
+struct psr_socket_info *socket_info =
+get_socket_info(cpu_to_socket(smp_processor_id()));
-for ( i = 0; i < cos_num; i++ )
+/*
+ * Iterate all featuers to write different value (not same as MSR) for
+ * each feature.
+ */
+for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
{
-if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
+struct feat_node *feat = socket_info->features[i];
+const struct feat_props *props = feat_props[i];
+unsigned int cos_num, j;
+
+if ( !feat || !props )
+continue;
+
+cos_num = props->cos_num;
+if ( info->array_len < index + cos_num )
+{
+info->result = -ENOSPC;
+return;
+}
+
+for ( j = 0; j < cos_num; j++ )
{
-feat->cos_reg_val[cos * cos_num + i] = info->val[i];
-props->write_msr(cos, info->val[i], props->type[i]);
+if ( feat->cos_reg_val[cos * cos_num + j] != info->val[index + j] )
+{
+feat->cos_reg_val[cos * cos_num + j] = info->val[index + j];
+props->write_msr(cos, info->val[index + j], props->type[j]);
+}
}
+
+index += cos_num;
}
}
@@ -1137,30 +1160,17 @@ static int write_psr_msrs(unsigned int socket, unsigned
int cos,
const uint32_t val[], unsigned int array_len,
enum psr_feat_type feat_type)
{
-int ret;
struct psr_socket_info *info = get_socket_info(socket);
struct cos_write_info data =
{
.cos = cos,
-.feature = info->features[feat_type],
-.props = feat_props[feat_type],
+.val = val,
+.array_len = array_len,
};
if ( cos > info->features[feat_type]->cos_max )
return -EINVAL;
-/* Skip to the feature's value head. */
-ret = skip_prior_features(&array_len, feat_type);
-if ( ret < 0 )
-return ret;
-
-val += ret;
-
-if ( array_len < feat_props[feat_type]->cos_num )
-return -ENOSPC;
-
-data.val = val;
-
if ( socket == cpu_to_socket(smp_processor_id()) )
[Xen-devel] [PATCH v3] x86: psr: support co-exist features' values setting
It changes the memebers in 'cos_write_info' to transfer the feature array,
feature properties array and value array. Then, we can write all features
values on the cos id into MSRs.
Because multiple features may co-exist, we need handle all features to write
values of them into a COS register with new COS ID. E.g:
1. L3 CAT and L2 CAT co-exist.
2. Dom1 and Dom2 share the same COS ID (2). The L3 CAT CBM of Dom1 is 0x1ff,
the L2 CAT CBM of Dom1 is 0x1f.
3. User wants to change L2 CBM of Dom1 to be 0xf. Because COS ID 2 is
used by Dom2 too, we have to pick a new COS ID 3. The values of Dom1 on
COS ID 3 are all default values as below:
-
| COS 3 |
-
L3 CAT | 0x7ff |
-
L2 CAT | 0xff |
-
4. After setting, the L3 CAT CBM value of Dom1 should be kept and the new L2
CAT CBM is set. So, the values on COS ID 3 should be below.
-
| COS 3 |
-
L3 CAT | 0x1ff |
-
L2 CAT | 0xf |
-
Signed-off-by: Yi Sun
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Julien Grall
v2:
- add 'result' in 'cos_write_info' to return error code.
(suggested by Roger Pau Monné)
v1:
- fix issues in commit message.
(suggested by Roger Pau Monné)
- remove unnecessary local variable 'val_array'.
(suggested by Roger Pau Monné)
---
xen/arch/x86/psr.c | 63 --
1 file changed, 37 insertions(+), 26 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index daa2aeb..4f47a0b 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -,25 +,47 @@ static unsigned int get_socket_cpu(unsigned int socket)
struct cos_write_info
{
unsigned int cos;
-struct feat_node *feature;
+struct feat_node **features;
const uint32_t *val;
-const struct feat_props *props;
+unsigned int array_len;
+int result;
};
static void do_write_psr_msrs(void *data)
{
-const struct cos_write_info *info = data;
-struct feat_node *feat = info->feature;
-const struct feat_props *props = info->props;
-unsigned int i, cos = info->cos, cos_num = props->cos_num;
+struct cos_write_info *info = data;
+unsigned int i, index = 0, cos = info->cos;
-for ( i = 0; i < cos_num; i++ )
+/*
+ * Iterate all featuers to write different value (not same as MSR) for
+ * each feature.
+ */
+for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
{
-if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
+struct feat_node *feat = info->features[i];
+const struct feat_props *props = feat_props[i];
+unsigned int cos_num, j;
+
+if ( !feat || !props )
+continue;
+
+cos_num = props->cos_num;
+if ( info->array_len < index + cos_num )
+{
+info->result = -ENOSPC;
+return;
+}
+
+for ( j = 0; j < cos_num; j++ )
{
-feat->cos_reg_val[cos * cos_num + i] = info->val[i];
-props->write_msr(cos, info->val[i], props->type[i]);
+if ( feat->cos_reg_val[cos * cos_num + j] != info->val[index + j] )
+{
+feat->cos_reg_val[cos * cos_num + j] = info->val[index + j];
+props->write_msr(cos, info->val[index + j], props->type[j]);
+}
}
+
+index += cos_num;
}
}
@@ -1137,30 +1159,19 @@ static int write_psr_msrs(unsigned int socket, unsigned
int cos,
const uint32_t val[], unsigned int array_len,
enum psr_feat_type feat_type)
{
-int ret;
struct psr_socket_info *info = get_socket_info(socket);
struct cos_write_info data =
{
.cos = cos,
-.feature = info->features[feat_type],
-.props = feat_props[feat_type],
+.features = info->features,
+.val = val,
+.array_len = array_len,
+.result = 0,
};
if ( cos > info->features[feat_type]->cos_max )
return -EINVAL;
-/* Skip to the feature's value head. */
-ret = skip_prior_features(&array_len, feat_type);
-if ( ret < 0 )
-return ret;
-
-val += ret;
-
-if ( array_len < feat_props[feat_type]->cos_num )
-return -ENOSPC;
-
-data.val = val;
-
if ( socket == cpu_to_socket(smp_processor_id()) )
do_write_psr_msrs(&data);
else
@@ -1172,7 +1183,7 @@ static int write_psr_msrs(unsigned int socket, unsigned
int cos,
on_selected_cpus(cpumask_of(cpu), do_write_psr_msrs, &data, 1);
}
-return 0;
+return data.result;
}
int psr_set_val(struct domain *d, unsigned int socket,
--
1.9.1
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v2] x86: psr: support co-exist features' values setting
On 17-10-09 15:03:25, Roger Pau Monn� wrote:
> On Sun, Oct 08, 2017 at 04:22:00AM +0000, Yi Sun wrote:
[...]
> > static void do_write_psr_msrs(void *data)
>
> Should this be "static int do_write_psr_msrs"...
>
This function is a parameter of 'on_selected_cpus()' which requires it to be
'void'.
> > {
> > const struct cos_write_info *info = data;
> > -struct feat_node *feat = info->feature;
> > -const struct feat_props *props = info->props;
> > -unsigned int i, cos = info->cos, cos_num = props->cos_num;
> > +unsigned int i, index = 0, cos = info->cos;
> >
> > -for ( i = 0; i < cos_num; i++ )
> > +/*
> > + * Iterate all featuers to write different value (not same as MSR) for
> > + * each feature.
> > + */
> > +for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
> > {
> > -if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
> > +struct feat_node *feat = info->features[i];
> > +const struct feat_props *props = feat_props[i];
> > +unsigned int cos_num, j;
> > +
> > +if ( !feat || !props )
> > +continue;
> > +
> > +cos_num = props->cos_num;
> > +if ( info->array_len < index + cos_num )
> > +return;
>
> So that you can return -ENOSPC here (inline with what was previously
> done in write_psr_msrs)?
>
> > +
> > +for ( j = 0; j < cos_num; j++ )
> > {
> > -feat->cos_reg_val[cos * cos_num + i] = info->val[i];
> > -props->write_msr(cos, info->val[i], props->type[i]);
> > +if ( feat->cos_reg_val[cos * cos_num + j] != info->val[index +
> > j] )
> > +{
> > +feat->cos_reg_val[cos * cos_num + j] = info->val[index +
> > j];
> > +props->write_msr(cos, info->val[index + j],
> > props->type[j]);
> > +}
> > }
> > +
> > +index += cos_num;
> > }
> > }
> >
> > @@ -1137,30 +1155,18 @@ static int write_psr_msrs(unsigned int socket,
> > unsigned int cos,
> >const uint32_t val[], unsigned int array_len,
> >enum psr_feat_type feat_type)
> > {
> > -int ret;
> > struct psr_socket_info *info = get_socket_info(socket);
> > struct cos_write_info data =
> > {
> > .cos = cos,
> > -.feature = info->features[feat_type],
> > -.props = feat_props[feat_type],
> > +.features = info->features,
> > +.val = val,
> > +.array_len = array_len,
> > };
> >
> > if ( cos > info->features[feat_type]->cos_max )
> > return -EINVAL;
> >
> > -/* Skip to the feature's value head. */
> > -ret = skip_prior_features(&array_len, feat_type);
> > -if ( ret < 0 )
> > -return ret;
> > -
> > -val += ret;
> > -
> > -if ( array_len < feat_props[feat_type]->cos_num )
>
> When moved inside of do_write_psr_msrs this becomes:
>
> info->array_len < index + cos_num
>
> Where cos_num == feat_props[feat_type]->cos_num. Is this correct?
>
> I'm asking because the check used to be array_len < cos_num.
>
The removed old codes here only check one feature cos_num. Because the old
codes can only handle one feature. That is the reason I submit this patch
to support multiple co-exist features' values setting.
> Thanks, Roger.
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v6 15/16] tools: implement new generic set value interface and MBA set value command
This patch implements new generic set value interfaces in libxc and libxl.
These interfaces are suitable for all allocation features. It also adds a
new MBA set value command in xl.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Wei Liu
---
CC: Wei Liu
CC: Ian Jackson
CC: Roger Pau Monné
CC: Chao Peng
v5:
- move xc_type definition and value get out of the loop.
(suggested by Roger Pau Monné)
v4:
- remove 'ALLOC_' from macro name.
(suggested by Roger Pau Monné)
- adjust place of argc check and return EXIT_FAILURE.
(suggested by Roger Pau Monné)
- fix indentation issue.
(suggested by Roger Pau Monné)
- move same type local variables declaration to a single line.
(suggested by Roger Pau Monné)
v3:
- add 'const' for 'opts[]' in 'main_psr_mba_set'.
(suggested by Roger Pau Monné)
- replace 'libxl_psr_cbm_type' to 'libxl_psr_type' for newly defined
interfaces.
(suggested by Roger Pau Monné)
---
tools/libxc/include/xenctrl.h | 6 ++---
tools/libxc/xc_psr.c | 9 ---
tools/libxl/libxl_psr.c | 52
tools/xl/xl.h | 1 +
tools/xl/xl_cmdtable.c| 6 +
tools/xl/xl_psr.c | 55 +++
6 files changed, 96 insertions(+), 33 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index dcea09e..f2463f9 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2512,9 +2512,9 @@ int xc_psr_cmt_get_data(xc_interface *xch, uint32_t rmid,
uint32_t cpu,
uint64_t *tsc);
int xc_psr_cmt_enabled(xc_interface *xch);
-int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_type type, uint32_t target,
- uint64_t data);
+int xc_psr_set_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_type type, uint32_t target,
+ uint64_t data);
int xc_psr_get_domain_data(xc_interface *xch, uint32_t domid,
xc_psr_type type, uint32_t target,
uint64_t *data);
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 191de97..1609185 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -248,9 +248,9 @@ int xc_psr_cmt_enabled(xc_interface *xch)
return 0;
}
-int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_type type, uint32_t target,
- uint64_t data)
+int xc_psr_set_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_type type, uint32_t target,
+ uint64_t data)
{
DECLARE_DOMCTL;
uint32_t cmd;
@@ -269,6 +269,9 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t
domid,
case XC_PSR_CAT_L2_CBM:
cmd = XEN_DOMCTL_PSR_SET_L2_CBM;
break;
+case XC_PSR_MBA_THRTL:
+cmd = XEN_DOMCTL_PSR_SET_MBA_THRTL;
+break;
default:
errno = EINVAL;
return -1;
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index 7c560bc..9ced7d1 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -328,32 +328,7 @@ int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
libxl_psr_cbm_type type, libxl_bitmap *target_map,
uint64_t cbm)
{
-GC_INIT(ctx);
-int rc;
-int socketid, nr_sockets;
-
-rc = libxl__count_physical_sockets(gc, &nr_sockets);
-if (rc) {
-LOGED(ERROR, domid, "failed to get system socket count");
-goto out;
-}
-
-libxl_for_each_set_bit(socketid, *target_map) {
-xc_psr_type xc_type = libxl__psr_type_to_libxc_psr_type(type);
-
-if (socketid >= nr_sockets)
-break;
-
-if (xc_psr_cat_set_domain_data(ctx->xch, domid, xc_type,
- socketid, cbm)) {
-libxl__psr_alloc_log_err_msg(gc, errno, type);
-rc = ERROR_FAIL;
-}
-}
-
-out:
-GC_FREE;
-return rc;
+return libxl_psr_set_val(ctx, domid, type, target_map, cbm);
}
int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
@@ -453,7 +428,30 @@ int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
libxl_psr_type type, libxl_bitmap *target_map,
uint64_t val)
{
-return ERROR_FAIL;
+GC_INIT(ctx);
+int rc, socketid, nr_sockets;
+xc_psr_type xc_type = libxl__psr_type_to_libxc_psr_type(type);
+
+rc = libxl__count_physical_sockets(gc, &nr_sockets);
+if (rc) {
+LOG(ERROR, "failed to get system socket count");
+goto out;
+}
+
+libxl_for_each_set_bit(s
[Xen-devel] [PATCH v6 07/16] x86: implement get value interface for MBA
This patch implements get value domctl interface for MBA.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Jan Beulich
---
CC: Andrew Cooper
CC: Jan Beulich
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v5:
- use newly defined macro to get MBA thrtl.
(suggested by Roger Pau Monné)
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
v3:
- change 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
---
xen/arch/x86/domctl.c | 4
xen/include/public/domctl.h | 1 +
2 files changed, 5 insertions(+)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 0cd18a6..17fd3ad 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1491,6 +1491,10 @@ long arch_do_domctl(
ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L2_CBM, copyback);
break;
+case XEN_DOMCTL_PSR_GET_MBA_THRTL:
+ret = domctl_psr_get_val(d, domctl, PSR_TYPE_MBA_THRTL, copyback);
+break;
+
default:
ret = -EOPNOTSUPP;
break;
diff --git a/xen/include/public/domctl.h b/xen/include/public/domctl.h
index c099334..e8f4c4c 100644
--- a/xen/include/public/domctl.h
+++ b/xen/include/public/domctl.h
@@ -1069,6 +1069,7 @@ struct xen_domctl_psr_alloc {
#define XEN_DOMCTL_PSR_GET_L3_DATA5
#define XEN_DOMCTL_PSR_SET_L2_CBM 6
#define XEN_DOMCTL_PSR_GET_L2_CBM 7
+#define XEN_DOMCTL_PSR_GET_MBA_THRTL 9
uint32_t cmd; /* IN: XEN_DOMCTL_PSR_* */
uint32_t target;/* IN */
uint64_t data; /* IN/OUT */
--
1.9.1
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v6 00/16] Enable Memory Bandwidth Allocation in Xen
Hi, all, We plan to bring a new PSR (Platform Shared Resource) feature called Intel Memory Bandwidth Allocation (MBA) to Xen. Besides the MBA enabling, we change some interfaces to make them more general but not only for CAT. Any comments are welcome! You can find this series at: https://github.com/yisun-git/xen_mba mba_v6 This version base on below pre-fix patch: "x86: psr: support co-exist features' values setting" https://lists.xen.org/archives/html/xen-devel/2017-10/msg00866.html CC: Jan Beulich CC: Andrew Cooper CC: Wei Liu CC: Ian Jackson CC: Daniel De Graaf CC: Roger Pau Monné CC: Konrad Rzeszutek Wilk CC: Chao Peng CC: Julien Grall --- Acked and Reviewed list before V6: a - Acked-by r - Reviewed-by r patch 1 - docs: create Memory Bandwidth Allocation (MBA) feature document r patch 2 - Rename PSR sysctl/domctl interfaces and xsm policy to make them be general ar patch 3 - x86: rename 'cbm_type' to 'psr_type' to make it general r patch 4 - x86: a few optimizations to psr codes r patch 5 - x86: implement data structure and CPU init flow for MBA r patch 6 - x86: implement get hw info flow for MBA ar patch 7 - x86: implement get value interface for MBA ar patch 9 - tools: create general interfaces to support psr allocation features ar patch 10 - tools: implement the new libxc get hw info interface ar patch 11 - tools: implement the new libxl get hw info interface ar patch 12 - tools: implement the new xl get hw info interface ar patch 13 - tools: rename 'xc_psr_cat_type' to 'xc_psr_type' ar patch 14 - tools: implement new generic get value interface and MBA get value command ar patch 15 - tools: implement new generic set value interface and MBA set value command ar patch 16 - docs: add MBA description in docs --- V6 change list: Patch 1: - fix some words. (suggested by Roger Pau Monné) Patch 2: - move macro definition into the function and undefine it after use. (suggested by Roger Pau Monné) - do not bump sysctl version because it has been bumped for 4.10. (suggested by Roger Pau Monné) Patch 4: - restore 'write_msr()' type to 'void'. (suggested by Jan Beulich and Roger Pau Monné) - change 'ebx' in 'psr_cpu_init' to 'feat_mask'. (suggested by Jan Beulich and Roger Pau Monné) Patch 5: - restore type of 'mba_write_msr' to 'void'. Patch 8: - split co-exist features' values setting flow to a new patch. (suggested by Jan Beulich) - restore codes related to 'mba_check_thrtl' and 'check_value'. (suggested by Jan Beulich) Patch 10: - remove unnecessary spaces in brackets. (suggested by Wei Liu) - use assert to check input lvl. (suggested by Roger Pau Monné) Patch 14: - fix one coding style issue. (suggested by Roger Pau Monné) Patch 15: - move xc_type definition and value get out of the loop. (suggested by Roger Pau Monné) Yi Sun (16): docs: create Memory Bandwidth Allocation (MBA) feature document Rename PSR sysctl/domctl interfaces and xsm policy to make them be general x86: rename 'cbm_type' to 'psr_type' to make it general x86: a few optimizations to psr codes x86: implement data structure and CPU init flow for MBA x86: implement get hw info flow for MBA x86: implement get value interface for MBA x86: implement set value flow for MBA tools: create general interfaces to support psr allocation features tools: implement the new libxc get hw info interface tools: implement the new libxl get hw info interface tools: implement the new xl get hw info interface tools: rename 'xc_psr_cat_type' to 'xc_psr_type' tools: implement new generic get value interface and MBA get value command tools: implement new generic set value interface and MBA set value command docs: add MBA description in docs docs/features/intel_psr_mba.pandoc | 297 ++ docs/man/xl.pod.1.in| 33 docs/misc/xl-psr.markdown | 62 tools/flask/policy/modules/dom0.te | 4 +- tools/libxc/include/xenctrl.h | 44 +++-- tools/libxc/xc_psr.c| 109 - tools/libxl/libxl.h | 37 + tools/libxl/libxl_psr.c | 223 -- tools/libxl/libxl_types.idl | 22 +++ tools/xl/xl.h | 2 + tools/xl/xl_cmdtable.c | 12 ++ tools/xl/xl_psr.c | 279 +--- xen/arch/x86/domctl.c | 87 +- xen/arch/x86/psr.c | 310 +++- xen/arch/x86/sysctl.c | 53 -- xen/include/asm-x86/msr-index.h | 1 + xen/include/asm-x86/psr.h |
[Xen-devel] [PATCH v6 13/16] tools: rename 'xc_psr_cat_type' to 'xc_psr_type'
This patch renames 'xc_psr_cat_type' to 'xc_psr_type' so that
the structure name is common for all allocation features.
Signed-off-by: Yi Sun
Acked-by: Wei Liu
Reviewed-by: Chao Peng
Reviewed-by: Roger Pau Monné
---
CC: Ian Jackson
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v5:
- remove a duplicated ';'.
(suggested by Roger Pau Monné)
v4:
- move assignment of xc_type to its declaration place.
(suggested by Roger Pau Monné)
v3:
- change 'xc_psr_val_type' to 'xc_psr_type'.
(suggested by Roger Pau Monné)
---
tools/libxc/include/xenctrl.h | 8
tools/libxc/xc_psr.c | 4 ++--
tools/libxl/libxl_psr.c | 11 +--
3 files changed, 11 insertions(+), 12 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 2d977c8..2736bc5 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2466,13 +2466,13 @@ enum xc_psr_cmt_type {
};
typedef enum xc_psr_cmt_type xc_psr_cmt_type;
-enum xc_psr_cat_type {
+enum xc_psr_type {
XC_PSR_CAT_L3_CBM = 1,
XC_PSR_CAT_L3_CBM_CODE = 2,
XC_PSR_CAT_L3_CBM_DATA = 3,
XC_PSR_CAT_L2_CBM = 4,
};
-typedef enum xc_psr_cat_type xc_psr_cat_type;
+typedef enum xc_psr_type xc_psr_type;
enum xc_psr_feat_type {
XC_PSR_CAT_L3,
@@ -2512,10 +2512,10 @@ int xc_psr_cmt_get_data(xc_interface *xch, uint32_t
rmid, uint32_t cpu,
int xc_psr_cmt_enabled(xc_interface *xch);
int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_type type, uint32_t target,
uint64_t data);
int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_type type, uint32_t target,
uint64_t *data);
int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
xc_psr_feat_type type, xc_psr_hw_info *hw_info);
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 2c605a7..01f4ba7 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -249,7 +249,7 @@ int xc_psr_cmt_enabled(xc_interface *xch)
return 0;
}
int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_type type, uint32_t target,
uint64_t data)
{
DECLARE_DOMCTL;
@@ -284,7 +284,7 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t
domid,
}
int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_cat_type type, uint32_t target,
+ xc_psr_type type, uint32_t target,
uint64_t *data)
{
int rc;
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index b053abd..c54cb6f 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -303,11 +303,11 @@ out:
return rc;
}
-static inline xc_psr_cat_type libxl__psr_cbm_type_to_libxc_psr_cat_type(
+static inline xc_psr_type libxl__psr_cbm_type_to_libxc_psr_type(
libxl_psr_cbm_type type)
{
-BUILD_BUG_ON(sizeof(libxl_psr_cbm_type) != sizeof(xc_psr_cat_type));
-return (xc_psr_cat_type)type;
+BUILD_BUG_ON(sizeof(libxl_psr_cbm_type) != sizeof(xc_psr_type));
+return (xc_psr_type)type;
}
int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
@@ -325,12 +325,11 @@ int libxl_psr_cat_set_cbm(libxl_ctx *ctx, uint32_t domid,
}
libxl_for_each_set_bit(socketid, *target_map) {
-xc_psr_cat_type xc_type;
+xc_psr_type xc_type = libxl__psr_cbm_type_to_libxc_psr_type(type);
if (socketid >= nr_sockets)
break;
-xc_type = libxl__psr_cbm_type_to_libxc_psr_cat_type(type);
if (xc_psr_cat_set_domain_data(ctx->xch, domid, xc_type,
socketid, cbm)) {
libxl__psr_cat_log_err_msg(gc, errno);
@@ -349,7 +348,7 @@ int libxl_psr_cat_get_cbm(libxl_ctx *ctx, uint32_t domid,
{
GC_INIT(ctx);
int rc = 0;
-xc_psr_cat_type xc_type = libxl__psr_cbm_type_to_libxc_psr_cat_type(type);
+xc_psr_type xc_type = libxl__psr_cbm_type_to_libxc_psr_type(type);
if (xc_psr_cat_get_domain_data(ctx->xch, domid, xc_type,
target, cbm_r)) {
--
1.9.1
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v6 06/16] x86: implement get hw info flow for MBA
This patch implements get HW info flow for MBA including its callback
function and sysctl interface.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v5:
- use ASSERT in 'mba_get_feat_info'.
(suggested by Roger Pau Monné)
- correct initialization format of 'data[PSR_INFO_ARRAY_SIZE]'.
(suggested by Roger Pau Monné and Jan Beulich)
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- initialize 'data[PSR_INFO_ARRAY_SIZE]' to 0 to prevent to leak stack data.
(suggested by Roger Pau Monné)
v3:
- replace 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
v2:
- use 'XEN_SYSCTL_PSR_MBA_LINEAR' to set MBA feature HW info.
(suggested by Chao Peng)
v1:
- sort 'PSR_INFO_IDX_' macros as feature.
(suggested by Chao Peng)
- rename 'PSR_INFO_IDX_MBA_LINEAR' to 'PSR_INFO_IDX_MBA_FLAG'.
- rename 'linear' in 'struct mba_info' to 'flags' for future extension.
(suggested by Chao Peng)
---
xen/arch/x86/psr.c | 14 +-
xen/arch/x86/sysctl.c | 21 -
xen/include/asm-x86/psr.h | 2 ++
xen/include/public/sysctl.h | 8
4 files changed, 43 insertions(+), 2 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 157e11f..03f24c0 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -263,6 +263,10 @@ static enum psr_feat_type psr_type_to_feat_type(enum
psr_type type)
feat_type = FEAT_TYPE_L2_CAT;
break;
+case PSR_TYPE_MBA_THRTL:
+feat_type = FEAT_TYPE_MBA;
+break;
+
default:
ASSERT_UNREACHABLE();
}
@@ -483,7 +487,15 @@ static const struct feat_props l2_cat_props = {
static bool mba_get_feat_info(const struct feat_node *feat,
uint32_t data[], unsigned int array_len)
{
-return false;
+ASSERT(array_len == PSR_INFO_ARRAY_SIZE);
+
+data[PSR_INFO_IDX_COS_MAX] = feat->cos_max;
+data[PSR_INFO_IDX_MBA_THRTL_MAX] = feat->mba.thrtl_max;
+
+if ( feat->mba.linear )
+data[PSR_INFO_IDX_MBA_FLAG] |= XEN_SYSCTL_PSR_MBA_LINEAR;
+
+return true;
}
static void mba_write_msr(unsigned int cos, uint32_t val,
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index 6867ee1..f48d6fd 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -174,7 +174,7 @@ long arch_do_sysctl(
case XEN_SYSCTL_psr_alloc:
switch ( sysctl->u.psr_alloc.cmd )
{
-uint32_t data[PSR_INFO_ARRAY_SIZE];
+uint32_t data[PSR_INFO_ARRAY_SIZE] = { };
case XEN_SYSCTL_PSR_get_l3_info:
{
@@ -214,6 +214,25 @@ long arch_do_sysctl(
break;
}
+case XEN_SYSCTL_PSR_get_mba_info:
+{
+ret = psr_get_info(sysctl->u.psr_alloc.target,
+ PSR_TYPE_MBA_THRTL, data, ARRAY_SIZE(data));
+if ( ret )
+break;
+
+sysctl->u.psr_alloc.u.mba_info.cos_max =
+ data[PSR_INFO_IDX_COS_MAX];
+sysctl->u.psr_alloc.u.mba_info.thrtl_max =
+ data[PSR_INFO_IDX_MBA_THRTL_MAX];
+sysctl->u.psr_alloc.u.mba_info.flags =
+ data[PSR_INFO_IDX_MBA_FLAG];
+
+if ( __copy_field_to_guest(u_sysctl, sysctl, u.psr_alloc) )
+ret = -EFAULT;
+break;
+}
+
default:
ret = -EOPNOTSUPP;
break;
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index 9d14264..084ae97 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -39,6 +39,8 @@
#define PSR_INFO_IDX_COS_MAX0
#define PSR_INFO_IDX_CAT_CBM_LEN1
#define PSR_INFO_IDX_CAT_FLAG 2
+#define PSR_INFO_IDX_MBA_THRTL_MAX 1
+#define PSR_INFO_IDX_MBA_FLAG 2
#define PSR_INFO_ARRAY_SIZE 3
struct psr_cmt_l3 {
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index a50e345..f7f26c3 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -698,6 +698,7 @@ struct xen_sysctl_pcitopoinfo {
#define XEN_SYSCTL_PSR_get_l3_info 0
#define XEN_SYSCTL_PSR_get_l2_info 1
+#define XEN_SYSCTL_PSR_get_mba_info 2
struct xen_sysctl_psr_alloc {
uint32_t cmd; /* IN: XEN_SYSCTL_PSR_* */
uint32_t target;/* IN */
@@ -708,6 +709,13 @@ struct xen_sysctl_psr_alloc {
#define XEN_SYSCTL_PSR_CAT_L3_CDP (1u << 0)
uint32_t flags; /* OUT: CAT flags */
} cat_info;
+
+struct {
+uint32_t thrtl_
[Xen-devel] [PATCH v6 11/16] tools: implement the new libxl get hw info interface
This patch implements the new libxl get hw info interface,
'libxl_psr_get_hw_info', which is suitable to all psr allocation
features. It also implements corresponding list free function,
'libxl_psr_hw_info_list_free' and makes 'libxl_psr_cat_get_info' call
'libxl_psr_get_hw_info' to avoid redundant code in libxl_psr.c.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Wei Liu
---
CC: Wei Liu
CC: Ian Jackson
CC: Roger Pau Monné
CC: Chao Peng
v5:
- change 'if (rc < 0)' to 'if (rc)'.
(suggested by Roger Pau Monné)
v4:
- remove 'xc_' from struct name.
(suggested by Roger Pau Monné)
- fix words in commit message.
(suggested by Roger Pau Monné)
- change type of 'libxl__hw_info_to_libxl_cat_info' to void and use
assert to check invalid type. Then, remove check for
'libxl__hw_info_to_libxl_cat_info'.
(suggested by Roger Pau Monné)
- change type of 'libxl__xc_hw_info_to_libxl_hw_info' to void and use
assert to check invalid type. Then, remove check for
'libxl__xc_hw_info_to_libxl_hw_info'.
(suggested by Roger Pau Monné)
v3:
- remove casting.
(suggested by Roger Pau Monné)
- remove inline.
(suggested by Roger Pau Monné)
- change 'libxc__psr_hw_info_to_libxl_psr_hw_info' to
'libxl__xc_hw_info_to_libxl_hw_info'.
(suggested by Roger Pau Monné)
- remove '_hw' from parameter names.
(suggested by Roger Pau Monné)
- change some 'LOGE' to 'LOG'.
(suggested by Roger Pau Monné)
- check returned 'xc_type' and remove redundant 'lvl' check.
(suggested by Roger Pau Monné)
v2:
- split this patch out from a big patch in v1.
(suggested by Wei Liu)
- change 'CAT_INFO'/'MBA_INFO' to 'CAT' and 'MBA. Also the libxl structure
name 'cat_info'/'mba_info' is changed to 'cat'/'mba'.
(suggested by Chao Peng)
- call 'libxl_psr_hw_info_list_free' in 'libxl_psr_cat_get_info' to free
allocated resources.
(suggested by Chao Peng)
---
tools/libxl/libxl_psr.c | 131 ++--
1 file changed, 93 insertions(+), 38 deletions(-)
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index e1cc250..b053abd 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -386,56 +386,41 @@ static xc_psr_feat_type
libxl__feat_type_to_libxc_feat_type(
return xc_type;
}
+static void libxl__hw_info_to_libxl_cat_info(
+libxl_psr_feat_type type, libxl_psr_hw_info *hw_info,
+libxl_psr_cat_info *cat_info)
+{
+assert(type == LIBXL_PSR_FEAT_TYPE_CAT);
+
+cat_info->id = hw_info->id;
+cat_info->cos_max = hw_info->u.cat.cos_max;
+cat_info->cbm_len = hw_info->u.cat.cbm_len;
+cat_info->cdp_enabled = hw_info->u.cat.cdp_enabled;
+}
+
int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
unsigned int *nr, unsigned int lvl)
{
GC_INIT(ctx);
int rc;
-int i = 0, socketid, nr_sockets;
-libxl_bitmap socketmap;
+unsigned int i;
+libxl_psr_hw_info *hw_info;
libxl_psr_cat_info *ptr;
-xc_psr_hw_info hw_info;
-xc_psr_feat_type xc_type;
-
-libxl_bitmap_init(&socketmap);
-
-rc = libxl__count_physical_sockets(gc, &nr_sockets);
-if (rc) {
-LOGE(ERROR, "failed to get system socket count");
-goto out;
-}
-libxl_socket_bitmap_alloc(ctx, &socketmap, nr_sockets);
-rc = libxl_get_online_socketmap(ctx, &socketmap);
-if (rc < 0) {
-LOGE(ERROR, "failed to get available sockets");
+rc = libxl_psr_get_hw_info(ctx, LIBXL_PSR_FEAT_TYPE_CAT, lvl, nr,
&hw_info);
+if (rc)
goto out;
-}
-
-xc_type = libxl__feat_type_to_libxc_feat_type(LIBXL_PSR_FEAT_TYPE_CAT,
lvl);
-
-ptr = libxl__malloc(NOGC, nr_sockets * sizeof(libxl_psr_cat_info));
-
-libxl_for_each_set_bit(socketid, socketmap) {
-ptr[i].id = socketid;
-if (xc_psr_get_hw_info(ctx->xch, socketid, xc_type, &hw_info)) {
-LOGE(ERROR, "failed to get hw info");
-rc = ERROR_FAIL;
-free(ptr);
-goto out;
-}
-ptr[i].cos_max = hw_info.cat.cos_max;
-ptr[i].cbm_len = hw_info.cat.cbm_len;
-ptr[i].cdp_enabled = hw_info.cat.cdp_enabled;
+ptr = libxl__malloc(NOGC, *nr * sizeof(libxl_psr_cat_info));
-i++;
-}
+for (i = 0; i < *nr; i++)
+libxl__hw_info_to_libxl_cat_info(LIBXL_PSR_FEAT_TYPE_CAT,
+ &hw_info[i],
+ &ptr[i])
[Xen-devel] [PATCH v6 09/16] tools: create general interfaces to support psr allocation features
This patch creates general interfaces in libxl to support all psr
allocation features.
Add 'LIBXL_HAVE_PSR_GENERIC' to indicate interface change.
Please note, the functionality cannot work until later patches
are applied.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Wei Liu
---
CC: Wei Liu
CC: Ian Jackson
CC: Roger Pau Monné
CC: Chao Peng
v5:
- adjust parameters position in 'libxl_psr_get_hw_info'.
(suggested by Roger Pau Monné)
v4:
- add description for LIBXL_HAVE_PSR_GENERIC to mention newly added
public functions.
(suggested by Roger Pau Monné)
v3:
- change 'LIBXL_HAVE_PSR_MBA' to 'LIBXL_HAVE_PSR_GENERIC'.
(suggested by Roger Pau Monné)
- 'typedef enum libxl_psr_cbm_type libxl_psr_type;' in libxl.h.
(suggested by Roger Pau Monné and Wei Liu)
- change 'libxl_psr_cbm_type' to 'libxl_psr_type' in newly defined
interfaces.
(suggested by Roger Pau Monné)
v2:
- remove '_INFO' in 'libxl_psr_feat_type' and make corresponding
changes in 'libxl_psr_hw_info'.
(suggested by Chao Peng)
---
tools/libxl/libxl.h | 37 +
tools/libxl/libxl_psr.c | 25 +
tools/libxl/libxl_types.idl | 22 ++
3 files changed, 84 insertions(+)
diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h
index 827272e..0d2dee8 100644
--- a/tools/libxl/libxl.h
+++ b/tools/libxl/libxl.h
@@ -967,6 +967,17 @@ void libxl_mac_copy(libxl_ctx *ctx, libxl_mac *dst, const
libxl_mac *src);
#define LIBXL_HAVE_PSR_L2_CAT 1
/*
+ * LIBXL_HAVE_PSR_GENERIC
+ *
+ * If this is defined, the Memory Bandwidth Allocation feature is supported.
+ * The following public functions are available:
+ * libxl_psr_{set/get}_val
+ * libxl_psr_get_hw_info
+ * libxl_psr_hw_info_list_free
+ */
+#define LIBXL_HAVE_PSR_GENERIC 1
+
+/*
* LIBXL_HAVE_MCA_CAPS
*
* If this is defined, setting MCA capabilities for HVM domain is supported.
@@ -2287,6 +2298,32 @@ int libxl_psr_cat_get_info(libxl_ctx *ctx,
libxl_psr_cat_info **info,
int libxl_psr_cat_get_l3_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
int *nr);
void libxl_psr_cat_info_list_free(libxl_psr_cat_info *list, int nr);
+
+typedef enum libxl_psr_cbm_type libxl_psr_type;
+
+/*
+ * Function to set a domain's value. It operates on a single or multiple
+ * target(s) defined in 'target_map'. 'target_map' specifies all the sockets
+ * to be operated on.
+ */
+int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_type type, libxl_bitmap *target_map,
+ uint64_t val);
+/*
+ * Function to get a domain's cbm. It operates on a single 'target'.
+ * 'target' specifies which socket to be operated on.
+ */
+int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_type type, unsigned int target,
+ uint64_t *val);
+/*
+ * On success, the function returns an array of elements in 'info',
+ * and the length in 'nr'.
+ */
+int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_feat_type type,
+ unsigned int lvl, unsigned int *nr,
+ libxl_psr_hw_info **info);
+void libxl_psr_hw_info_list_free(libxl_psr_hw_info *list, unsigned int nr);
#endif
/* misc */
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index 197505a..d4f5f67 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -428,6 +428,31 @@ void libxl_psr_cat_info_list_free(libxl_psr_cat_info
*list, int nr)
free(list);
}
+int libxl_psr_set_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_type type, libxl_bitmap *target_map,
+ uint64_t val)
+{
+return ERROR_FAIL;
+}
+
+int libxl_psr_get_val(libxl_ctx *ctx, uint32_t domid,
+ libxl_psr_type type, unsigned int target,
+ uint64_t *val)
+{
+return ERROR_FAIL;
+}
+
+int libxl_psr_get_hw_info(libxl_ctx *ctx, libxl_psr_feat_type type,
+ unsigned int lvl, unsigned int *nr,
+ libxl_psr_hw_info **info)
+{
+return ERROR_FAIL;
+}
+
+void libxl_psr_hw_info_list_free(libxl_psr_hw_info *list, unsigned int nr)
+{
+}
+
/*
* Local variables:
* mode: C
diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl
index c2a1141..6f53b2d 100644
--- a/tools/libxl/libxl_types.idl
+++ b/tools/libxl/libxl_types.idl
@@ -1025,6 +1025,7 @@ libxl_psr_cbm_type = Enumeration("psr_cbm_type", [
(2, "L3_CBM_CODE"),
(3, "L3_CBM_DATA"),
(4, "L2_CBM"),
+(5, "MBA_THRTL"),
])
libxl_psr_cat_info = Struct("psr_cat_info", [
@@ -1033,3 +1034,24
[Xen-devel] [PATCH v6 16/16] docs: add MBA description in docs
This patch adds MBA description in related documents. Signed-off-by: Yi Sun Acked-by: Wei Liu Reviewed-by: Roger Pau Monné --- CC: Ian Jackson CC: Wei Liu CC: Roger Pau Monné CC: Chao Peng v5: - remove 'closed-loop' in 'xl-psr.markdown' (suggested by Roger Pau Monné) v4: - modify description of MBA in 'xl.pod.1.in' to be same as feature doc. (suggested by Roger Pau Monné) - fix words issue. (suggested by Roger Pau Monné) v2: - state the value type shown by 'psr-mba-show'. For linear mode, it shows decimal value. For non-linear mode, it shows hexadecimal value. (suggested by Chao Peng) --- docs/man/xl.pod.1.in | 33 + docs/misc/xl-psr.markdown | 62 +++ 2 files changed, 95 insertions(+) diff --git a/docs/man/xl.pod.1.in b/docs/man/xl.pod.1.in index cd8bb1c..324ef9e 100644 --- a/docs/man/xl.pod.1.in +++ b/docs/man/xl.pod.1.in @@ -1845,6 +1845,39 @@ processed. =back +=head2 Memory Bandwidth Allocation + +Intel Skylake and later server platforms offer capabilities to configure and +make use of the Memory Bandwidth Allocation (MBA) mechanisms, which provides +OS/VMMs the ability to slow misbehaving apps/VMs by using a credit-based +throttling mechanism. In the Xen implementation, MBA is used to control memory +bandwidth on VM basis. To enforce bandwidth on a specific domain, just set +throttling value (THRTL) for the domain. + +=over 4 + +=item B [I] I I + +Set throttling value (THRTL) for a domain. For how to specify I +please refer to L<http://xenbits.xen.org/docs/unstable/misc/xl-psr.html>. + +B + +=over 4 + +=item B<-s SOCKET>, B<--socket=SOCKET> + +Specify the socket to process, otherwise all sockets are processed. + +=back + +=item B [I] + +Show MBA settings for a certain domain or all domains. For linear mode, it +shows the decimal value. For non-linear mode, it shows hexadecimal value. + +=back + =head1 IGNORED FOR COMPATIBILITY WITH XM xl is mostly command-line compatible with the old xm utility used with diff --git a/docs/misc/xl-psr.markdown b/docs/misc/xl-psr.markdown index 04dd957..3d196ed 100644 --- a/docs/misc/xl-psr.markdown +++ b/docs/misc/xl-psr.markdown @@ -186,6 +186,68 @@ Setting data CBM for a domain: Setting the same code and data CBM for a domain: `xl psr-cat-set ` +## Memory Bandwidth Allocation (MBA) + +Memory Bandwidth Allocation (MBA) is a new feature available on Intel +Skylake and later server platforms that allows an OS or Hypervisor/VMM to +slow misbehaving apps/VMs by using a credit-based throttling mechanism. To +enforce bandwidth on a specific domain, just set throttling value (THRTL) +into Class of Service (COS). MBA provides two THRTL mode. One is linear mode +and the other is non-linear mode. + +In the linear mode the input precision is defined as 100-(THRTL_MAX). Values +not an even multiple of the precision (e.g., 12%) will be rounded down (e.g., +to 10% delay by the hardware). + +If linear values are not supported then input delay values are powers-of-two +from zero to the THRTL_MAX value from CPUID. In this case any values not a power +of two will be rounded down the next nearest power of two. + +For example, assuming a system with 2 domains: + + * A THRTL of 0x0 for every domain means each domain can access the whole cache + without any delay. This is the default. + + * Linear mode: Giving one domain a THRTL of 0xC and the other domain's 0 means + that the first domain gets 10% delay to access the cache and the other one + without any delay. + + * Non-linear mode: Giving one domain a THRTL of 0xC and the other domain's 0 + means that the first domain gets 8% delay to access the cache and the other + one without any delay. + +For more detailed information please refer to Intel SDM chapter +"Introduction to Memory Bandwidth Allocation". + +In Xen's implementation, THRTL can be configured with libxl/xl interfaces but +COS is maintained in hypervisor only. The cache partition granularity is per +domain, each domain has COS=0 assigned by default, the corresponding THRTL is +0, which means all the cache resource can be accessed without delay. + +### xl interfaces + +System MBA information such as maximum COS and maximum THRTL can be obtained by: + +`xl psr-hwinfo --mba` + +The simplest way to change a domain's THRTL from its default is running: + +`xl psr-mba-set [OPTIONS] ` + +In a multi-socket system, the same thrtl will be set on each socket by default. +Per socket thrtl can be specified with the `--socket SOCKET` option. + +Setting the THRTL may not be successful if insufficient COS is available. In +such case unused COS(es) may be freed by setting THRTL of all related domains to +its default value(0). + +Per domain THRTL settings can be shown by: + +`xl psr-mba-show [OPTIONS] ` + +For linear mode, it shows the decimal v
[Xen-devel] [PATCH v6 02/16] Rename PSR sysctl/domctl interfaces and xsm policy to make them be general
This patch renames PSR sysctl/domctl interfaces and related xsm policy to
make them be general for all resource allocation features but not only
for CAT. Then, we can resuse the interfaces for all allocation features.
Basically, it changes 'psr_cat_op' to 'psr_alloc', and remove 'CAT_' from some
macros. E.g.:
1. psr_cat_op -> psr_alloc
2. XEN_DOMCTL_psr_cat_op -> XEN_DOMCTL_psr_alloc
3. XEN_SYSCTL_psr_cat_op -> XEN_SYSCTL_psr_alloc
4. XEN_DOMCTL_PSR_CAT_SET_L3_CBM -> XEN_DOMCTL_PSR_SET_L3_CBM
5. XEN_SYSCTL_PSR_CAT_get_l3_info -> XEN_SYSCTL_PSR_get_l3_info
Signed-off-by: Yi Sun
Reviewed-by: Wei Liu
Reviewed-by: Roger Pau Monné
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Ian Jackson
CC: Daniel De Graaf
CC: Roger Pau Monné
CC: Chao Peng
v6:
- move macro definition into the function and undefine it after use.
(suggested by Roger Pau Monné)
- do not bump sysctl version because it has been bumped for 4.10.
(suggested by Roger Pau Monné)
v5:
- remove domctl version number upgrade.
(suggested by Jan Beulich)
- restore 'XEN_SYSCTL_PSR_CAT_L3_CDP'.
(suggested by Jan Beulich)
- define a local macro to complete psr get value flow.
(suggested by Roger Pau Monné)
- remove 'Reviewed-by' and 'Acked-by'.
(suggested by Wei Liu)
v4:
- remove 'ALLOC_' from names.
(suggested by Roger Pau Monné)
- fix comments.
(suggested by Roger Pau Monné)
v3:
- remove 'op/OP' from names and modify some names from 'PSR_CAT' to
'PSR_ALLOC'.
(suggested by Roger Pau Monné)
v1:
- add description about what to be changed in commit message.
(suggested by Wei Liu)
- bump sysctl/domctl version numbers.
(suggested by Wei Liu)
---
tools/flask/policy/modules/dom0.te | 4 +--
tools/libxc/xc_psr.c| 50 +-
xen/arch/x86/domctl.c | 71 ++---
xen/arch/x86/sysctl.c | 28 +++
xen/include/public/domctl.h | 24 ++---
xen/include/public/sysctl.h | 12 +++
xen/xsm/flask/hooks.c | 8 ++---
xen/xsm/flask/policy/access_vectors | 8 ++---
8 files changed, 102 insertions(+), 103 deletions(-)
diff --git a/tools/flask/policy/modules/dom0.te
b/tools/flask/policy/modules/dom0.te
index 1643b40..07de3d5 100644
--- a/tools/flask/policy/modules/dom0.te
+++ b/tools/flask/policy/modules/dom0.te
@@ -14,7 +14,7 @@ allow dom0_t xen_t:xen {
tmem_control getscheduler setscheduler
};
allow dom0_t xen_t:xen2 {
- resource_op psr_cmt_op psr_cat_op pmu_ctrl get_symbol
+ resource_op psr_cmt_op psr_alloc pmu_ctrl get_symbol
get_cpu_levelling_caps get_cpu_featureset livepatch_op
gcov_op set_parameter
};
@@ -39,7 +39,7 @@ allow dom0_t dom0_t:domain {
};
allow dom0_t dom0_t:domain2 {
set_cpuid gettsc settsc setscheduler set_max_evtchn set_vnumainfo
- get_vnumainfo psr_cmt_op psr_cat_op set_gnttab_limits
+ get_vnumainfo psr_cmt_op psr_alloc set_gnttab_limits
};
allow dom0_t dom0_t:resource { add remove };
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 039b920..5c54a35 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -258,27 +258,27 @@ int xc_psr_cat_set_domain_data(xc_interface *xch,
uint32_t domid,
switch ( type )
{
case XC_PSR_CAT_L3_CBM:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_CBM;
+cmd = XEN_DOMCTL_PSR_SET_L3_CBM;
break;
case XC_PSR_CAT_L3_CBM_CODE:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_CODE;
+cmd = XEN_DOMCTL_PSR_SET_L3_CODE;
break;
case XC_PSR_CAT_L3_CBM_DATA:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L3_DATA;
+cmd = XEN_DOMCTL_PSR_SET_L3_DATA;
break;
case XC_PSR_CAT_L2_CBM:
-cmd = XEN_DOMCTL_PSR_CAT_OP_SET_L2_CBM;
+cmd = XEN_DOMCTL_PSR_SET_L2_CBM;
break;
default:
errno = EINVAL;
return -1;
}
-domctl.cmd = XEN_DOMCTL_psr_cat_op;
+domctl.cmd = XEN_DOMCTL_psr_alloc;
domctl.domain = (domid_t)domid;
-domctl.u.psr_cat_op.cmd = cmd;
-domctl.u.psr_cat_op.target = target;
-domctl.u.psr_cat_op.data = data;
+domctl.u.psr_alloc.cmd = cmd;
+domctl.u.psr_alloc.target = target;
+domctl.u.psr_alloc.data = data;
return do_domctl(xch, &domctl);
}
@@ -294,31 +294,31 @@ int xc_psr_cat_get_domain_data(xc_interface *xch,
uint32_t domid,
switch ( type )
{
case XC_PSR_CAT_L3_CBM:
-cmd = XEN_DOMCTL_PSR_CAT_OP_GET_L3_CBM;
+cmd = XEN_DOMCTL_PSR_GET_L3_CBM;
break;
case XC_PSR_CAT_L3_CBM_CODE:
-cmd = XEN_DOMCTL_PSR_CAT_OP_GET_L3_CODE;
+cmd = XEN_DOMCTL_PSR_GET_L3_CODE;
break;
case XC_PSR_CAT_L3_CBM_DATA:
-cmd = XEN_DOMCTL_PSR_C
[Xen-devel] [PATCH v6 04/16] x86: a few optimizations to psr codes
This patch refines psr codes:
1. Change type of 'cat_init_feature' to 'bool' to remove the pointless
returning of error code.
2. Move printk in 'cat_init_feature' to reduce a return path.
3. Define a local variable 'feat_mask' in 'psr_cpu_init' to reduce calling of
'cpuid_count_leaf()'.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v6:
- restore 'write_msr()' type to 'void'.
(suggested by Jan Beulich and Roger Pau Monné)
- change 'ebx' in 'psr_cpu_init' to 'feat_mask'.
(suggested by Jan Beulich and Roger Pau Monné)
v5:
- create this patch to make codes clearer.
(suggested by Jan Beulich and Roger Pau Monné)
---
xen/arch/x86/psr.c | 35 +--
1 file changed, 17 insertions(+), 18 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 263ddbe..5945efa 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -273,10 +273,10 @@ static bool psr_check_cbm(unsigned int cbm_len, unsigned
long cbm)
}
/* CAT common functions implementation. */
-static int cat_init_feature(const struct cpuid_leaf *regs,
-struct feat_node *feat,
-struct psr_socket_info *info,
-enum psr_feat_type type)
+static bool cat_init_feature(const struct cpuid_leaf *regs,
+ struct feat_node *feat,
+ struct psr_socket_info *info,
+ enum psr_feat_type type)
{
const char *const cat_feat_name[FEAT_TYPE_NUM] = {
[FEAT_TYPE_L3_CAT] = "L3 CAT",
@@ -286,7 +286,7 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
/* No valid value so do not enable feature. */
if ( !regs->a || !regs->d )
-return -ENOENT;
+return false;
feat->cbm_len = (regs->a & CAT_CBM_LEN_MASK) + 1;
feat->cos_max = min(opt_cos_max, regs->d & CAT_COS_MAX_MASK);
@@ -296,7 +296,7 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
case FEAT_TYPE_L3_CAT:
case FEAT_TYPE_L2_CAT:
if ( feat->cos_max < 1 )
-return -ENOENT;
+return false;
/* We reserve cos=0 as default cbm (all bits within cbm_len are 1). */
feat->cos_reg_val[0] = cat_default_val(feat->cbm_len);
@@ -313,7 +313,7 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
uint64_t val;
if ( feat->cos_max < 3 )
-return -ENOENT;
+return false;
/* Cut half of cos_max when CDP is enabled. */
feat->cos_max = (feat->cos_max - 1) >> 1;
@@ -332,20 +332,18 @@ static int cat_init_feature(const struct cpuid_leaf *regs,
}
default:
-return -ENOENT;
+return false;
}
/* Add this feature into array. */
info->features[type] = feat;
-if ( !opt_cpu_info )
-return 0;
-
-printk(XENLOG_INFO "%s: enabled on socket %u, cos_max:%u, cbm_len:%u\n",
- cat_feat_name[type], cpu_to_socket(smp_processor_id()),
- feat->cos_max, feat->cbm_len);
+if ( opt_cpu_info )
+printk(XENLOG_INFO "%s: enabled on socket %u, cos_max:%u,
cbm_len:%u\n",
+ cat_feat_name[type], cpu_to_socket(smp_processor_id()),
+ feat->cos_max, feat->cbm_len);
-return 0;
+return true;
}
static bool cat_get_feat_info(const struct feat_node *feat,
@@ -1416,6 +1414,7 @@ static void psr_cpu_init(void)
unsigned int socket, cpu = smp_processor_id();
struct feat_node *feat;
struct cpuid_leaf regs;
+uint32_t feat_mask;
if ( !psr_alloc_feat_enabled() || !boot_cpu_has(X86_FEATURE_PQE) )
goto assoc_init;
@@ -1434,7 +1433,8 @@ static void psr_cpu_init(void)
spin_lock_init(&info->ref_lock);
cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 0, ®s);
-if ( regs.b & PSR_RESOURCE_TYPE_L3 )
+feat_mask = regs.b;
+if ( feat_mask & PSR_RESOURCE_TYPE_L3 )
{
cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 1, ®s);
@@ -1455,8 +1455,7 @@ static void psr_cpu_init(void)
}
}
-cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 0, ®s);
-if ( regs.b & PSR_RESOURCE_TYPE_L2 )
+if ( feat_mask & PSR_RESOURCE_TYPE_L2 )
{
cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 2, ®s);
--
1.9.1
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v6 10/16] tools: implement the new libxc get hw info interface
This patch implements a new libxc get hw info interface and corresponding
data structures. It also changes libxl_psr.c to call this new interface.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Wei Liu
---
CC: Wei Liu
CC: Ian Jackson
CC: Roger Pau Monné
CC: Chao Peng
v6:
- remove unnecessary spaces in brackets.
(suggested by Wei Liu)
- use assert to check input lvl.
(suggested by Roger Pau Monné)
v5:
- directly define 'xc_psr_hw_info' as union type.
(suggested by Roger Pau Monné)
- converge L2 and L3 cases in 'xc_psr_get_hw_info'.
(suggested by Roger Pau Monné)
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- remove 'XC_PSR_FEAT_UNKNOWN' which is not necessary.
(suggested by Roger Pau Monné)
- remove 'FEAT_' from enum item names.
(suggested by Roger Pau Monné)
- remove 'xc_' from struct name.
(suggested by Roger Pau Monné)
- adjust codes to reduce indentation.
(suggested by Roger Pau Monné)
- assert for not happened case.
(suggested by Roger Pau Monné)
- add LOGE to show errno.
(suggested by Roger Pau Monné)
v3:
- rename 'psr_cat/PSR_CAT' to 'psr_alloc/PSR_ALLOC' and remove 'op/OP'
from name.
(suggested by Roger Pau Monné)
- remove 'info' from 'xc_cat_info' and 'xc_mba_info'.
(suggested by Roger Pau Monné)
- set errno in 'xc_psr_get_hw_info'.
(suggested by Roger Pau Monné)
- remove 'inline'.
(suggested by Roger Pau Monné)
- remove 'psr' from 'libxl__psr_feat_type_to_libxc_psr_feat_type' to make
function name shorter.
(suggested by Roger Pau Monné)
- check 'xc_type' in 'libxl_psr_cat_get_info'.
(suggested by Roger Pau Monné)
v2:
- split this patch out from a big patch in v1.
(suggested by Wei Liu)
- change 'CAT_INFO' and 'MBA_INFO' to 'CAT' and 'MBA'.
(suggested by Chao Peng)
---
tools/libxc/include/xenctrl.h | 27 ++---
tools/libxc/xc_psr.c | 55 +++
tools/libxl/libxl_psr.c | 38 --
3 files changed, 95 insertions(+), 25 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index b970905..2d977c8 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2474,6 +2474,28 @@ enum xc_psr_cat_type {
};
typedef enum xc_psr_cat_type xc_psr_cat_type;
+enum xc_psr_feat_type {
+XC_PSR_CAT_L3,
+XC_PSR_CAT_L2,
+XC_PSR_MBA,
+};
+typedef enum xc_psr_feat_type xc_psr_feat_type;
+
+union xc_psr_hw_info {
+struct {
+uint32_t cos_max;
+uint32_t cbm_len;
+bool cdp_enabled;
+} cat;
+
+struct {
+uint32_t cos_max;
+uint32_t thrtl_max;
+bool linear;
+} mba;
+};
+typedef union xc_psr_hw_info xc_psr_hw_info;
+
int xc_psr_cmt_attach(xc_interface *xch, uint32_t domid);
int xc_psr_cmt_detach(xc_interface *xch, uint32_t domid);
int xc_psr_cmt_get_domain_rmid(xc_interface *xch, uint32_t domid,
@@ -2495,9 +2517,8 @@ int xc_psr_cat_set_domain_data(xc_interface *xch,
uint32_t domid,
int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
xc_psr_cat_type type, uint32_t target,
uint64_t *data);
-int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
-uint32_t *cos_max, uint32_t *cbm_len,
-bool *cdp_enabled);
+int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
+ xc_psr_feat_type type, xc_psr_hw_info *hw_info);
int xc_get_cpu_levelling_caps(xc_interface *xch, uint32_t *caps);
int xc_get_cpu_featureset(xc_interface *xch, uint32_t index,
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 5c54a35..2c605a7 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -323,37 +323,52 @@ int xc_psr_cat_get_domain_data(xc_interface *xch,
uint32_t domid,
return rc;
}
-int xc_psr_cat_get_info(xc_interface *xch, uint32_t socket, unsigned int lvl,
-uint32_t *cos_max, uint32_t *cbm_len, bool
*cdp_enabled)
+int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
+ xc_psr_feat_type type, xc_psr_hw_info *hw_info)
{
int rc = -1;
DECLARE_SYSCTL;
+if ( !hw_info )
+{
+errno = EINVAL;
+return rc;
+}
+
sysctl.cmd = XEN_SYSCTL_psr_alloc;
sysctl.u.psr_alloc.target = socket;
-switch ( lvl )
+switch ( type )
{
-case 2:
-sysctl.u.psr_alloc.cmd = XEN_SYSCTL_PSR_get_l2_info;
+case XC_PSR_CAT_L2:
+case XC_PSR_CAT_L3:
+
[Xen-devel] [PATCH v6 01/16] docs: create Memory Bandwidth Allocation (MBA) feature document
This patch creates MBA feature document in doc/features/. It describes key points to implement MBA which is described in details in Intel SDM "Introduction to Memory Bandwidth Allocation". Signed-off-by: Yi Sun Reviewed-by: Roger Pau Monné --- CC: Jan Beulich CC: Andrew Cooper CC: Wei Liu CC: Ian Jackson CC: Daniel De Graaf CC: Roger Pau Monné CC: Konrad Rzeszutek Wilk CC: Chao Peng CC: Julien Grall v6: - fix some words. (suggested by Roger Pau Monné) v5: - correct some words. (suggested by Roger Pau Monné) - change 'xl psr-mba-set 1 0xa' to 'xl psr-mba-set 1 10'. (suggested by Roger Pau Monné) v4: - add 'domain-name' as parameter of 'psr-mba-show/psr-mba-set'. (suggested by Roger Pau Monné) - fix some wordings. (suggested by Roger Pau Monné) - explain how user can know the MBA_MAX. (suggested by Roger Pau Monné) - move the description of 'Linear mode/Non-linear mode' into section of 'psr-mba-show'. (suggested by Roger Pau Monné) - change 'per-thread' to 'per-hyper-thread' to make it clearer. (suggested by Roger Pau Monné) - upgrade revision number. v3: - remove 'closed-loop' related description. (suggested by Roger Pau Monné) - explain 'linear' and 'non-linear' before mentioning them. (suggested by Roger Pau Monné) - adjust desription of 'psr-mba-set'. (suggested by Roger Pau Monné) - explain 'MBA_MAX'. (suggested by Roger Pau Monné) - remove 'n<64'. (suggested by Roger Pau Monné) - fix some wordings. (suggested by Roger Pau Monné) - add context in 'Testing' part to make things more clear. (suggested by Roger Pau Monné) v2: - declare 'HW' in Terminology. (suggested by Chao Peng) - replace 'COS ID of VCPU' to 'COS ID of domain'. (suggested by Chao Peng) - replace 'COS register' to 'Thrtl MSR'. (suggested by Chao Peng) - add description for 'psr-mba-show' to state that the decimal value is shown for linear mode but hexadecimal value is shown for non-linear mode. (suggested by Chao Peng) - remove content in 'Areas for improvement'. (suggested by Chao Peng) - use '<>' to specify mandatory argument to a command. (suggested by Wei Liu) v1: - remove a special character to avoid the error when building pandoc. --- docs/features/intel_psr_mba.pandoc | 297 + 1 file changed, 297 insertions(+) create mode 100644 docs/features/intel_psr_mba.pandoc diff --git a/docs/features/intel_psr_mba.pandoc b/docs/features/intel_psr_mba.pandoc new file mode 100644 index 000..86df661 --- /dev/null +++ b/docs/features/intel_psr_mba.pandoc @@ -0,0 +1,297 @@ +% Intel Memory Bandwidth Allocation (MBA) Feature +% Revision 1.8 + +\clearpage + +# Basics + + + Status: **Tech Preview** + +Architecture(s): Intel x86 + + Component(s): Hypervisor, toolstack + + Hardware: MBA is supported on Skylake Server and beyond + + +# Terminology + +* CAT Cache Allocation Technology +* CBM Capacity BitMasks +* CDP Code and Data Prioritization +* COS/CLOSClass of Service +* HW Hardware +* MBA Memory Bandwidth Allocation +* MSRsMachine Specific Registers +* PSR Intel Platform Shared Resource +* THRTL Throttle value or delay value + +# Overview + +The Memory Bandwidth Allocation (MBA) feature provides indirect and approximate +control over memory bandwidth available per-core. This feature provides OS/ +hypervisor the ability to slow misbehaving apps/domains by using a credit-based +throttling mechanism. + +# User details + +* Feature Enabling: + + Add "psr=mba" to boot line parameter to enable MBA feature. + +* xl interfaces: + + 1. `psr-mba-show [domain-id|domain-name]`: + + Show memory bandwidth throttling for domain. Under different modes, it + shows different type of data. + + There are two modes: + Linear mode: the input precision is defined as 100-(MBA_MAX). For instance, + if the MBA_MAX value is 90, the input precision is 10%. Values not an even + multiple of the precision (e.g., 12%) will be rounded down (e.g., to 10% + delay applied) by HW automatically. The response of throttling value is + linear. + + Non-linear mode: input delay values are powers-of-two from zero to the + MBA_MAX value from CPUID. In this case any values not a power of two will + be rounded down the next nearest power of two by HW automatically. The + response of throttli
[Xen-devel] [PATCH v6 05/16] x86: implement data structure and CPU init flow for MBA
This patch implements main data structures of MBA.
Like CAT features, MBA HW info has cos_max which means the max thrtl
register number, and thrtl_max which means the max throttle value
(delay value). It also has a flag to represent if the throttle
value is linear or non-linear.
One thrtl register of MBA stores a throttle value for one or more
domains. The throttle value means the delay between L2 cache and next
cache level.
This patch also implements init flow for MBA and register stub
callback functions.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v6:
- restore type of 'mba_write_msr' to 'void'.
v5:
- move out some CAT codes optimization to a new patch.
(suggested by Jan Beulich)
- modify commit message.
(suggested by Jan Beulich)
- change print type of 'linear' to be %d.
(suggested by Jan Beulich)
- change type of 'mba_write_msr' to uint32_t.
- move printk in 'mba_init_feature' to reduce one return path.
(suggested by Roger Pau Monné)
- move the MBA format string in printk to a new line.
(suggested by Roger Pau Monné)
v4:
- modify commit message.
(suggested by Roger Pau Monné)
- fix a comment.
(suggested by Roger Pau Monné)
- join two checks in a single if.
(suggested by Roger Pau Monné)
- remove redundant initialization of 'feat->cos_reg_val[0]'.
(suggested by Roger Pau Monné)
- change 'reg_b' to 'ebx'.
(suggested by Jan Beulich)
- change type of 'mba_init_feature' from 'int' to 'bool'.
(suggested by Roger Pau Monné)
- change type of 'cat_init_feature' from 'int' to 'bool'.
v3:
- replace 'psr_val_type' to 'psr_type'. Also, change 'PSR_VAL_TYPE_MBA' to
'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
- replace 'MBA_LINEAR' to 'MBA_LINEAR_MASK' to make the name more clear.
(suggested by Roger Pau Monné)
- replase 'cat_info'/'mba_info' to 'cat'/'mba' to make the names shorter.
(suggested by Roger Pau Monné)
- change type of 'linear' to 'bool'.
(suggested by Roger Pau Monné)
- make format string of printf in one line.
(suggested by Roger Pau Monné)
v2:
- modify commit message to replace 'cos register' to 'thrtl register' to
make it accurate.
(suggested by Chao Peng)
- restore the place of the sentence to assign value to 'feat->cbm_len'
because the MBA init flow is splitted out as a separate function in v1.
(suggested by Chao Peng)
- add comment to explain what the MBA thrtl defaul value '0' stands for.
(suggested by Chao Peng)
- check 'thrtl_max' under linear mode. It could not be euqal or larger than
100.
(suggested by Chao Peng)
v1:
- rebase codes onto L2 CAT v15.
- move comment to appropriate place.
(suggested by Chao Peng)
- implement 'mba_init_feature' and keep 'cat_init_feature'.
(suggested by Chao Peng)
- keep 'regs.b' into a local variable to avoid reading CPUID every time.
(suggested by Chao Peng)
---
xen/arch/x86/psr.c | 135 ++--
xen/include/asm-x86/msr-index.h | 1 +
xen/include/asm-x86/psr.h | 2 +
3 files changed, 119 insertions(+), 19 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 5945efa..157e11f 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -27,13 +27,16 @@
* - CMT Cache Monitoring Technology
* - COS/CLOSClass of Service. Also mean COS registers.
* - COS_MAX Max number of COS for the feature (minus 1)
+ * - MBA Memory Bandwidth Allocation
* - MSRsMachine Specific Registers
* - PSR Intel Platform Shared Resource
+ * - THRTL_MAX Max throttle value (delay value) of MBA
*/
#define PSR_CMT(1u << 0)
#define PSR_CAT(1u << 1)
#define PSR_CDP(1u << 2)
+#define PSR_MBA(1u << 3)
#define CAT_CBM_LEN_MASK 0x1f
#define CAT_COS_MAX_MASK 0x
@@ -60,10 +63,14 @@
*/
#define MAX_COS_NUM 2
+#define MBA_LINEAR_MASK(1u << 2)
+#define MBA_THRTL_MAX_MASK 0xfff
+
enum psr_feat_type {
FEAT_TYPE_L3_CAT,
FEAT_TYPE_L3_CDP,
FEAT_TYPE_L2_CAT,
+FEAT_TYPE_MBA,
FEAT_TYPE_NUM,
FEAT_TYPE_UNKNOWN,
};
@@ -71,7 +78,6 @@ enum psr_feat_type {
/*
* This structure represents one feature.
* cos_max - The max COS registers number got through CPUID.
- * cbm_len - The length of CBM got through CPUID.
* cos_reg_val - Array to store the values o
[Xen-devel] [PATCH v6 03/16] x86: rename 'cbm_type' to 'psr_type' to make it general
This patch renames 'cbm_type' to 'psr_type' to generalize it.
Then, we can reuse this for all psr allocation features.
Signed-off-by: Yi Sun
Reviewed-by: Wei Liu
Reviewed-by: Roger Pau Monné
Acked-by: Jan Beulich
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v5:
- correct character of reviewer's name.
(suggested by Jan Beulich)
v4:
- fix words in commit message.
(suggested by Roger Pau Monné)
v3:
- replace 'psr_val_type' to 'psr_type' and remove '_VAL' from the enum
items.
(suggested by Roger Pau Monné)
v2:
- replace 'PSR_VAL_TYPE_{L3, L2}' to 'PSR_VAL_TYPE_{L3, L2}_CBM'.
(suggested by Chao Peng)
---
xen/arch/x86/domctl.c | 16 ++--
xen/arch/x86/psr.c| 62 +--
xen/arch/x86/sysctl.c | 4 +--
xen/include/asm-x86/psr.h | 18 +++---
4 files changed, 52 insertions(+), 48 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 8a6004b..0cd18a6 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1454,41 +1454,41 @@ long arch_do_domctl(
case XEN_DOMCTL_PSR_SET_L3_CBM:
ret = psr_set_val(d, domctl->u.psr_alloc.target,
domctl->u.psr_alloc.data,
- PSR_CBM_TYPE_L3);
+ PSR_TYPE_L3_CBM);
break;
case XEN_DOMCTL_PSR_SET_L3_CODE:
ret = psr_set_val(d, domctl->u.psr_alloc.target,
domctl->u.psr_alloc.data,
- PSR_CBM_TYPE_L3_CODE);
+ PSR_TYPE_L3_CODE);
break;
case XEN_DOMCTL_PSR_SET_L3_DATA:
ret = psr_set_val(d, domctl->u.psr_alloc.target,
domctl->u.psr_alloc.data,
- PSR_CBM_TYPE_L3_DATA);
+ PSR_TYPE_L3_DATA);
break;
case XEN_DOMCTL_PSR_SET_L2_CBM:
ret = psr_set_val(d, domctl->u.psr_alloc.target,
domctl->u.psr_alloc.data,
- PSR_CBM_TYPE_L2);
+ PSR_TYPE_L2_CBM);
break;
case XEN_DOMCTL_PSR_GET_L3_CBM:
-ret = domctl_psr_get_val(d, domctl, PSR_CBM_TYPE_L3, copyback);
+ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L3_CBM, copyback);
break;
case XEN_DOMCTL_PSR_GET_L3_CODE:
-ret = domctl_psr_get_val(d, domctl, PSR_CBM_TYPE_L3_CODE,
copyback);
+ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L3_CODE, copyback);
break;
case XEN_DOMCTL_PSR_GET_L3_DATA:
-ret = domctl_psr_get_val(d, domctl, PSR_CBM_TYPE_L3_DATA,
copyback);
+ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L3_DATA, copyback);
break;
case XEN_DOMCTL_PSR_GET_L2_CBM:
-ret = domctl_psr_get_val(d, domctl, PSR_CBM_TYPE_L2, copyback);
+ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L2_CBM, copyback);
break;
default:
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index dbf7a4c..263ddbe 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -100,24 +100,24 @@ static const struct feat_props {
unsigned int cos_num;
/*
- * An array to save all 'enum cbm_type' values of the feature. It is
+ * An array to save all 'enum psr_type' values of the feature. It is
* used with cos_num together to get/write a feature's COS registers
* values one by one.
*/
-enum cbm_type type[MAX_COS_NUM];
+enum psr_type type[MAX_COS_NUM];
/*
* alt_type is 'alternative type'. When this 'alt_type' is input, the
* feature does some special operations.
*/
-enum cbm_type alt_type;
+enum psr_type alt_type;
/* get_feat_info is used to return feature HW info through sysctl. */
bool (*get_feat_info)(const struct feat_node *feat,
uint32_t data[], unsigned int array_len);
/* write_msr is used to write out feature MSR register. */
-void (*write_msr)(unsigned int cos, uint32_t val, enum cbm_type type);
+void (*write_msr)(unsigned int cos, uint32_t val, enum psr_type type);
} *feat_props[FEAT_TYPE_NUM];
/*
@@ -215,13 +215,13 @@ static void free_socket_resources(unsigned int socket)
bitmap_zero(info->dom_set, DOMID_IDLE + 1);
}
-static enum psr_feat_type psr_cbm_type_to_feat_type(enum cbm_type type)
+static enum psr_feat_type psr_type_to_feat_type(enum psr_type type)
{
enum psr_feat_type feat_type = FEAT_TYPE_UNKNOWN;
switch ( type )
{
-case PSR_CBM_TYPE_L3:
+case PSR_TYPE_L3_CBM:
feat_type = FEAT_TYPE
[Xen-devel] [PATCH v6 12/16] tools: implement the new xl get hw info interface
This patch implements a new xl get HW info interface. A new argument
is added for psr-hwinfo command to get and show MBA HW info.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
Acked-by: Wei Liu
---
CC: Wei Liu
CC: Ian Jackson
CC: Roger Pau Monné
CC: Chao Peng
v3:
- change the format string of printf in 'psr_mba_hwinfo'.
(suggested by Roger Pau Monné)
- add 'const' for 'opts[]' in 'main_psr_hwinfo'.
(suggested by Roger Pau Monné)
v2:
- split out this patch from a big patch in v1.
(suggested by Wei Liu)
- change 'MBA_INFO' to 'MBA'. Also, change 'mba_info' to 'mba'.
(suggested by Chao Peng)
---
tools/xl/xl_cmdtable.c | 1 +
tools/xl/xl_psr.c | 39 ---
2 files changed, 37 insertions(+), 3 deletions(-)
diff --git a/tools/xl/xl_cmdtable.c b/tools/xl/xl_cmdtable.c
index c304a85..dbbfc02 100644
--- a/tools/xl/xl_cmdtable.c
+++ b/tools/xl/xl_cmdtable.c
@@ -549,6 +549,7 @@ struct cmd_spec cmd_table[] = {
"[options]",
"-m, --cmt Show Cache Monitoring Technology (CMT) hardware info\n"
"-a, --cat Show Cache Allocation Technology (CAT) hardware info\n"
+ "-b, --mba Show Memory Bandwidth Allocation (MBA) hardware info\n"
},
{ "psr-cmt-attach",
&main_psr_cmt_attach, 0, 1,
diff --git a/tools/xl/xl_psr.c b/tools/xl/xl_psr.c
index ef00048..ab47d96 100644
--- a/tools/xl/xl_psr.c
+++ b/tools/xl/xl_psr.c
@@ -475,6 +475,31 @@ static int psr_l2_cat_hwinfo(void)
return rc;
}
+static int psr_mba_hwinfo(void)
+{
+int rc;
+unsigned int i, nr;
+libxl_psr_hw_info *info;
+
+rc = libxl_psr_get_hw_info(ctx, LIBXL_PSR_FEAT_TYPE_MBA, 0, &nr, &info);
+if (rc)
+return rc;
+
+printf("Memory Bandwidth Allocation (MBA):\n");
+
+for (i = 0; i < nr; i++) {
+printf("Socket ID : %u\n", info[i].id);
+printf("Linear Mode : %s\n",
+ info[i].u.mba.linear ? "Enabled" : "Disabled");
+printf("Maximum COS : %u\n", info[i].u.mba.cos_max);
+printf("Maximum Throttling Value: %u\n", info[i].u.mba.thrtl_max);
+printf("Default Throttling Value: %u\n", 0);
+}
+
+libxl_psr_hw_info_list_free(info, nr);
+return rc;
+}
+
int main_psr_cat_cbm_set(int argc, char **argv)
{
uint32_t domid;
@@ -593,20 +618,24 @@ int main_psr_cat_show(int argc, char **argv)
int main_psr_hwinfo(int argc, char **argv)
{
int opt, ret = 0;
-bool all = true, cmt = false, cat = false;
-static struct option opts[] = {
+bool all = true, cmt = false, cat = false, mba = false;
+static const struct option opts[] = {
{"cmt", 0, 0, 'm'},
{"cat", 0, 0, 'a'},
+{"mba", 0, 0, 'b'},
COMMON_LONG_OPTS
};
-SWITCH_FOREACH_OPT(opt, "ma", opts, "psr-hwinfo", 0) {
+SWITCH_FOREACH_OPT(opt, "mab", opts, "psr-hwinfo", 0) {
case 'm':
all = false; cmt = true;
break;
case 'a':
all = false; cat = true;
break;
+case 'b':
+all = false; mba = true;
+break;
}
if (!ret && (all || cmt))
@@ -619,6 +648,10 @@ int main_psr_hwinfo(int argc, char **argv)
if (all || cat)
ret = psr_l2_cat_hwinfo();
+/* MBA is independent of CMT and CAT */
+if (all || mba)
+ret = psr_mba_hwinfo();
+
return ret;
}
--
1.9.1
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v6 08/16] x86: implement set value flow for MBA
This patch implements set value flow for MBA including its callback
function and domctl interface.
Signed-off-by: Yi Sun
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v6:
- split co-exist features' values setting flow to a new patch.
(suggested by Jan Beulich)
- restore codes related to 'mba_check_thrtl' and 'check_value'.
(suggested by Jan Beulich)
v5:
- adjust position of 'cat_check_cbm' to not to make changes so big.
(suggested by Roger Pau Monné)
- remove 'props' from 'struct cos_write_info'.
(suggested by Roger Pau Monné)
- make a single return statement in 'mba_check_thrtl'.
(suggested by Jan Beulich)
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- join two checks into a single if.
(suggested by Roger Pau Monné)
- remove redundant local variable 'array_len'.
(suggested by Roger Pau Monné)
v3:
- modify commit message to make it clear.
(suggested by Roger Pau Monné)
- modify functionality of 'check_val' to make it simple to only check value.
Change the last parameter type from 'unsigned long *' to 'unsigned long'.
(suggested by Roger Pau Monné)
- call rdmsrl to get value just written into MSR for MBA. Because HW can
automatically change input value to what it wants.
(suggested by Roger Pau Monné)
- change type of 'write_msr' to 'uint32_t' to return the value actually
written into MSR. Then, change 'do_write_psr_msrs' to set the returned
value into 'cos_reg_val[]'
- move the declaration of 'j' into loop in 'do_write_psr_msrs'.
(suggested by Roger Pau Monné)
- change 'mba_info' to 'mba'.
(suggested by Roger Pau Monné)
- change 'cat_info' to 'cat'.
(suggested by Roger Pau Monné)
- rename 'psr_cat/PSR_CAT' to 'psr_alloc/PSR_ALLOC' and remove 'op/OP'
from name.
(suggested by Roger Pau Monné)
- change 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
v2:
- remove linear mode 'thrtl_max' check in 'mba_check_thrtl' because it has
been checked in 'mba_init_feature'.
(suggested by Chao Peng)
- for non-linear mode, check if '*thrtl' is not 0 in 'mba_check_thrtl'. If
it is 0, we do not need to change it.
(suggested by Chao Peng)
- move comments to explain changes of 'cos_write_info' from psr.c to commit
message.
(suggested by Chao Peng)
---
xen/arch/x86/domctl.c | 6
xen/arch/x86/psr.c | 74 +
xen/include/public/domctl.h | 1 +
3 files changed, 69 insertions(+), 12 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 17fd3ad..bbfd76e 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1475,6 +1475,12 @@ long arch_do_domctl(
PSR_TYPE_L2_CBM);
break;
+case XEN_DOMCTL_PSR_SET_MBA_THRTL:
+ret = psr_set_val(d, domctl->u.psr_alloc.target,
+ domctl->u.psr_alloc.data,
+ PSR_TYPE_MBA_THRTL);
+break;
+
case XEN_DOMCTL_PSR_GET_L3_CBM:
ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L3_CBM, copyback);
break;
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 03f24c0..cffb377 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -138,6 +138,12 @@ static const struct feat_props {
/* write_msr is used to write out feature MSR register. */
void (*write_msr)(unsigned int cos, uint32_t val, enum psr_type type);
+
+/*
+ * check_val is used to check if input val fulfills SDM requirement.
+ * Change it to valid value if SDM allows.
+ */
+bool (*check_val)(const struct feat_node *feat, unsigned long *val);
} *feat_props[FEAT_TYPE_NUM];
/*
@@ -274,30 +280,30 @@ static enum psr_feat_type psr_type_to_feat_type(enum
psr_type type)
return feat_type;
}
-static bool psr_check_cbm(unsigned int cbm_len, unsigned long cbm)
+/* Implementation of allocation features' functions. */
+static bool cat_check_cbm(const struct feat_node *feat, unsigned long *cbm)
{
unsigned int first_bit, zero_bit;
+unsigned int cbm_len = feat->cat.cbm_len;
-/* Set bits should only in the range of [0, cbm_len]. */
-if ( cbm & (~0ul << cbm_len) )
-return false;
-
-/* At least one bit need to be set. */
-if ( cbm == 0 )
+/*
+ * Set bits should only in the range of [0, cbm_len].
+ * And, at least one bit need to
[Xen-devel] [PATCH v6 14/16] tools: implement new generic get value interface and MBA get value command
This patch implements generic get value interfaces in libxc and libxl.
It also refactors the get value flow in xl to make it be suitable for all
allocation features. Based on that, a new MBA get value command is added in xl.
Signed-off-by: Yi Sun
Acked-by: Wei Liu
Reviewed-by: Roger Pau Monné
---
CC: Ian Jackson
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v6:
- fix one coding style issue.
(suggested by Roger Pau Monné)
v5:
- start a newline for "CDP" because it exceeds 80 characters.
(suggested by Roger Pau Monné)
- remove a duplicated ';'.
(suggested by Roger Pau Monné)
- remove a extra newline.
(suggested by Roger Pau Monné)
- correct words in log message.
(suggested by Roger Pau Monné)
v4:
- use designated initializers for 'feat_name[]'.
(suggested by Roger Pau Monné)
- use LOG in 'libxl__psr_alloc_log_err_msg'.
(suggested by Roger Pau Monné)
v3:
- replace 'libxl_psr_cbm_type' to 'libxl_psr_type' in newly defined
interfaces.
(suggested by Roger Pau Monné)
v2:
- change 'CAT_INFO'/'MBA_INFO' to 'CAT'/'MBA'. The related structure names
are changed too.
(suggested by Chao Peng)
---
tools/libxc/include/xenctrl.h | 7 +-
tools/libxc/xc_psr.c | 9 +-
tools/libxl/libxl_psr.c | 58 -
tools/xl/xl.h | 1 +
tools/xl/xl_cmdtable.c| 5 ++
tools/xl/xl_psr.c | 185 ++
6 files changed, 183 insertions(+), 82 deletions(-)
diff --git a/tools/libxc/include/xenctrl.h b/tools/libxc/include/xenctrl.h
index 2736bc5..dcea09e 100644
--- a/tools/libxc/include/xenctrl.h
+++ b/tools/libxc/include/xenctrl.h
@@ -2471,6 +2471,7 @@ enum xc_psr_type {
XC_PSR_CAT_L3_CBM_CODE = 2,
XC_PSR_CAT_L3_CBM_DATA = 3,
XC_PSR_CAT_L2_CBM = 4,
+XC_PSR_MBA_THRTL = 5,
};
typedef enum xc_psr_type xc_psr_type;
@@ -2514,9 +2515,9 @@ int xc_psr_cmt_enabled(xc_interface *xch);
int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t domid,
xc_psr_type type, uint32_t target,
uint64_t data);
-int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_type type, uint32_t target,
- uint64_t *data);
+int xc_psr_get_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_type type, uint32_t target,
+ uint64_t *data);
int xc_psr_get_hw_info(xc_interface *xch, uint32_t socket,
xc_psr_feat_type type, xc_psr_hw_info *hw_info);
diff --git a/tools/libxc/xc_psr.c b/tools/libxc/xc_psr.c
index 01f4ba7..191de97 100644
--- a/tools/libxc/xc_psr.c
+++ b/tools/libxc/xc_psr.c
@@ -283,9 +283,9 @@ int xc_psr_cat_set_domain_data(xc_interface *xch, uint32_t
domid,
return do_domctl(xch, &domctl);
}
-int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t domid,
- xc_psr_type type, uint32_t target,
- uint64_t *data)
+int xc_psr_get_domain_data(xc_interface *xch, uint32_t domid,
+ xc_psr_type type, uint32_t target,
+ uint64_t *data)
{
int rc;
DECLARE_DOMCTL;
@@ -305,6 +305,9 @@ int xc_psr_cat_get_domain_data(xc_interface *xch, uint32_t
domid,
case XC_PSR_CAT_L2_CBM:
cmd = XEN_DOMCTL_PSR_GET_L2_CBM;
break;
+case XC_PSR_MBA_THRTL:
+cmd = XEN_DOMCTL_PSR_GET_MBA_THRTL;
+break;
default:
errno = EINVAL;
return -1;
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index c54cb6f..7c560bc 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -71,16 +71,30 @@ static void libxl__psr_cmt_log_err_msg(libxl__gc *gc, int
err)
LOGE(ERROR, "%s", msg);
}
-static void libxl__psr_cat_log_err_msg(libxl__gc *gc, int err)
+static void libxl__psr_alloc_log_err_msg(libxl__gc *gc,
+ int err,
+ libxl_psr_type type)
{
+/*
+ * Index is 'libxl_psr_type' so we set two 'CDP' to correspond to
+ * DATA and CODE.
+ */
+const char * const feat_name[] = {
+[LIBXL_PSR_CBM_TYPE_UNKNOWN] = "UNKNOWN",
+[LIBXL_PSR_CBM_TYPE_L3_CBM] = "L3 CAT",
+[LIBXL_PSR_CBM_TYPE_L3_CBM_CODE...LIBXL_PSR_CBM_TYPE_L3_CBM_DATA] =
+ "CDP",
+[LIBXL_PSR_CBM_TYPE_L2_CBM] = "L2 CAT",
+[LIBXL_PSR_CBM_TYPE_MBA_THRTL] = "MBA",
+};
char *msg;
switch (err) {
case ENODEV:
-msg = "CAT is not supported in this system";
+msg = "is not supported
[Xen-devel] [PATCH v2] x86: psr: support co-exist features' values setting
It changes the memebers in 'cos_write_info' to transfer the feature array,
feature properties array and value array. Then, we can write all features
values on the cos id into MSRs.
Because multiple features may co-exist, we need handle all features to write
values of them into a COS register with new COS ID. E.g:
1. L3 CAT and L2 CAT co-exist.
2. Dom1 and Dom2 share the same COS ID (2). The L3 CAT CBM of Dom1 is 0x1ff,
the L2 CAT CBM of Dom1 is 0x1f.
3. User wants to change L2 CBM of Dom1 to be 0xf. Because COS ID 2 is
used by Dom2 too, we have to pick a new COS ID 3. The values of Dom1 on
COS ID 3 are all default values as below:
-
| COS 3 |
-
L3 CAT | 0x7ff |
-
L2 CAT | 0xff |
-
4. After setting, the L3 CAT CBM value of Dom1 should be kept and the new L2
CAT CBM is set. So, the values on COS ID 3 should be below.
-
| COS 3 |
-
L3 CAT | 0x1ff |
-
L2 CAT | 0xf |
-
Signed-off-by: Yi Sun
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Julien Grall
---
xen/arch/x86/psr.c | 54 ++
1 file changed, 30 insertions(+), 24 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index daa2aeb..dbf7a4c 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -,25 +,43 @@ static unsigned int get_socket_cpu(unsigned int socket)
struct cos_write_info
{
unsigned int cos;
-struct feat_node *feature;
+struct feat_node **features;
const uint32_t *val;
-const struct feat_props *props;
+unsigned int array_len;
};
static void do_write_psr_msrs(void *data)
{
const struct cos_write_info *info = data;
-struct feat_node *feat = info->feature;
-const struct feat_props *props = info->props;
-unsigned int i, cos = info->cos, cos_num = props->cos_num;
+unsigned int i, index = 0, cos = info->cos;
-for ( i = 0; i < cos_num; i++ )
+/*
+ * Iterate all featuers to write different value (not same as MSR) for
+ * each feature.
+ */
+for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
{
-if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
+struct feat_node *feat = info->features[i];
+const struct feat_props *props = feat_props[i];
+unsigned int cos_num, j;
+
+if ( !feat || !props )
+continue;
+
+cos_num = props->cos_num;
+if ( info->array_len < index + cos_num )
+return;
+
+for ( j = 0; j < cos_num; j++ )
{
-feat->cos_reg_val[cos * cos_num + i] = info->val[i];
-props->write_msr(cos, info->val[i], props->type[i]);
+if ( feat->cos_reg_val[cos * cos_num + j] != info->val[index + j] )
+{
+feat->cos_reg_val[cos * cos_num + j] = info->val[index + j];
+props->write_msr(cos, info->val[index + j], props->type[j]);
+}
}
+
+index += cos_num;
}
}
@@ -1137,30 +1155,18 @@ static int write_psr_msrs(unsigned int socket, unsigned
int cos,
const uint32_t val[], unsigned int array_len,
enum psr_feat_type feat_type)
{
-int ret;
struct psr_socket_info *info = get_socket_info(socket);
struct cos_write_info data =
{
.cos = cos,
-.feature = info->features[feat_type],
-.props = feat_props[feat_type],
+.features = info->features,
+.val = val,
+.array_len = array_len,
};
if ( cos > info->features[feat_type]->cos_max )
return -EINVAL;
-/* Skip to the feature's value head. */
-ret = skip_prior_features(&array_len, feat_type);
-if ( ret < 0 )
-return ret;
-
-val += ret;
-
-if ( array_len < feat_props[feat_type]->cos_num )
-return -ENOSPC;
-
-data.val = val;
-
if ( socket == cpu_to_socket(smp_processor_id()) )
do_write_psr_msrs(&data);
else
--
1.9.1
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v1] x86: psr: support co-exist features' values setting
On 17-10-06 15:38:35, Roger Pau Monn� wrote:
> On Fri, Oct 06, 2017 at 09:13:00AM +0000, Yi Sun wrote:
> > It changes the memebers in 'cos_write_info' to transfer the feature array,
> > feature properties array and value array. Then, we can write all features
> > values on the cos id into MSRs.
> >
> > Because multiple features may co-exist, we need handle all features to write
> > values of them into a COS register with new COS ID. E.g:
> > 1. L3 CAT and L2 CAT co-exist.
> > 2. Dom1 and Dom2 share a same COS ID (2). The L3 CAT CBM of Dom1 is 0x1ff,
> ^ the
> >the L2 CAT CBM of Dom1 is 0x1f.
> > 3. User wants to change L2 CBM of Dom1 to be 0xf. Because COS ID 2 is
> >used by Dom2 too, we have to pick a new COS ID 3. The values of Dom1 on
> >COS ID 3 are all default values as below:
> >-
> >| COS 3 |
> >-
> >L3 CAT | 0x7ff |
> >-
> >L2 CAT | 0xff |
> >-
> > 4. After setting, the L3 CAT CBM value of Dom1 should be kept and the new L2
> >CAT CBM is set. So, the values on COS ID 3 should be below.
> >-
> >| COS 3 |
> >-
> >L3 CAT | 0x1ff |
> >-
> >L2 CAT | 0xf |
> >-
> >
> > So, we should write all features values into their MSRs. That requires the
> > feature array, feature properties array and value array are input.
> ^ as?
>
> I'm not sure the last sentence is helpful.
>
Ok, will remove it.
> >
> > Signed-off-by: Yi Sun
> > ---
> > CC: Jan Beulich
> > CC: Andrew Cooper
> > CC: Wei Liu
> > CC: Roger Pau Monn?
> > CC: Julien Grall
> > ---
> > xen/arch/x86/psr.c | 51 +++
> > 1 file changed, 27 insertions(+), 24 deletions(-)
> >
> > diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
> > index daa2aeb..596b0ca 100644
> > --- a/xen/arch/x86/psr.c
> > +++ b/xen/arch/x86/psr.c
> > @@ -,25 +,40 @@ static unsigned int get_socket_cpu(unsigned int
> > socket)
> > struct cos_write_info
> > {
> > unsigned int cos;
> > -struct feat_node *feature;
> > +struct feat_node **features;
> > const uint32_t *val;
> > -const struct feat_props *props;
> > +unsigned int array_len;
> > };
> >
> > static void do_write_psr_msrs(void *data)
> > {
> > const struct cos_write_info *info = data;
> > -struct feat_node *feat = info->feature;
> > -const struct feat_props *props = info->props;
> > -unsigned int i, cos = info->cos, cos_num = props->cos_num;
> > +unsigned int i, index = 0, cos = info->cos;
> > +const uint32_t *val_array = info->val;
> >
> > -for ( i = 0; i < cos_num; i++ )
> > +for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
> > {
> > -if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
> > +struct feat_node *feat = info->features[i];
> > +const struct feat_props *props = feat_props[i];
> > +unsigned int cos_num, j;
> > +
> > +if ( !feat || !props )
> > +continue;
> > +
> > +cos_num = props->cos_num;
> > +if ( info->array_len < index + cos_num )
>
> Shouldn't this be '<='? index + cos_num is an index position with base
> 0 AFAICT.
>
Nope. E.g. there are L2 CAT and CDP co-exist. cos_num of L2 is 1, cos_num of CDP
is 2. CDP is the first element in feature array. array_len is 3.
1. First loop to handle CDP: index is changed from 0 to 2.
2. Second loop to handle L2:
cos_num = 1;
index + cos_num = 3;
array_len = 3;
So, we must use '<' here to check if overflow happens.
> > +return;
> > +
> > +for ( j = 0; j < cos_num; j++ )
> > {
> > -feat->cos_reg_val[cos * cos_num + i] = info->val[i];
> > -props->write_msr(cos, info->val[i], props->type[i]);
> > +if ( feat->cos_reg_val[cos * cos_num + j] != val_array[index +
> > j] )
>
> I'm afraid this code could benefit from a comment (or comments)
> explaining what all this arrays are supposed to contain. IMHO it's not
> trivial to follow what you are trying to do here.
>
Will add comment.
> Also names like val_array are not specially helpful, it's quite clear
> that 'val_array' is an array just by looking at it's usage.
>
Ok, may consider to remove 'val_array'.
> Thanks, Roger.
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v1] x86: psr: support co-exist features' values setting
It changes the memebers in 'cos_write_info' to transfer the feature array,
feature properties array and value array. Then, we can write all features
values on the cos id into MSRs.
Because multiple features may co-exist, we need handle all features to write
values of them into a COS register with new COS ID. E.g:
1. L3 CAT and L2 CAT co-exist.
2. Dom1 and Dom2 share a same COS ID (2). The L3 CAT CBM of Dom1 is 0x1ff,
the L2 CAT CBM of Dom1 is 0x1f.
3. User wants to change L2 CBM of Dom1 to be 0xf. Because COS ID 2 is
used by Dom2 too, we have to pick a new COS ID 3. The values of Dom1 on
COS ID 3 are all default values as below:
-
| COS 3 |
-
L3 CAT | 0x7ff |
-
L2 CAT | 0xff |
-
4. After setting, the L3 CAT CBM value of Dom1 should be kept and the new L2
CAT CBM is set. So, the values on COS ID 3 should be below.
-
| COS 3 |
-
L3 CAT | 0x1ff |
-
L2 CAT | 0xf |
-
So, we should write all features values into their MSRs. That requires the
feature array, feature properties array and value array are input.
Signed-off-by: Yi Sun
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Julien Grall
---
xen/arch/x86/psr.c | 51 +++
1 file changed, 27 insertions(+), 24 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index daa2aeb..596b0ca 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -,25 +,40 @@ static unsigned int get_socket_cpu(unsigned int socket)
struct cos_write_info
{
unsigned int cos;
-struct feat_node *feature;
+struct feat_node **features;
const uint32_t *val;
-const struct feat_props *props;
+unsigned int array_len;
};
static void do_write_psr_msrs(void *data)
{
const struct cos_write_info *info = data;
-struct feat_node *feat = info->feature;
-const struct feat_props *props = info->props;
-unsigned int i, cos = info->cos, cos_num = props->cos_num;
+unsigned int i, index = 0, cos = info->cos;
+const uint32_t *val_array = info->val;
-for ( i = 0; i < cos_num; i++ )
+for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
{
-if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
+struct feat_node *feat = info->features[i];
+const struct feat_props *props = feat_props[i];
+unsigned int cos_num, j;
+
+if ( !feat || !props )
+continue;
+
+cos_num = props->cos_num;
+if ( info->array_len < index + cos_num )
+return;
+
+for ( j = 0; j < cos_num; j++ )
{
-feat->cos_reg_val[cos * cos_num + i] = info->val[i];
-props->write_msr(cos, info->val[i], props->type[i]);
+if ( feat->cos_reg_val[cos * cos_num + j] != val_array[index + j] )
+{
+feat->cos_reg_val[cos * cos_num + j] = val_array[index + j];
+props->write_msr(cos, val_array[index + j], props->type[j]);
+}
}
+
+index += cos_num;
}
}
@@ -1137,30 +1152,18 @@ static int write_psr_msrs(unsigned int socket, unsigned
int cos,
const uint32_t val[], unsigned int array_len,
enum psr_feat_type feat_type)
{
-int ret;
struct psr_socket_info *info = get_socket_info(socket);
struct cos_write_info data =
{
.cos = cos,
-.feature = info->features[feat_type],
-.props = feat_props[feat_type],
+.features = info->features,
+.val = val,
+.array_len = array_len,
};
if ( cos > info->features[feat_type]->cos_max )
return -EINVAL;
-/* Skip to the feature's value head. */
-ret = skip_prior_features(&array_len, feat_type);
-if ( ret < 0 )
-return ret;
-
-val += ret;
-
-if ( array_len < feat_props[feat_type]->cos_num )
-return -ENOSPC;
-
-data.val = val;
-
if ( socket == cpu_to_socket(smp_processor_id()) )
do_write_psr_msrs(&data);
else
--
1.9.1
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v4 07/15] x86: implement set value flow for MBA
On 17-10-05 02:55:17, Jan Beulich wrote:
> >>> On 05.10.17 at 06:48, wrote:
> > On 17-10-03 23:59:46, Jan Beulich wrote:
> >> >>> Yi Sun 09/29/17 4:58 AM >>>
> >> >On 17-09-28 05:36:11, Jan Beulich wrote:
> >> >> >>> On 23.09.17 at 11:48, wrote:
[...]
> >> >> > {
> >> >> > const struct cos_write_info *info = data;
> >> >> > -struct feat_node *feat = info->feature;
> >> >> > -const struct feat_props *props = info->props;
> >> >> > -unsigned int i, cos = info->cos, cos_num = props->cos_num;
> >> >> > +unsigned int i, index = 0, cos = info->cos;
> >> >> > +const uint32_t *val_array = info->val;
> >> >> >
> >> >> > -for ( i = 0; i < cos_num; i++ )
> >> >> > +for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
> >> >> > {
> >> >> > -if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
> >> >> > +struct feat_node *feat = info->features[i];
> >> >> > +const struct feat_props *props = info->props[i];
> >> >> > +unsigned int cos_num, j;
> >> >> > +
> >> >> > +if ( !feat || !props )
> >> >> > +continue;
> >> >> > +
> >> >> > +cos_num = props->cos_num;
> >> >> > +if ( info->array_len < index + cos_num )
> >> >> > +return;
> >> >> > +
> >> >> > +for ( j = 0; j < cos_num; j++ )
> >> >> > {
> >> >> > -feat->cos_reg_val[cos * cos_num + i] = info->val[i];
> >> >> > -props->write_msr(cos, info->val[i], props->type[i]);
> >> >> > +if ( feat->cos_reg_val[cos * cos_num + j] !=
> >> >> > val_array[index + j] )
> >> >> > +feat->cos_reg_val[cos * cos_num + j] =
> >> >> > +props->write_msr(cos, val_array[index + j],
> >> >> > props->type[j]);
> >> >>
> >> >> This renders partly useless the check: If hardware can alter the
> >> >> value, repeatedly requesting the same value to be written will
> >> >> no longer guarantee the MSR write to be skipped. If hardware
> >> >> behavior can't be predicted you may want to consider recording
> >> >> both the value in found by reading back the register written and
> >> >> the value that was written - a match with either would eliminate
> >> >> the need to do the write.
> >> >>
> >> >The hardware behavior is explicitly defined by SDM and mentioned in
> >> >'xl-psr.markdown' and 'intel_psr_mba.pandoc'. User should know that HW
> >> >can alter MBA value if the value is not valid.
> >>
> >> So if hardware behavior is fully defined, why don't you pre-adjust what is
> >> to be written to the value hardware would alter it to?
> >>
> > In previous version of MBA patch set, I pre-adjust the value in
> > 'mba_check_thrtl'.
> > But Roger did not like that. So, the pre-adjust codes are removed.
>
> Could you point me at or repeat the reason(s) of his dislike?
>
Roger has replied.
> >> >This check is not only for MBA but also for CAT features that the HW
> >> >cannot alter CAT value.
> >>
> >> I don't understand this part.
> >>
> > I mean the check here are for all features so we cannot remove it.
>
> I _still_ don't understand: If the check can't be removed (even
> without MBA in mind), then the implication would be that the
> code prior to this series is buggy. In which case I'd expect you to
> submit a standalone bug fix, rather than mixing the fix into here.
>
Ok, I will send out a stand alone patch to fix this.
> >> > Although this check is not a critical check,
> >> >it can prevent some non-necessary MSR write.
> >>
> >> That's my point - while previously all unnecessary writes were avoided,
> >> you now avoid only some.
> >>
> > Without the pre-adjust codes in 'mba_check_thrtl', if user inputs value,
> > e.g.
> > 11/22/33/..., this check cannot prevent the write action. So, only some can
> > be avoided in current codes.
>
> Right. If it's worthwhile avoiding the writes, all of them should be
> avoided when the resulting value isn't going to change. Otherwise
> the write avoidance logic can/should be dropped altogether.
>
Per discussion in other mails, I think I will restore codes in
'mba_check_thrtl'.
> Jan
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v4 07/15] x86: implement set value flow for MBA
On 17-10-05 03:39:06, Jan Beulich wrote: > >>> On 05.10.17 at 10:39, wrote: > > On Thu, Oct 05, 2017 at 04:48:12AM +, Yi Sun wrote: > >> On 17-10-03 23:59:46, Jan Beulich wrote: > >> > >>> Yi Sun 09/29/17 4:58 AM >>> > >> > >On 17-09-28 05:36:11, Jan Beulich wrote: > >> > >> >>> On 23.09.17 at 11:48, wrote: > >> > >> > -feat->cos_reg_val[cos * cos_num + i] = info->val[i]; > >> > >> > -props->write_msr(cos, info->val[i], props->type[i]); > >> > >> > +if ( feat->cos_reg_val[cos * cos_num + j] != > >> > >> > val_array[index + > > j] ) > >> > >> > +feat->cos_reg_val[cos * cos_num + j] = > >> > >> > +props->write_msr(cos, val_array[index + j], > > props->type[j]); > >> > >> > >> > >> This renders partly useless the check: If hardware can alter the > >> > >> value, repeatedly requesting the same value to be written will > >> > >> no longer guarantee the MSR write to be skipped. If hardware > >> > >> behavior can't be predicted you may want to consider recording > >> > >> both the value in found by reading back the register written and > >> > >> the value that was written - a match with either would eliminate > >> > >> the need to do the write. > >> > >> > >> > >The hardware behavior is explicitly defined by SDM and mentioned in > >> > >'xl-psr.markdown' and 'intel_psr_mba.pandoc'. User should know that HW > >> > >can alter MBA value if the value is not valid. > >> > > >> > So if hardware behavior is fully defined, why don't you pre-adjust what > >> > is > >> > to be written to the value hardware would alter it to? > >> > > >> In previous version of MBA patch set, I pre-adjust the value in > > 'mba_check_thrtl'. > >> But Roger did not like that. So, the pre-adjust codes are removed. > > > > IMHO it's quite pointless to do such adjustments when the hardware > > performs them already. Also, I fear that our adjustments might get > > out-of-sync in the future with what hardware actually does. > > > > Maybe the result read back from the hardware (ie: adjusted) can be > > stored and used in order to check whether a new value should be > > written or not when switching? (I think this is the same that Jan > > suggested above). > > Not exactly, no - I'd like to avoid the write for _any_ value > resulting in the one currently stored in the hardware register. > Hence my earlier question on whether the transformation > done by hardware is well defined (i.e. _not_ model dependent > or fully defined by CPUID output). > SDM does not mention it is model dependent. So, it is _not_ model dependent. > Jan ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v4 04/15] x86: implement data structure and CPU init flow for MBA
On 17-10-05 02:49:59, Jan Beulich wrote:
> >>> On 05.10.17 at 06:42, wrote:
> > On 17-10-03 23:52:09, Jan Beulich wrote:
> >> >>> Yi Sun 09/29/17 3:55 AM >>>
> >> >On 17-09-28 05:00:09, Jan Beulich wrote:
> >> >> >>> On 23.09.17 at 11:48, wrote:
> >> >> > @@ -1410,6 +1496,7 @@ static void psr_cpu_init(void)
> >> >> > unsigned int socket, cpu = smp_processor_id();
> >> >> > struct feat_node *feat;
> >> >> > struct cpuid_leaf regs;
> >> >> > +uint32_t ebx;
> >> >>
> >> >> Is this local variable really a big help? To me it looks like it only
> >> >> makes the patch larger without actually improving anything,
> >> >> and without being related to the subject of the patch.
> >> >>
> >> >IMHO, it can avoid the 'cpuid_count_leaf()' being repeatedly called.
> >> >Without it,
> >> >we have to call 'cpuid_count_leaf()' for 2 more times.
> >>
> >> Hmm, didn't you simply replace regs.b uses with ebx? Or did I overlook a
> >> place
> >> where regs is being overwritten before the last of these regs.b uses (in
> >> which case
> >> I think your change is fine)?
> >>
> > The regs is overwritten when a feature presents. The old codes are below
> >
> > cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 0, ®s);
> > if ( regs.b & PSR_RESOURCE_TYPE_L3 )
> > {
> > cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 1, ®s); //It is
> > overwritten here.
> > ..
> > }
> >
> > cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 0, ®s); //So, we have to call
> > cpuid to get regs again.
> > if ( regs.b & PSR_RESOURCE_TYPE_L2 )
> > {
> > cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 2, ®s);
> > ..
> >
> > Because above reason, I defined this ebx local variable to avoid calling
> > cpuid
> > again for next feature.
>
> I see. But then please give the variable a better name, reflecting
> the data it holds.
>
Then, how about 'feat_mask'?
> Jan
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v4 07/15] x86: implement set value flow for MBA
On 17-10-03 23:59:46, Jan Beulich wrote:
> >>> Yi Sun 09/29/17 4:58 AM >>>
> >On 17-09-28 05:36:11, Jan Beulich wrote:
> >> >>> On 23.09.17 at 11:48, wrote:
> >> > This patch implements set value flow for MBA including its callback
> >> > function and domctl interface.
> >> >
> >> > It also changes the memebers in 'cos_write_info' to transfer the
> >> > feature array, feature properties array and value array. Then, we
> >> > can write all features values on the cos id into MSRs.
> >> >
> >> > Because multiple features may co-exist, we need handle all features to
> >> > write
> >> > values of them into a COS register with new COS ID. E.g:
> >> > 1. L3 CAT and MBA co-exist.
> >> > 2. Dom1 and Dom2 share a same COS ID (2). The L3 CAT CBM of Dom1 is
> >> > 0x1ff,
> >> >the MBA Thrtle of Dom1 is 0xa.
> >> > 3. User wants to change MBA Thrtl of Dom1 to be 0x14. Because COS ID 2 is
> >> >used by Dom2 too, we have to pick a new COS ID 3. The values of Dom1
> >> > on
> >> >COS ID 3 are all default values as below:
> >> >-
> >> >| COS 3 |
> >> >-
> >> >L3 CAT | 0x7ff |
> >> >-
> >> >MBA | 0x0 |
> >> >-
> >> > 4. After setting, the L3 CAT CBM value of Dom1 should be kept and the
> >> > new MBA
> >> >Thrtl is set. So, the values on COS ID 3 should be below.
> >> >-
> >> >| COS 3 |
> >> >-
> >> >L3 CAT | 0x1ff |
> >> >-
> >> >MBA | 0x14 |
> >> >-
> >> >
> >> > So, we should write all features values into their MSRs. That requires
> >> > the
> >> > feature array, feature properties array and value array are input.
> >>
> >> How is this last aspect (and the respective changes) related to MBA?
> >> I.e. why isn't this needed with the (also independent but possibly
> >> co-existing) L2/L3 CAT features?
> >>
> >I tried to introduce this in L2 CAT patch set but did not succeed. As there
> >is
> >no HW that L2 CAT and L3 CAT co-exist so far, I did not insist on this.
>
> Hmm, I'm afraid this wasn't then made clear enough to understand. I would
> certainly not have been against something that could in theory occur with
> L2/L3 CAT alone. In any event this means you don't want to mix this into this
> MBA specific change here.
>
Anyway, I think you suggest to split this as a new patch, right?
> >> > static void do_write_psr_msrs(void *data)
> >> > {
> >> > const struct cos_write_info *info = data;
> >> > -struct feat_node *feat = info->feature;
> >> > -const struct feat_props *props = info->props;
> >> > -unsigned int i, cos = info->cos, cos_num = props->cos_num;
> >> > +unsigned int i, index = 0, cos = info->cos;
> >> > +const uint32_t *val_array = info->val;
> >> >
> >> > -for ( i = 0; i < cos_num; i++ )
> >> > +for ( i = 0; i < ARRAY_SIZE(feat_props); i++ )
> >> > {
> >> > -if ( feat->cos_reg_val[cos * cos_num + i] != info->val[i] )
> >> > +struct feat_node *feat = info->features[i];
> >> > +const struct feat_props *props = info->props[i];
> >> > +unsigned int cos_num, j;
> >> > +
> >> > +if ( !feat || !props )
> >> > +continue;
> >> > +
> >> > +cos_num = props->cos_num;
> >> > +if ( info->array_len < index + cos_num )
> >> > +return;
> >> > +
> >> > +for ( j = 0; j < cos_num; j++ )
> >> > {
> >> > -feat->cos_reg_val[cos * cos_num + i] = info->val[i];
> >> > -props->write_msr(cos, info->val[i], props->type[i]);
> >> > +if ( feat->cos_reg_val[cos * cos_num + j] !=
> >> > val_array[index + j] )
> >> > +feat->cos_reg_val[cos * cos_num + j] =
> >> > +props->write_msr(cos, val_array[index + j],
> >> > props
Re: [Xen-devel] [PATCH v4 04/15] x86: implement data structure and CPU init flow for MBA
On 17-10-03 23:52:09, Jan Beulich wrote:
> >>> Yi Sun 09/29/17 3:55 AM >>>
> >On 17-09-28 05:00:09, Jan Beulich wrote:
> >> >>> On 23.09.17 at 11:48, wrote:
> >> > One thrtl register of MBA stores a throttle value for one or more
> >> > domains. The throttle value means the delay between L2 cache and next
> >> > cache level.
> >>
> >> What is a delay between two cache levels?
> >>
> >There is a "programmable rate controller" between them to indirectly control
> >the bandwidth.
>
> I'm afraid I don't feel like this answers my question. Anyway - the sentence
> needs some re-phrasing so it becomes clear what you're talking about.
>
I may draw the figure shown in SDM to make things clear.
> >> > @@ -1410,6 +1496,7 @@ static void psr_cpu_init(void)
> >> > unsigned int socket, cpu = smp_processor_id();
> >> > struct feat_node *feat;
> >> > struct cpuid_leaf regs;
> >> > +uint32_t ebx;
> >>
> >> Is this local variable really a big help? To me it looks like it only
> >> makes the patch larger without actually improving anything,
> >> and without being related to the subject of the patch.
> >>
> >IMHO, it can avoid the 'cpuid_count_leaf()' being repeatedly called. Without
> >it,
> >we have to call 'cpuid_count_leaf()' for 2 more times.
>
> Hmm, didn't you simply replace regs.b uses with ebx? Or did I overlook a place
> where regs is being overwritten before the last of these regs.b uses (in
> which case
> I think your change is fine)?
>
The regs is overwritten when a feature presents. The old codes are below
cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 0, ®s);
if ( regs.b & PSR_RESOURCE_TYPE_L3 )
{
cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 1, ®s); //It is overwritten
here.
..
}
cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 0, ®s); //So, we have to call
cpuid to get regs again.
if ( regs.b & PSR_RESOURCE_TYPE_L2 )
{
cpuid_count_leaf(PSR_CPUID_LEVEL_CAT, 2, ®s);
..
Because above reason, I defined this ebx local variable to avoid calling cpuid
again for next feature.
> Jan
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel
[Xen-devel] [PATCH v5 01/16] docs: create Memory Bandwidth Allocation (MBA) feature document
This patch creates MBA feature document in doc/features/. It describes key points to implement MBA which is described in details in Intel SDM "Introduction to Memory Bandwidth Allocation". Signed-off-by: Yi Sun --- CC: Jan Beulich CC: Andrew Cooper CC: Wei Liu CC: Ian Jackson CC: Daniel De Graaf CC: Roger Pau Monné CC: Konrad Rzeszutek Wilk CC: Chao Peng CC: Julien Grall v5: - correct some words. (suggested by Roger Pau Monné) - change 'xl psr-mba-set 1 0xa' to 'xl psr-mba-set 1 10'. (suggested by Roger Pau Monné) v4: - add 'domain-name' as parameter of 'psr-mba-show/psr-mba-set'. (suggested by Roger Pau Monné) - fix some wordings. (suggested by Roger Pau Monné) - explain how user can know the MBA_MAX. (suggested by Roger Pau Monné) - move the description of 'Linear mode/Non-linear mode' into section of 'psr-mba-show'. (suggested by Roger Pau Monné) - change 'per-thread' to 'per-hyper-thread' to make it clearer. (suggested by Roger Pau Monné) - upgrade revision number. v3: - remove 'closed-loop' related description. (suggested by Roger Pau Monné) - explain 'linear' and 'non-linear' before mentioning them. (suggested by Roger Pau Monné) - adjust desription of 'psr-mba-set'. (suggested by Roger Pau Monné) - explain 'MBA_MAX'. (suggested by Roger Pau Monné) - remove 'n<64'. (suggested by Roger Pau Monné) - fix some wordings. (suggested by Roger Pau Monné) - add context in 'Testing' part to make things more clear. (suggested by Roger Pau Monné) v2: - declare 'HW' in Terminology. (suggested by Chao Peng) - replace 'COS ID of VCPU' to 'COS ID of domain'. (suggested by Chao Peng) - replace 'COS register' to 'Thrtl MSR'. (suggested by Chao Peng) - add description for 'psr-mba-show' to state that the decimal value is shown for linear mode but hexadecimal value is shown for non-linear mode. (suggested by Chao Peng) - remove content in 'Areas for improvement'. (suggested by Chao Peng) - use '<>' to specify mandatory argument to a command. (suggested by Wei Liu) v1: - remove a special character to avoid the error when building pandoc. --- docs/features/intel_psr_mba.pandoc | 295 + 1 file changed, 295 insertions(+) create mode 100644 docs/features/intel_psr_mba.pandoc diff --git a/docs/features/intel_psr_mba.pandoc b/docs/features/intel_psr_mba.pandoc new file mode 100644 index 000..57bd4dc --- /dev/null +++ b/docs/features/intel_psr_mba.pandoc @@ -0,0 +1,295 @@ +% Intel Memory Bandwidth Allocation (MBA) Feature +% Revision 1.7 + +\clearpage + +# Basics + + + Status: **Tech Preview** + +Architecture(s): Intel x86 + + Component(s): Hypervisor, toolstack + + Hardware: MBA is supported on Skylake Server and beyond + + +# Terminology + +* CAT Cache Allocation Technology +* CBM Capacity BitMasks +* CDP Code and Data Prioritization +* COS/CLOSClass of Service +* HW Hardware +* MBA Memory Bandwidth Allocation +* MSRsMachine Specific Registers +* PSR Intel Platform Shared Resource +* THRTL Throttle value or delay value + +# Overview + +The Memory Bandwidth Allocation (MBA) feature provides indirect and approximate +control over memory bandwidth available per-core. This feature provides OS/ +hypervisor the ability to slow misbehaving apps/domains by using a credit-based +throttling mechanism. + +# User details + +* Feature Enabling: + + Add "psr=mba" to boot line parameter to enable MBA feature. + +* xl interfaces: + + 1. `psr-mba-show [domain-id|domain-name]`: + + Show memory bandwidth throttling for domain. Under different modes, it + shows different type of data. + + There are two modes: + Linear mode: the input precision is defined as 100-(MBA_MAX). For instance, + if the MBA_MAX value is 90, the input precision is 10%. Values not an even + multiple of the precision (e.g., 12%) will be rounded down (e.g., to 10% + delay applied) by HW automatically. The response of throttling value is + linear. + + Non-linear mode: input delay values are powers-of-two from zero to the + MBA_MAX value from CPUID. In this case any values not a power of two will + be rounded down the next nearest power of two by HW automatically. The + response of throttling value is non-linear. + + For linear mode, it shows the decimal value. For non-lin
[Xen-devel] [PATCH v5 11/16] tools: implement the new libxl get hw info interface
This patch implements the new libxl get hw info interface,
'libxl_psr_get_hw_info', which is suitable to all psr allocation
features. It also implements corresponding list free function,
'libxl_psr_hw_info_list_free' and makes 'libxl_psr_cat_get_info' call
'libxl_psr_get_hw_info' to avoid redundant code in libxl_psr.c.
Signed-off-by: Yi Sun
Reviewed-by: Roger Pau Monné
---
CC: Wei Liu
CC: Ian Jackson
CC: Chao Peng
v5:
- change 'if (rc < 0)' to 'if (rc)'.
(suggested by Roger Pau Monné)
v4:
- remove 'xc_' from struct name.
(suggested by Roger Pau Monné)
- fix words in commit message.
(suggested by Roger Pau Monné)
- change type of 'libxl__hw_info_to_libxl_cat_info' to void and use
assert to check invalid type. Then, remove check for
'libxl__hw_info_to_libxl_cat_info'.
(suggested by Roger Pau Monné)
- change type of 'libxl__xc_hw_info_to_libxl_hw_info' to void and use
assert to check invalid type. Then, remove check for
'libxl__xc_hw_info_to_libxl_hw_info'.
(suggested by Roger Pau Monné)
v3:
- remove casting.
(suggested by Roger Pau Monné)
- remove inline.
(suggested by Roger Pau Monné)
- change 'libxc__psr_hw_info_to_libxl_psr_hw_info' to
'libxl__xc_hw_info_to_libxl_hw_info'.
(suggested by Roger Pau Monné)
- remove '_hw' from parameter names.
(suggested by Roger Pau Monné)
- change some 'LOGE' to 'LOG'.
(suggested by Roger Pau Monné)
- check returned 'xc_type' and remove redundant 'lvl' check.
(suggested by Roger Pau Monné)
v2:
- split this patch out from a big patch in v1.
(suggested by Wei Liu)
- change 'CAT_INFO'/'MBA_INFO' to 'CAT' and 'MBA. Also the libxl structure
name 'cat_info'/'mba_info' is changed to 'cat'/'mba'.
(suggested by Chao Peng)
- call 'libxl_psr_hw_info_list_free' in 'libxl_psr_cat_get_info' to free
allocated resources.
(suggested by Chao Peng)
---
tools/libxl/libxl_psr.c | 131 ++--
1 file changed, 93 insertions(+), 38 deletions(-)
diff --git a/tools/libxl/libxl_psr.c b/tools/libxl/libxl_psr.c
index e8d62e1..cd82ac0 100644
--- a/tools/libxl/libxl_psr.c
+++ b/tools/libxl/libxl_psr.c
@@ -384,56 +384,41 @@ static xc_psr_feat_type
libxl__feat_type_to_libxc_feat_type(
return xc_type;
}
+static void libxl__hw_info_to_libxl_cat_info(
+libxl_psr_feat_type type, libxl_psr_hw_info *hw_info,
+libxl_psr_cat_info *cat_info)
+{
+assert(type == LIBXL_PSR_FEAT_TYPE_CAT);
+
+cat_info->id = hw_info->id;
+cat_info->cos_max = hw_info->u.cat.cos_max;
+cat_info->cbm_len = hw_info->u.cat.cbm_len;
+cat_info->cdp_enabled = hw_info->u.cat.cdp_enabled;
+}
+
int libxl_psr_cat_get_info(libxl_ctx *ctx, libxl_psr_cat_info **info,
unsigned int *nr, unsigned int lvl)
{
GC_INIT(ctx);
int rc;
-int i = 0, socketid, nr_sockets;
-libxl_bitmap socketmap;
+unsigned int i;
+libxl_psr_hw_info *hw_info;
libxl_psr_cat_info *ptr;
-xc_psr_hw_info hw_info;
-xc_psr_feat_type xc_type;
-
-libxl_bitmap_init(&socketmap);
-
-rc = libxl__count_physical_sockets(gc, &nr_sockets);
-if (rc) {
-LOGE(ERROR, "failed to get system socket count");
-goto out;
-}
-libxl_socket_bitmap_alloc(ctx, &socketmap, nr_sockets);
-rc = libxl_get_online_socketmap(ctx, &socketmap);
-if (rc < 0) {
-LOGE(ERROR, "failed to get available sockets");
+rc = libxl_psr_get_hw_info(ctx, LIBXL_PSR_FEAT_TYPE_CAT, lvl, nr,
&hw_info);
+if (rc)
goto out;
-}
-
-xc_type = libxl__feat_type_to_libxc_feat_type(LIBXL_PSR_FEAT_TYPE_CAT,
lvl);
-
-ptr = libxl__malloc(NOGC, nr_sockets * sizeof(libxl_psr_cat_info));
-
-libxl_for_each_set_bit(socketid, socketmap) {
-ptr[i].id = socketid;
-if (xc_psr_get_hw_info(ctx->xch, socketid, xc_type, &hw_info)) {
-LOGE(ERROR, "failed to get hw info");
-rc = ERROR_FAIL;
-free(ptr);
-goto out;
-}
-ptr[i].cos_max = hw_info.cat.cos_max;
-ptr[i].cbm_len = hw_info.cat.cbm_len;
-ptr[i].cdp_enabled = hw_info.cat.cdp_enabled;
+ptr = libxl__malloc(NOGC, *nr * sizeof(libxl_psr_cat_info));
-i++;
-}
+for (i = 0; i < *nr; i++)
+libxl__hw_info_to_libxl_cat_info(LIBXL_PSR_FEAT_TYPE_CAT,
+ &hw_info[i],
+ &ptr[i]);
[Xen-devel] [PATCH v5 08/16] x86: implement set value flow for MBA
This patch implements set value flow for MBA including its callback
function and domctl interface.
It also changes the memebers in 'cos_write_info' to transfer the
feature array, feature properties array and value array. Then, we
can write all features values on the cos id into MSRs.
Because multiple features may co-exist, we need handle all features to write
values of them into a COS register with new COS ID. E.g:
1. L3 CAT and MBA co-exist.
2. Dom1 and Dom2 share a same COS ID (2). The L3 CAT CBM of Dom1 is 0x1ff,
the MBA Thrtle of Dom1 is 0xa.
3. User wants to change MBA Thrtl of Dom1 to be 0x14. Because COS ID 2 is
used by Dom2 too, we have to pick a new COS ID 3. The values of Dom1 on
COS ID 3 are all default values as below:
-
| COS 3 |
-
L3 CAT | 0x7ff |
-
MBA | 0x0 |
-
4. After setting, the L3 CAT CBM value of Dom1 should be kept and the new MBA
Thrtl is set. So, the values on COS ID 3 should be below.
-
| COS 3 |
-
L3 CAT | 0x1ff |
-
MBA | 0x14 |
-
So, we should write all features values into their MSRs. That requires the
feature array, feature properties array and value array are input.
Signed-off-by: Yi Sun
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v5:
- adjust position of 'cat_check_cbm' to not to make changes so big.
(suggested by Roger Pau Monné)
- remove 'props' from 'struct cos_write_info'.
(suggested by Roger Pau Monné)
- make a single return statement in 'mba_check_thrtl'.
(suggested by Jan Beulich)
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- join two checks into a single if.
(suggested by Roger Pau Monné)
- remove redundant local variable 'array_len'.
(suggested by Roger Pau Monné)
v3:
- modify commit message to make it clear.
(suggested by Roger Pau Monné)
- modify functionality of 'check_val' to make it simple to only check value.
Change the last parameter type from 'unsigned long *' to 'unsigned long'.
(suggested by Roger Pau Monné)
- call rdmsrl to get value just written into MSR for MBA. Because HW can
automatically change input value to what it wants.
(suggested by Roger Pau Monné)
- change type of 'write_msr' to 'uint32_t' to return the value actually
written into MSR. Then, change 'do_write_psr_msrs' to set the returned
value into 'cos_reg_val[]'
- move the declaration of 'j' into loop in 'do_write_psr_msrs'.
(suggested by Roger Pau Monné)
- change 'mba_info' to 'mba'.
(suggested by Roger Pau Monné)
- change 'cat_info' to 'cat'.
(suggested by Roger Pau Monné)
- rename 'psr_cat/PSR_CAT' to 'psr_alloc/PSR_ALLOC' and remove 'op/OP'
from name.
(suggested by Roger Pau Monné)
- change 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
v2:
- remove linear mode 'thrtl_max' check in 'mba_check_thrtl' because it has
been checked in 'mba_init_feature'.
(suggested by Chao Peng)
- for non-linear mode, check if '*thrtl' is not 0 in 'mba_check_thrtl'. If
it is 0, we do not need to change it.
(suggested by Chao Peng)
- move comments to explain changes of 'cos_write_info' from psr.c to commit
message.
(suggested by Chao Peng)
---
xen/arch/x86/domctl.c | 6
xen/arch/x86/psr.c | 86 +++--
xen/include/public/domctl.h | 1 +
3 files changed, 59 insertions(+), 34 deletions(-)
diff --git a/xen/arch/x86/domctl.c b/xen/arch/x86/domctl.c
index 5fb443f..e5f6a24 100644
--- a/xen/arch/x86/domctl.c
+++ b/xen/arch/x86/domctl.c
@@ -1476,6 +1476,12 @@ long arch_do_domctl(
PSR_TYPE_L2_CBM);
break;
+case XEN_DOMCTL_PSR_SET_MBA_THRTL:
+ret = psr_set_val(d, domctl->u.psr_alloc.target,
+ domctl->u.psr_alloc.data,
+ PSR_TYPE_MBA_THRTL);
+break;
+
case XEN_DOMCTL_PSR_GET_L3_CBM:
ret = domctl_psr_get_val(d, domctl, PSR_TYPE_L3_CBM, copyback);
break;
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 66e20a7..32e5122 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -138,6 +138,9 @@ static const struct feat_props {
/* write_msr is used to write out feature MSR register. */
uint32_t (*write_msr)(unsigned int co
[Xen-devel] [PATCH v5 06/16] x86: implement get hw info flow for MBA
This patch implements get HW info flow for MBA including its callback
function and sysctl interface.
Signed-off-by: Yi Sun
---
CC: Jan Beulich
CC: Andrew Cooper
CC: Wei Liu
CC: Roger Pau Monné
CC: Chao Peng
v5:
- use ASSERT in 'mba_get_feat_info'.
(suggested by Roger Pau Monné)
- correct initialization format of 'data[PSR_INFO_ARRAY_SIZE]'.
(suggested by Roger Pau Monné and Jan Beulich)
v4:
- remove 'ALLOC_' from macro names.
(suggested by Roger Pau Monné)
- initialize 'data[PSR_INFO_ARRAY_SIZE]' to 0 to prevent to leak stack data.
(suggested by Roger Pau Monné)
v3:
- replace 'PSR_VAL_TYPE_MBA' to 'PSR_TYPE_MBA_THRTL'.
(suggested by Roger Pau Monné)
v2:
- use 'XEN_SYSCTL_PSR_MBA_LINEAR' to set MBA feature HW info.
(suggested by Chao Peng)
v1:
- sort 'PSR_INFO_IDX_' macros as feature.
(suggested by Chao Peng)
- rename 'PSR_INFO_IDX_MBA_LINEAR' to 'PSR_INFO_IDX_MBA_FLAG'.
- rename 'linear' in 'struct mba_info' to 'flags' for future extension.
(suggested by Chao Peng)
---
xen/arch/x86/psr.c | 14 +-
xen/arch/x86/sysctl.c | 21 -
xen/include/asm-x86/psr.h | 2 ++
xen/include/public/sysctl.h | 8
4 files changed, 43 insertions(+), 2 deletions(-)
diff --git a/xen/arch/x86/psr.c b/xen/arch/x86/psr.c
index 8520709..66e20a7 100644
--- a/xen/arch/x86/psr.c
+++ b/xen/arch/x86/psr.c
@@ -263,6 +263,10 @@ static enum psr_feat_type psr_type_to_feat_type(enum
psr_type type)
feat_type = FEAT_TYPE_L2_CAT;
break;
+case PSR_TYPE_MBA_THRTL:
+feat_type = FEAT_TYPE_MBA;
+break;
+
default:
ASSERT_UNREACHABLE();
}
@@ -489,7 +493,15 @@ static const struct feat_props l2_cat_props = {
static bool mba_get_feat_info(const struct feat_node *feat,
uint32_t data[], unsigned int array_len)
{
-return false;
+ASSERT( array_len == PSR_INFO_ARRAY_SIZE );
+
+data[PSR_INFO_IDX_COS_MAX] = feat->cos_max;
+data[PSR_INFO_IDX_MBA_THRTL_MAX] = feat->mba.thrtl_max;
+
+if ( feat->mba.linear )
+data[PSR_INFO_IDX_MBA_FLAG] |= XEN_SYSCTL_PSR_MBA_LINEAR;
+
+return true;
}
static uint32_t mba_write_msr(unsigned int cos, uint32_t val,
diff --git a/xen/arch/x86/sysctl.c b/xen/arch/x86/sysctl.c
index 9dee163..11e8482 100644
--- a/xen/arch/x86/sysctl.c
+++ b/xen/arch/x86/sysctl.c
@@ -174,7 +174,7 @@ long arch_do_sysctl(
case XEN_SYSCTL_psr_alloc:
switch ( sysctl->u.psr_alloc.cmd )
{
-uint32_t data[PSR_INFO_ARRAY_SIZE];
+uint32_t data[PSR_INFO_ARRAY_SIZE] = { };
case XEN_SYSCTL_PSR_get_l3_info:
{
@@ -214,6 +214,25 @@ long arch_do_sysctl(
break;
}
+case XEN_SYSCTL_PSR_get_mba_info:
+{
+ret = psr_get_info(sysctl->u.psr_alloc.target,
+ PSR_TYPE_MBA_THRTL, data, ARRAY_SIZE(data));
+if ( ret )
+break;
+
+sysctl->u.psr_alloc.u.mba_info.cos_max =
+ data[PSR_INFO_IDX_COS_MAX];
+sysctl->u.psr_alloc.u.mba_info.thrtl_max =
+ data[PSR_INFO_IDX_MBA_THRTL_MAX];
+sysctl->u.psr_alloc.u.mba_info.flags =
+ data[PSR_INFO_IDX_MBA_FLAG];
+
+if ( __copy_field_to_guest(u_sysctl, sysctl, u.psr_alloc) )
+ret = -EFAULT;
+break;
+}
+
default:
ret = -EOPNOTSUPP;
break;
diff --git a/xen/include/asm-x86/psr.h b/xen/include/asm-x86/psr.h
index 9d14264..084ae97 100644
--- a/xen/include/asm-x86/psr.h
+++ b/xen/include/asm-x86/psr.h
@@ -39,6 +39,8 @@
#define PSR_INFO_IDX_COS_MAX0
#define PSR_INFO_IDX_CAT_CBM_LEN1
#define PSR_INFO_IDX_CAT_FLAG 2
+#define PSR_INFO_IDX_MBA_THRTL_MAX 1
+#define PSR_INFO_IDX_MBA_FLAG 2
#define PSR_INFO_ARRAY_SIZE 3
struct psr_cmt_l3 {
diff --git a/xen/include/public/sysctl.h b/xen/include/public/sysctl.h
index 24191f0..b6930b8 100644
--- a/xen/include/public/sysctl.h
+++ b/xen/include/public/sysctl.h
@@ -745,6 +745,7 @@ DEFINE_XEN_GUEST_HANDLE(xen_sysctl_pcitopoinfo_t);
#define XEN_SYSCTL_PSR_get_l3_info 0
#define XEN_SYSCTL_PSR_get_l2_info 1
+#define XEN_SYSCTL_PSR_get_mba_info 2
struct xen_sysctl_psr_alloc {
uint32_t cmd; /* IN: XEN_SYSCTL_PSR_* */
uint32_t target;/* IN */
@@ -755,6 +756,13 @@ struct xen_sysctl_psr_alloc {
#define XEN_SYSCTL_PSR_CAT_L3_CDP (1u << 0)
uint32_t flags; /* OUT: CAT flags */
} cat_info;
+
+struct {
+uint32_t thrtl_max; /
