Re: [PATCH] mem/cxl_type3: Fix overlapping region validation error

[Fan Ni]

On Thu, Jul 18, 2024 at 05:07:53AM -0400, Yao Xingtao wrote:
>  style="display:none 
> !important;display:none;visibility:hidden;mso-hide:all;font-size:1px;color:#ff;line-height:1px;height:0px;max-height:0px;opacity:0;overflow:hidden;">
>  When injecting a new poisoned region through qmp_cxl_inject_poison(), the 
> newly injected region should not overlap with existing poisoned regions. The 
> current validation method does not consider the following overlapping region: 
> ┌───┬───┬───┐
> 
> 
> 
> 
> ZjQcmQRYFpfptBannerStart
> 
> 
> 
> 
>   
> 
>   
>   This Message Is From an External Sender
>   
>   
> Use caution opening files, clicking links or responding to requests.
>   
> 
> 
> 
> 
>   
> 
> 
> ZjQcmQRYFpfptBannerEnd
> 
> 
> 
> 
> 
> 
>   #pfptBannergkdcmiu { all: revert !important; display: block !important; 
> visibility: visible !important; opacity: 1 !important; 
> background-color: #F07B7B !important; 
> max-width: none !important; max-height: none !important }
>   .pfptPrimaryButtongkdcmiu:hover, .pfptPrimaryButtongkdcmiu:focus {
> background-color: #ef5b5b !important; }
>   .pfptPrimaryButtongkdcmiu:active {
> background-color: #f32323 !important; }
> 
> 
> 
> When injecting a new poisoned 
> region through qmp_cxl_inject_poison(),
> the newly injected region should not overlap with existing poisoned
> regions.
> 
> The current validation method does not consider the following
> overlapping region:
> ┌───┬───┬───┐
> │a  │  b(a) │a  │
> └───┴───┴───┘
> (a is a newly added region, b is an existing region, and b is a
>  subregion of a)
> 
> Signed-off-by: Yao Xingtao yaoxt.f...@fujitsu.com
> ---
>  hw/mem/cxl_type3.c | 4 ---
>  1 file changed, 1 insertion(), 3 deletions(-)
> 
> diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
> index 35ac59883a5b..8e32de327908 100644
> --- a/hw/mem/cxl_type3.c
>  b/hw/mem/cxl_type3.c
> @@ -1331,9 1331,7 @@ void qmp_cxl_inject_poison(const char *path, 
> uint64_t start, uint64_t length,
>  ct3d = CXL_TYPE3(obj);
>  
>  QLIST_FOREACH(p, ct3d-poison_list, node) {
> -if (((start = p-start)  (start  p-start 
>  p-length)) ||
> -((start  length  p-start) 
> - (start  length = p-start  p-length))) {
> if ((start  p-start  p-length)  
> (start  length  p-start)) {
>  error_setg(errp,
> Overlap with existing poisoned region not 
> supported);
>  return;

As mentioned by Peter, we can use ranges_overlap() to improve the
code readability. Other than that, looks good t me.

btw, not sure only me or not, but the message does not display
correctly in mutt, seems not a plain text message, but looks fine in
outlook.

Fan
> -- 
> 2.37.3
> 
> 

Re: [PATCH] hw/cxl: Fix local variable shadowing of cap_hdrs

[Fan Ni]

On Mon, Sep 25, 2023 at 04:22:58PM +0100, Jonathan Cameron wrote:

> Rename the version not burried in the macro to cap_h.
> 
> Signed-off-by: Jonathan Cameron 
> ---
> 
> I had another instance of this in a series I'll post later today.
> Cleaned that up the same way.
> 
>  hw/cxl/cxl-device-utils.c | 8 
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/hw/cxl/cxl-device-utils.c b/hw/cxl/cxl-device-utils.c
> index 517f06d869..bd68328032 100644
> --- a/hw/cxl/cxl-device-utils.c
> +++ b/hw/cxl/cxl-device-utils.c
> @@ -283,13 +283,13 @@ static void memdev_reg_init_common(CXLDeviceState 
> *cxl_dstate) { }
>  
>  void cxl_device_register_init_common(CXLDeviceState *cxl_dstate)
>  {
> -uint64_t *cap_hdrs = cxl_dstate->caps_reg_state64;
> +uint64_t *cap_h = cxl_dstate->caps_reg_state64;
>  const int cap_count = 3;
>  
>  /* CXL Device Capabilities Array Register */
> -ARRAY_FIELD_DP64(cap_hdrs, CXL_DEV_CAP_ARRAY, CAP_ID, 0);
> -ARRAY_FIELD_DP64(cap_hdrs, CXL_DEV_CAP_ARRAY, CAP_VERSION, 1);
> -ARRAY_FIELD_DP64(cap_hdrs, CXL_DEV_CAP_ARRAY, CAP_COUNT, cap_count);
> +ARRAY_FIELD_DP64(cap_h, CXL_DEV_CAP_ARRAY, CAP_ID, 0);
> +ARRAY_FIELD_DP64(cap_h, CXL_DEV_CAP_ARRAY, CAP_VERSION, 1);
> +ARRAY_FIELD_DP64(cap_h, CXL_DEV_CAP_ARRAY, CAP_COUNT, cap_count);
>  
>  cxl_device_cap_init(cxl_dstate, DEVICE_STATUS, 1, 2);
>  device_reg_init_common(cxl_dstate);
> -- 
> 2.39.2
> 

Reviewed-by: Fan Ni 

Re: [PATCH] hw/cxl: Fix local variable shadowing of cap_hdrs

[Fan Ni]

On Fri, Sep 29, 2023 at 09:50:16AM +0200, Markus Armbruster wrote:

> Jonathan Cameron  writes:
> 
> > On Wed, 27 Sep 2023 19:13:35 +0000
> > Fan Ni  wrote:
> >
> >> On Mon, Sep 25, 2023 at 04:22:58PM +0100, Jonathan Cameron wrote:
> >> 
> >> > Rename the version not burried in the macro to cap_h.  
> >> The change looks good to me. Just one minor thing. why "version" get
> >> involved here?
> >>
> >
> > Used in the sense of two copies of something with slightly differences
> > given if it were straight code without a macro, we'd have just
> > have used the copy being changed here for all of the calls. 
> > With hindsight, not the best word to choose given the many other meanings!
> 
> Fan, good enough to get your R-by?
> 

Yes.

Fan

Re: [PATCH 05/19] hw/pci-bridge/cxl_upstream: Move defintion of device to header.

[Fan Ni]

On Mon, Sep 25, 2023 at 05:11:10PM +0100, Jonathan Cameron wrote:
> To avoid repetition of switch upstream port specific data in the
> CXLDeviceState structure it will be necessary to access the switch USP
> specific data from mailbox callbacks. Hence move it to cxl_device.h so it
> is no longer an opaque structure.
>
> Signed-off-by: Jonathan Cameron 
> ---

Reviewed-by: Fan Ni 

>  include/hw/pci-bridge/cxl_upstream_port.h | 18 ++
>  hw/pci-bridge/cxl_upstream.c  | 11 +--
>  2 files changed, 19 insertions(+), 10 deletions(-)
>
> diff --git a/include/hw/pci-bridge/cxl_upstream_port.h 
> b/include/hw/pci-bridge/cxl_upstream_port.h
> new file mode 100644
> index 00..b02aa8f659
> --- /dev/null
> +++ b/include/hw/pci-bridge/cxl_upstream_port.h
> @@ -0,0 +1,18 @@
> +
> +#ifndef CXL_USP_H
> +#define CXL_USP_H
> +#include "hw/pci/pcie.h"
> +#include "hw/pci/pcie_port.h"
> +#include "hw/cxl/cxl.h"
> +
> +typedef struct CXLUpstreamPort {
> +/*< private >*/
> +PCIEPort parent_obj;
> +
> +/*< public >*/
> +CXLComponentState cxl_cstate;
> +DOECap doe_cdat;
> +uint64_t sn;
> +} CXLUpstreamPort;
> +
> +#endif /* CXL_SUP_H */
> diff --git a/hw/pci-bridge/cxl_upstream.c b/hw/pci-bridge/cxl_upstream.c
> index b81bb5fec9..36737189c6 100644
> --- a/hw/pci-bridge/cxl_upstream.c
> +++ b/hw/pci-bridge/cxl_upstream.c
> @@ -14,6 +14,7 @@
>  #include "hw/pci/msi.h"
>  #include "hw/pci/pcie.h"
>  #include "hw/pci/pcie_port.h"
> +#include "hw/pci-bridge/cxl_upstream_port.h"
>  /*
>   * Null value of all Fs suggested by IEEE RA guidelines for use of
>   * EU, OUI and CID
> @@ -30,16 +31,6 @@
>  #define CXL_UPSTREAM_PORT_DVSEC_OFFSET \
>  (CXL_UPSTREAM_PORT_SN_OFFSET + PCI_EXT_CAP_DSN_SIZEOF)
>
> -typedef struct CXLUpstreamPort {
> -/*< private >*/
> -PCIEPort parent_obj;
> -
> -/*< public >*/
> -CXLComponentState cxl_cstate;
> -DOECap doe_cdat;
> -uint64_t sn;
> -} CXLUpstreamPort;
> -
>  CXLComponentState *cxl_usp_to_cstate(CXLUpstreamPort *usp)
>  {
>  return >cxl_cstate;
> --
> 2.39.2
>

Re: [PATCH 04/19] hw/cxl/mbox: Generalize the CCI command processing

[Fan Ni]

On Mon, Sep 25, 2023 at 05:11:09PM +0100, Jonathan Cameron wrote:
> By moving the parts of the mailbox command handling that are CCI type
> specific out to the caller, make the main handling code generic. Rename it
> to cxl_process_cci_message() to reflect this new generality.
>
> Signed-off-by: Jonathan Cameron 
> ---

Reviewed-by: Fan Ni 

>  include/hw/cxl/cxl_device.h |  5 +++-
>  hw/cxl/cxl-device-utils.c   | 51 -
>  hw/cxl/cxl-mailbox-utils.c  | 43 ---
>  3 files changed, 64 insertions(+), 35 deletions(-)
>
> diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
> index c883d9dd8f..0e3f6c3c0b 100644
> --- a/include/hw/cxl/cxl_device.h
> +++ b/include/hw/cxl/cxl_device.h
> @@ -270,7 +270,10 @@ CXL_DEVICE_CAPABILITY_HEADER_REGISTER(MEMORY_DEVICE,
>
>  void cxl_initialize_mailbox_t3(CXLCCI *cci, DeviceState *d, size_t 
> payload_max);
>  void cxl_init_cci(CXLCCI *cci, size_t payload_max);
> -void cxl_process_mailbox(CXLCCI *cci);
> +int cxl_process_cci_message(CXLCCI *cci, uint8_t set, uint8_t cmd,
> +size_t len_in, uint8_t *pl_in,
> +size_t *len_out, uint8_t *pl_out,
> +bool *bg_started);
>
>  #define cxl_device_cap_init(dstate, reg, cap_id, ver)  \
>  do {   \
> diff --git a/hw/cxl/cxl-device-utils.c b/hw/cxl/cxl-device-utils.c
> index 327949a805..f3a6e17154 100644
> --- a/hw/cxl/cxl-device-utils.c
> +++ b/hw/cxl/cxl-device-utils.c
> @@ -79,6 +79,25 @@ static uint64_t mailbox_reg_read(void *opaque, hwaddr 
> offset, unsigned size)
>  case 4:
>  return cxl_dstate->mbox_reg_state32[offset / size];
>  case 8:
> +if (offset == A_CXL_DEV_BG_CMD_STS) {
> +uint64_t bg_status_reg;
> +bg_status_reg = FIELD_DP64(0, CXL_DEV_BG_CMD_STS, OP,
> +   cci->bg.opcode);
> +bg_status_reg = FIELD_DP64(bg_status_reg, CXL_DEV_BG_CMD_STS,
> +   PERCENTAGE_COMP, 
> cci->bg.complete_pct);
> +bg_status_reg = FIELD_DP64(bg_status_reg, CXL_DEV_BG_CMD_STS,
> +   RET_CODE, cci->bg.ret_code);
> +/* endian? */
> +cxl_dstate->mbox_reg_state64[offset / size] = bg_status_reg;
> +}
> +if (offset == A_CXL_DEV_MAILBOX_STS) {
> +uint64_t status_reg = cxl_dstate->mbox_reg_state64[offset / 
> size];
> +if (cci->bg.complete_pct) {
> +status_reg = FIELD_DP64(status_reg, CXL_DEV_MAILBOX_STS, 
> BG_OP,
> +0);
> +cxl_dstate->mbox_reg_state64[offset / size] = status_reg;
> +}
> +}
>  return cxl_dstate->mbox_reg_state64[offset / size];
>  default:
>  g_assert_not_reached();
> @@ -157,7 +176,37 @@ static void mailbox_reg_write(void *opaque, hwaddr 
> offset, uint64_t value,
>
>  if (ARRAY_FIELD_EX32(cxl_dstate->mbox_reg_state32, CXL_DEV_MAILBOX_CTRL,
>   DOORBELL)) {
> -cxl_process_mailbox(cci);
> +uint64_t command_reg =
> +cxl_dstate->mbox_reg_state64[R_CXL_DEV_MAILBOX_CMD];
> +uint8_t cmd_set = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD,
> + COMMAND_SET);
> +uint8_t cmd = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, COMMAND);
> +size_t len_in = FIELD_EX64(command_reg, CXL_DEV_MAILBOX_CMD, LENGTH);
> +uint8_t *pl = cxl_dstate->mbox_reg_state + A_CXL_DEV_CMD_PAYLOAD;
> +size_t len_out;
> +uint64_t status_reg;
> +bool bg_started;
> +int rc;
> +
> +rc = cxl_process_cci_message(cci, cmd_set, cmd, len_in, pl,
> + _out, pl, _started);
> +
> +/* Set bg and the return code */
> +status_reg = FIELD_DP64(0, CXL_DEV_MAILBOX_STS, BG_OP,
> +bg_started ? 1 : 0);
> +status_reg = FIELD_DP64(status_reg, CXL_DEV_MAILBOX_STS, ERRNO, rc);
> +/* Set the return length */
> +command_reg = FIELD_DP64(0, CXL_DEV_MAILBOX_CMD, COMMAND_SET, 
> cmd_set);
> +command_reg = FIELD_DP64(command_reg, CXL_DEV_MAILBOX_CMD,
> + COMMAND, cmd);
> +command_reg = FIELD_DP64(command_reg, CXL_DEV_MAILBOX_CMD,
> + LENGTH, len_out);
> +
> +cxl_dstate->mbox_reg_state64[R_CXL_DEV_MAILBOX_CMD] = command_reg;
> +  

Re: [PATCH 03/19] hw/cxl/mbox: Pull the CCI definition out of the CXLDeviceState

[Fan Ni]

On Mon, Sep 25, 2023 at 05:11:08PM +0100, Jonathan Cameron wrote:
> Enables having multiple CCIs per devices. Each CCI (mailbox) has it's own
> state and command list, so they can't share a single structure.
>
> Signed-off-by: Jonathan Cameron 
> ---

Reviewed-by: Fan Ni 

>  include/hw/cxl/cxl_device.h | 45 ++
>  hw/cxl/cxl-device-utils.c   | 31 +---
>  hw/cxl/cxl-mailbox-utils.c  | 94 ++---
>  hw/mem/cxl_type3.c  |  5 +-
>  4 files changed, 119 insertions(+), 56 deletions(-)
>
> diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
> index d7a2c4009e..c883d9dd8f 100644
> --- a/include/hw/cxl/cxl_device.h
> +++ b/include/hw/cxl/cxl_device.h
> @@ -111,12 +111,13 @@ typedef enum {
>  CXL_MBOX_MAX = 0x17
>  } CXLRetCode;
>
> +typedef struct CXLCCI CXLCCI;
>  typedef struct cxl_device_state CXLDeviceState;
>  struct cxl_cmd;
>  typedef CXLRetCode (*opcode_handler)(const struct cxl_cmd *cmd,
>   uint8_t *payload_in, size_t len_in,
>   uint8_t *payload_out, size_t *len_out,
> - CXLDeviceState *cxl_dstate);
> + CXLCCI *cci);
>  struct cxl_cmd {
>  const char *name;
>  opcode_handler handler;
> @@ -140,6 +141,31 @@ typedef struct CXLEventLog {
>  QSIMPLEQ_HEAD(, CXLEvent) events;
>  } CXLEventLog;
>
> +typedef struct CXLCCI {
> +const struct cxl_cmd (*cxl_cmd_set)[256];
> +struct cel_log {
> +uint16_t opcode;
> +uint16_t effect;
> +} cel_log[1 << 16];
> +size_t cel_size;
> +
> +/* background command handling (times in ms) */
> +struct {
> +uint16_t opcode;
> +uint16_t complete_pct;
> +uint16_t ret_code; /* Current value of retcode */
> +uint64_t starttime;
> +/* set by each bg cmd, cleared by the bg_timer when complete */
> +uint64_t runtime;
> +QEMUTimer *timer;
> +} bg;
> +size_t payload_max;
> +/* Pointer to device hosting the CCI */
> +DeviceState *d;
> +/* Pointer to the device hosting the protocol conversion */
> +DeviceState *intf;
> +} CXLCCI;
> +
>  typedef struct cxl_device_state {
>  MemoryRegion device_registers;
>
> @@ -173,11 +199,6 @@ typedef struct cxl_device_state {
>  uint32_t mbox_reg_state32[CXL_MAILBOX_REGISTERS_LENGTH / 4];
>  uint64_t mbox_reg_state64[CXL_MAILBOX_REGISTERS_LENGTH / 8];
>  };
> -struct cel_log {
> -uint16_t opcode;
> -uint16_t effect;
> -} cel_log[1 << 16];
> -size_t cel_size;
>  };
>
>  struct {
> @@ -196,10 +217,12 @@ typedef struct cxl_device_state {
>  } CXLDeviceState;
>
>  /* Initialize the register block for a device */
> -void cxl_device_register_block_init(Object *obj, CXLDeviceState *dev);
> +void cxl_device_register_block_init(Object *obj, CXLDeviceState *dev,
> +CXLCCI *cci);
>
> +typedef struct CXLType3Dev CXLType3Dev;
>  /* Set up default values for the register block */
> -void cxl_device_register_init_common(CXLDeviceState *dev);
> +void cxl_device_register_init_t3(CXLType3Dev *ct3d);
>
>  /*
>   * CXL 2.0 - 8.2.8.1 including errata F4
> @@ -245,8 +268,9 @@ CXL_DEVICE_CAPABILITY_HEADER_REGISTER(MEMORY_DEVICE,
>CXL_DEVICE_CAP_HDR1_OFFSET +
>CXL_DEVICE_CAP_REG_SIZE * 2)
>
> -void cxl_initialize_mailbox(CXLDeviceState *cxl_dstate);
> -void cxl_process_mailbox(CXLDeviceState *cxl_dstate);
> +void cxl_initialize_mailbox_t3(CXLCCI *cci, DeviceState *d, size_t 
> payload_max);
> +void cxl_init_cci(CXLCCI *cci, size_t payload_max);
> +void cxl_process_mailbox(CXLCCI *cci);
>
>  #define cxl_device_cap_init(dstate, reg, cap_id, ver)  \
>  do {   \
> @@ -347,6 +371,7 @@ struct CXLType3Dev {
>  AddressSpace hostpmem_as;
>  CXLComponentState cxl_cstate;
>  CXLDeviceState cxl_dstate;
> +CXLCCI cci; /* Primary PCI mailbox CCI */
>
>  /* DOE */
>  DOECap doe_cdat;
> diff --git a/hw/cxl/cxl-device-utils.c b/hw/cxl/cxl-device-utils.c
> index eb7195272e..327949a805 100644
> --- a/hw/cxl/cxl-device-utils.c
> +++ b/hw/cxl/cxl-device-utils.c
> @@ -62,7 +62,14 @@ static uint64_t dev_reg_read(void *opaque, hwaddr offset, 
> unsigned size)
>
>  static uint64_t mailbox_reg_read(void *opaque, hwaddr offset, unsigned size)
>  {
> -CXLDe

Re: [PATCH 02/19] hw/cxl/mbox: Split mailbox command payload into separate input and output

[Fan Ni]

On Mon, Sep 25, 2023 at 05:11:07PM +0100, Jonathan Cameron wrote:

> New CCI types that will be supported shortly do not have a single buffer
> used in both directions. As such, split it up. For CXL mailboxes the two
> pointers will be aliases of the same memory so all callbacks must allow
> for that.
> 
> Signed-off-by: Jonathan Cameron 
> ---

Reviewed-by: Fan Ni 

>  include/hw/cxl/cxl_device.h |   7 +-
>  hw/cxl/cxl-events.c |   2 +-
>  hw/cxl/cxl-mailbox-utils.c  | 222 +---
>  3 files changed, 132 insertions(+), 99 deletions(-)
> 
> diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
> index 556953469c..d7a2c4009e 100644
> --- a/include/hw/cxl/cxl_device.h
> +++ b/include/hw/cxl/cxl_device.h
> @@ -114,8 +114,9 @@ typedef enum {
>  typedef struct cxl_device_state CXLDeviceState;
>  struct cxl_cmd;
>  typedef CXLRetCode (*opcode_handler)(const struct cxl_cmd *cmd,
> - uint8_t *payload,
> - CXLDeviceState *cxl_dstate, uint16_t 
> *len);
> + uint8_t *payload_in, size_t len_in,
> + uint8_t *payload_out, size_t *len_out,
> + CXLDeviceState *cxl_dstate);
>  struct cxl_cmd {
>  const char *name;
>  opcode_handler handler;
> @@ -390,7 +391,7 @@ bool cxl_event_insert(CXLDeviceState *cxlds, 
> CXLEventLogType log_type,
>CXLEventRecordRaw *event);
>  CXLRetCode cxl_event_get_records(CXLDeviceState *cxlds, CXLGetEventPayload 
> *pl,
>   uint8_t log_type, int max_recs,
> - uint16_t *len);
> + size_t *len);
>  CXLRetCode cxl_event_clear_records(CXLDeviceState *cxlds,
> CXLClearEventPayload *pl);
>  
> diff --git a/hw/cxl/cxl-events.c b/hw/cxl/cxl-events.c
> index e2172b94b9..bee6dfaf14 100644
> --- a/hw/cxl/cxl-events.c
> +++ b/hw/cxl/cxl-events.c
> @@ -143,7 +143,7 @@ bool cxl_event_insert(CXLDeviceState *cxlds, 
> CXLEventLogType log_type,
>  
>  CXLRetCode cxl_event_get_records(CXLDeviceState *cxlds, CXLGetEventPayload 
> *pl,
>   uint8_t log_type, int max_recs,
> - uint16_t *len)
> + size_t *len)
>  {
>  CXLEventLog *log;
>  CXLEvent *entry;
> diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
> index c02de06943..4bdbc2ee83 100644
> --- a/hw/cxl/cxl-mailbox-utils.c
> +++ b/hw/cxl/cxl-mailbox-utils.c
> @@ -71,9 +71,9 @@ enum {
>  
>  
>  static CXLRetCode cmd_events_get_records(const struct cxl_cmd *cmd,
> - uint8_t *payload,
> - CXLDeviceState *cxlds,
> - uint16_t *len)
> + uint8_t *payload_in, size_t len_in,
> + uint8_t *payload_out, size_t 
> *len_out,
> + CXLDeviceState *cxlds)
>  {
>  CXLGetEventPayload *pl;
>  uint8_t log_type;
> @@ -83,9 +83,9 @@ static CXLRetCode cmd_events_get_records(const struct 
> cxl_cmd *cmd,
>  return CXL_MBOX_INVALID_INPUT;
>  }
>  
> -log_type = payload[0];
> +log_type = payload_in[0];
>  
> -pl = (CXLGetEventPayload *)payload;
> +pl = (CXLGetEventPayload *)payload_out;
>  memset(pl, 0, sizeof(*pl));
>  
>  max_recs = (cxlds->payload_size - CXL_EVENT_PAYLOAD_HDR_SIZE) /
> @@ -94,30 +94,34 @@ static CXLRetCode cmd_events_get_records(const struct 
> cxl_cmd *cmd,
>  max_recs = 0x;
>  }
>  
> -return cxl_event_get_records(cxlds, pl, log_type, max_recs, len);
> +return cxl_event_get_records(cxlds, pl, log_type, max_recs, len_out);
>  }
>  
>  static CXLRetCode cmd_events_clear_records(const struct cxl_cmd *cmd,
> -   uint8_t *payload,
> -   CXLDeviceState *cxlds,
> -   uint16_t *len)
> +   uint8_t *payload_in,
> +   size_t len_in,
> +   uint8_t *payload_out,
> +   size_t *len_out,
> +   CXLDeviceState *cxlds)
>  {
>  CXLClearEventPayload *pl;
>  
> -pl = (CXLClearEventPayload *)payload;
> -*len = 0;
> +pl = (

Re: [PATCH 01/19] hw/cxl/mbox: Pull the payload out of struct cxl_cmd and make instances constant

[Fan Ni]

On Mon, Sep 25, 2023 at 05:11:06PM +0100, Jonathan Cameron wrote:
> Putting the pointer in the structure for command handling puts a single
> variable element inside an otherwise constant structure. Move it out as
> a directly passed variable and take the cxl_cmd structures constant.
> 
> Signed-off-by: Jonathan Cameron 
> ---

Reviewed-by: Fan Ni 

>  include/hw/cxl/cxl_device.h |  13 
>  hw/cxl/cxl-mailbox-utils.c  | 121 +++-
>  2 files changed, 78 insertions(+), 56 deletions(-)
> 
> diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
> index 007ddaf078..556953469c 100644
> --- a/include/hw/cxl/cxl_device.h
> +++ b/include/hw/cxl/cxl_device.h
> @@ -111,6 +111,18 @@ typedef enum {
>  CXL_MBOX_MAX = 0x17
>  } CXLRetCode;
>  
> +typedef struct cxl_device_state CXLDeviceState;
> +struct cxl_cmd;
> +typedef CXLRetCode (*opcode_handler)(const struct cxl_cmd *cmd,
> + uint8_t *payload,
> + CXLDeviceState *cxl_dstate, uint16_t 
> *len);
> +struct cxl_cmd {
> +const char *name;
> +opcode_handler handler;
> +ssize_t in;
> +uint16_t effect; /* Reported in CEL */
> +};
> +
>  typedef struct CXLEvent {
>  CXLEventRecordRaw data;
>  QSIMPLEQ_ENTRY(CXLEvent) node;
> @@ -178,6 +190,7 @@ typedef struct cxl_device_state {
>  uint64_t pmem_size;
>  uint64_t vmem_size;
>  
> +const struct cxl_cmd (*cxl_cmd_set)[256];
>  CXLEventLog event_logs[CXL_EVENT_TYPE_MAX];
>  } CXLDeviceState;
>  
> diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
> index ab082ec9de..c02de06943 100644
> --- a/hw/cxl/cxl-mailbox-utils.c
> +++ b/hw/cxl/cxl-mailbox-utils.c
> @@ -69,18 +69,9 @@ enum {
>  #define CLEAR_POISON   0x2
>  };
>  
> -struct cxl_cmd;
> -typedef CXLRetCode (*opcode_handler)(struct cxl_cmd *cmd,
> -   CXLDeviceState *cxl_dstate, uint16_t 
> *len);
> -struct cxl_cmd {
> -const char *name;
> -opcode_handler handler;
> -ssize_t in;
> -uint16_t effect; /* Reported in CEL */
> -uint8_t *payload;
> -};
>  
> -static CXLRetCode cmd_events_get_records(struct cxl_cmd *cmd,
> +static CXLRetCode cmd_events_get_records(const struct cxl_cmd *cmd,
> + uint8_t *payload,
>   CXLDeviceState *cxlds,
>   uint16_t *len)
>  {
> @@ -92,9 +83,9 @@ static CXLRetCode cmd_events_get_records(struct cxl_cmd 
> *cmd,
>  return CXL_MBOX_INVALID_INPUT;
>  }
>  
> -log_type = *((uint8_t *)cmd->payload);
> +log_type = payload[0];
>  
> -pl = (CXLGetEventPayload *)cmd->payload;
> +pl = (CXLGetEventPayload *)payload;
>  memset(pl, 0, sizeof(*pl));
>  
>  max_recs = (cxlds->payload_size - CXL_EVENT_PAYLOAD_HDR_SIZE) /
> @@ -106,25 +97,27 @@ static CXLRetCode cmd_events_get_records(struct cxl_cmd 
> *cmd,
>  return cxl_event_get_records(cxlds, pl, log_type, max_recs, len);
>  }
>  
> -static CXLRetCode cmd_events_clear_records(struct cxl_cmd *cmd,
> +static CXLRetCode cmd_events_clear_records(const struct cxl_cmd *cmd,
> +   uint8_t *payload,
> CXLDeviceState *cxlds,
> uint16_t *len)
>  {
>  CXLClearEventPayload *pl;
>  
> -pl = (CXLClearEventPayload *)cmd->payload;
> +pl = (CXLClearEventPayload *)payload;
>  *len = 0;
>  return cxl_event_clear_records(cxlds, pl);
>  }
>  
> -static CXLRetCode cmd_events_get_interrupt_policy(struct cxl_cmd *cmd,
> +static CXLRetCode cmd_events_get_interrupt_policy(const struct cxl_cmd *cmd,
> +  uint8_t *payload,
>CXLDeviceState *cxlds,
>uint16_t *len)
>  {
>  CXLEventInterruptPolicy *policy;
>  CXLEventLog *log;
>  
> -policy = (CXLEventInterruptPolicy *)cmd->payload;
> +policy = (CXLEventInterruptPolicy *)payload;
>  memset(policy, 0, sizeof(*policy));
>  
>  log = >event_logs[CXL_EVENT_TYPE_INFO];
> @@ -157,7 +150,8 @@ static CXLRetCode cmd_events_get_interrupt_policy(struct 
> cxl_cmd *cmd,
>  return CXL_MBOX_SUCCESS;
>  }
>  
> -static CXLRetCode cmd_events_set_interrupt_policy(struct cxl_cmd *cmd,
> +static CXLRetCode cmd_events_set_interrupt_policy(const struct cxl_cmd *cmd,
> +   

Re: [PATCH] hw/cxl: Fix local variable shadowing of cap_hdrs

[Fan Ni]

On Mon, Sep 25, 2023 at 04:22:58PM +0100, Jonathan Cameron wrote:

> Rename the version not burried in the macro to cap_h.
The change looks good to me. Just one minor thing. why "version" get
involved here?

Fan
> 
> Signed-off-by: Jonathan Cameron 
> ---
> 
> I had another instance of this in a series I'll post later today.
> Cleaned that up the same way.
> 
>  hw/cxl/cxl-device-utils.c | 8 
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/hw/cxl/cxl-device-utils.c b/hw/cxl/cxl-device-utils.c
> index 517f06d869..bd68328032 100644
> --- a/hw/cxl/cxl-device-utils.c
> +++ b/hw/cxl/cxl-device-utils.c
> @@ -283,13 +283,13 @@ static void memdev_reg_init_common(CXLDeviceState 
> *cxl_dstate) { }
>  
>  void cxl_device_register_init_common(CXLDeviceState *cxl_dstate)
>  {
> -uint64_t *cap_hdrs = cxl_dstate->caps_reg_state64;
> +uint64_t *cap_h = cxl_dstate->caps_reg_state64;
>  const int cap_count = 3;
>  
>  /* CXL Device Capabilities Array Register */
> -ARRAY_FIELD_DP64(cap_hdrs, CXL_DEV_CAP_ARRAY, CAP_ID, 0);
> -ARRAY_FIELD_DP64(cap_hdrs, CXL_DEV_CAP_ARRAY, CAP_VERSION, 1);
> -ARRAY_FIELD_DP64(cap_hdrs, CXL_DEV_CAP_ARRAY, CAP_COUNT, cap_count);
> +ARRAY_FIELD_DP64(cap_h, CXL_DEV_CAP_ARRAY, CAP_ID, 0);
> +ARRAY_FIELD_DP64(cap_h, CXL_DEV_CAP_ARRAY, CAP_VERSION, 1);
> +ARRAY_FIELD_DP64(cap_h, CXL_DEV_CAP_ARRAY, CAP_COUNT, cap_count);
>  
>  cxl_device_cap_init(cxl_dstate, DEVICE_STATUS, 1, 2);
>  device_reg_init_common(cxl_dstate);
> -- 
> 2.39.2
> 

Re: [PATCH v2 4/4] hw/cxl: Line length reductions

[Fan Ni]

On Fri, Sep 15, 2023 at 06:04:18PM +0100, Jonathan Cameron wrote:
> Michael Tsirkin observed that there were some unnecessarily
> long lines in the CXL code in a recent review.
> This patch is intended to rectify that where it does not
> hurt readability.
> 
> Reviewed-by: Michael Tokarev 
> Signed-off-by: Jonathan Cameron 
> ---

One minor comment inline. Other than that, looks good to me.


>  include/hw/cxl/cxl_component.h |  3 ++-
>  include/hw/cxl/cxl_device.h|  5 +++--
>  include/hw/cxl/cxl_events.h|  3 ++-
>  hw/cxl/cxl-cdat.c  |  3 ++-
>  hw/cxl/cxl-component-utils.c   | 10 ++
>  hw/cxl/cxl-events.c|  9 ++---
>  hw/cxl/cxl-mailbox-utils.c | 21 ++---
>  hw/mem/cxl_type3.c | 31 +++
>  hw/mem/cxl_type3_stubs.c   |  5 +++--
>  9 files changed, 57 insertions(+), 33 deletions(-)
> 
> diff --git a/include/hw/cxl/cxl_component.h b/include/hw/cxl/cxl_component.h
> index 3c795a6278..e52dd8d2b9 100644
> --- a/include/hw/cxl/cxl_component.h
> +++ b/include/hw/cxl/cxl_component.h
> @@ -175,7 +175,8 @@ HDM_DECODER_INIT(3);
>  (CXL_IDE_REGISTERS_OFFSET + CXL_IDE_REGISTERS_SIZE)
>  #define CXL_SNOOP_REGISTERS_SIZE   0x8
>  
> -QEMU_BUILD_BUG_MSG((CXL_SNOOP_REGISTERS_OFFSET + CXL_SNOOP_REGISTERS_SIZE) 
> >= 0x1000,
> +QEMU_BUILD_BUG_MSG((CXL_SNOOP_REGISTERS_OFFSET +
> +CXL_SNOOP_REGISTERS_SIZE) >= 0x1000,
> "No space for registers");
>  
>  typedef struct component_registers {
> diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
> index 51cd0d9ce3..007ddaf078 100644
> --- a/include/hw/cxl/cxl_device.h
> +++ b/include/hw/cxl/cxl_device.h
> @@ -192,7 +192,7 @@ void cxl_device_register_init_common(CXLDeviceState *dev);
>   * Documented as a 128 bit register, but 64 bit accesses and the second
>   * 64 bits are currently reserved.
>   */
> -REG64(CXL_DEV_CAP_ARRAY, 0) /* Documented as 128 bit register but 64 byte 
> accesses */
> +REG64(CXL_DEV_CAP_ARRAY, 0)
>  FIELD(CXL_DEV_CAP_ARRAY, CAP_ID, 0, 16)
>  FIELD(CXL_DEV_CAP_ARRAY, CAP_VERSION, 16, 8)
>  FIELD(CXL_DEV_CAP_ARRAY, CAP_COUNT, 32, 16)
> @@ -361,7 +361,8 @@ struct CXLType3Class {
>  uint64_t offset);
>  void (*set_lsa)(CXLType3Dev *ct3d, const void *buf, uint64_t size,
>  uint64_t offset);
> -bool (*set_cacheline)(CXLType3Dev *ct3d, uint64_t dpa_offset, uint8_t 
> *data);
> +bool (*set_cacheline)(CXLType3Dev *ct3d, uint64_t dpa_offset,
> +  uint8_t *data);
>  };
>  
>  MemTxResult cxl_type3_read(PCIDevice *d, hwaddr host_addr, uint64_t *data,
> diff --git a/include/hw/cxl/cxl_events.h b/include/hw/cxl/cxl_events.h
> index 089ba2091f..d778487b7e 100644
> --- a/include/hw/cxl/cxl_events.h
> +++ b/include/hw/cxl/cxl_events.h
> @@ -92,7 +92,8 @@ typedef enum CXLEventIntMode {
>  CXL_INT_RES  = 0x03,
>  } CXLEventIntMode;
>  #define CXL_EVENT_INT_MODE_MASK 0x3
> -#define CXL_EVENT_INT_SETTING(vector) uint8_t)vector & 0xf) << 4) | 
> CXL_INT_MSI_MSIX)
> +#define CXL_EVENT_INT_SETTING(vector) \
> +uint8_t)vector & 0xf) << 4) | CXL_INT_MSI_MSIX)
>  typedef struct CXLEventInterruptPolicy {
>  uint8_t info_settings;
>  uint8_t warn_settings;
> diff --git a/hw/cxl/cxl-cdat.c b/hw/cxl/cxl-cdat.c
> index d246d6885b..639a2db3e1 100644
> --- a/hw/cxl/cxl-cdat.c
> +++ b/hw/cxl/cxl-cdat.c
> @@ -60,7 +60,8 @@ static void ct3_build_cdat(CDATObject *cdat, Error **errp)
>  return;
>  }
>  
> -cdat->built_buf_len = cdat->build_cdat_table(>built_buf, 
> cdat->private);
> +cdat->built_buf_len = cdat->build_cdat_table(>built_buf,
> + cdat->private);
>  
>  if (!cdat->built_buf_len) {
>  /* Build later as not all data available yet */
> diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c
> index c8d632d540..915c208209 100644
> --- a/hw/cxl/cxl-component-utils.c
> +++ b/hw/cxl/cxl-component-utils.c
> @@ -240,7 +240,8 @@ static void hdm_init_common(uint32_t *reg_state, uint32_t 
> *write_msk,
>  ARRAY_FIELD_DP32(reg_state, CXL_HDM_DECODER_CAPABILITY, TARGET_COUNT, 1);
>  ARRAY_FIELD_DP32(reg_state, CXL_HDM_DECODER_CAPABILITY, INTERLEAVE_256B, 
> 1);
>  ARRAY_FIELD_DP32(reg_state, CXL_HDM_DECODER_CAPABILITY, INTERLEAVE_4K, 
> 1);
> -ARRAY_FIELD_DP32(reg_state, CXL_HDM_DECODER_CAPABILITY, 
> POISON_ON_ERR_CAP, 0);
> +ARRAY_FIELD_DP32(reg_state, CXL_HDM_DECODER_CAPABILITY,
> + POISON_ON_ERR_CAP, 0);
>  ARRAY_FIELD_DP32(reg_state, CXL_HDM_DECODER_GLOBAL_CONTROL,
>   HDM_DECODER_ENABLE, 0);
>  write_msk[R_CXL_HDM_DECODER_GLOBAL_CONTROL] = 0x3;
> @@ -263,13 +264,14 @@ static void hdm_init_common(uint32_t *reg_state, 
> uint32_t *write_msk,
>  }
>  }
>  
> -void cxl_component_register_init_common(uint32_t *reg_state, uint32_t 
> 

Re: [PATCH v2 3/4] hw/cxl: CXLDVSECPortExtensions renamed to CXLDVSECPortExt

[Fan Ni]

On Fri, Sep 15, 2023 at 06:04:17PM +0100, Jonathan Cameron wrote:

> Done to reduce line lengths where this is used.
> Ext seems sufficiently obvious that it need not be spelt out
> fully.
> 
> Signed-off-by: Jonathan Cameron 
> Reviewed-by: Philippe Mathieu-Daudé 
> ---

Reviewed-by: Fan Ni 

>  include/hw/cxl/cxl_pci.h   |  6 ++---
>  hw/cxl/cxl-component-utils.c   | 49 --
>  hw/pci-bridge/cxl_downstream.c |  2 +-
>  hw/pci-bridge/cxl_root_port.c  |  2 +-
>  hw/pci-bridge/cxl_upstream.c   |  2 +-
>  5 files changed, 35 insertions(+), 26 deletions(-)
> 
> diff --git a/include/hw/cxl/cxl_pci.h b/include/hw/cxl/cxl_pci.h
> index 407be95b9e..ddf01a543b 100644
> --- a/include/hw/cxl/cxl_pci.h
> +++ b/include/hw/cxl/cxl_pci.h
> @@ -86,7 +86,7 @@ typedef struct CXLDVSECDevice {
>  QEMU_BUILD_BUG_ON(sizeof(CXLDVSECDevice) != 0x38);
>  
>  /* CXL 2.0 - 8.1.5 (ID 0003) */
> -typedef struct CXLDVSECPortExtensions {
> +typedef struct CXLDVSECPortExt {
>  DVSECHeader hdr;
>  uint16_t status;
>  uint16_t control;
> @@ -100,8 +100,8 @@ typedef struct CXLDVSECPortExtensions {
>  uint32_t alt_prefetch_limit_high;
>  uint32_t rcrb_base;
>  uint32_t rcrb_base_high;
> -} CXLDVSECPortExtensions;
> -QEMU_BUILD_BUG_ON(sizeof(CXLDVSECPortExtensions) != 0x28);
> +} CXLDVSECPortExt;
> +QEMU_BUILD_BUG_ON(sizeof(CXLDVSECPortExt) != 0x28);
>  
>  #define PORT_CONTROL_OFFSET  0xc
>  #define PORT_CONTROL_UNMASK_SBR  1
> diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c
> index 7ef3ef2bd6..c8d632d540 100644
> --- a/hw/cxl/cxl-component-utils.c
> +++ b/hw/cxl/cxl-component-utils.c
> @@ -392,26 +392,35 @@ void cxl_component_create_dvsec(CXLComponentState *cxl,
>  case NON_CXL_FUNCTION_MAP_DVSEC:
>  break; /* Not yet implemented */
>  case EXTENSIONS_PORT_DVSEC:
> -wmask[offset + offsetof(CXLDVSECPortExtensions, control)] = 0x0F;
> -wmask[offset + offsetof(CXLDVSECPortExtensions, control) + 1] = 0x40;
> -wmask[offset + offsetof(CXLDVSECPortExtensions, alt_bus_base)] = 
> 0xFF;
> -wmask[offset + offsetof(CXLDVSECPortExtensions, alt_bus_limit)] = 
> 0xFF;
> -wmask[offset + offsetof(CXLDVSECPortExtensions, alt_memory_base)] = 
> 0xF0;
> -wmask[offset + offsetof(CXLDVSECPortExtensions, alt_memory_base) + 
> 1] = 0xFF;
> -wmask[offset + offsetof(CXLDVSECPortExtensions, alt_memory_limit)] = 
> 0xF0;
> -wmask[offset + offsetof(CXLDVSECPortExtensions, alt_memory_limit) + 
> 1] = 0xFF;
> -wmask[offset + offsetof(CXLDVSECPortExtensions, alt_prefetch_base)] 
> = 0xF0;
> -wmask[offset + offsetof(CXLDVSECPortExtensions, alt_prefetch_base) + 
> 1] = 0xFF;
> -wmask[offset + offsetof(CXLDVSECPortExtensions, alt_prefetch_limit)] 
> = 0xF0;
> -wmask[offset + offsetof(CXLDVSECPortExtensions, alt_prefetch_limit) 
> + 1] = 0xFF;
> -wmask[offset + offsetof(CXLDVSECPortExtensions, 
> alt_prefetch_base_high)] = 0xFF;
> -wmask[offset + offsetof(CXLDVSECPortExtensions, 
> alt_prefetch_base_high) + 1] = 0xFF;
> -wmask[offset + offsetof(CXLDVSECPortExtensions, 
> alt_prefetch_base_high) + 2] = 0xFF;
> -wmask[offset + offsetof(CXLDVSECPortExtensions, 
> alt_prefetch_base_high) + 3] = 0xFF;
> -wmask[offset + offsetof(CXLDVSECPortExtensions, 
> alt_prefetch_limit_high)] = 0xFF;
> -wmask[offset + offsetof(CXLDVSECPortExtensions, 
> alt_prefetch_limit_high) + 1] = 0xFF;
> -wmask[offset + offsetof(CXLDVSECPortExtensions, 
> alt_prefetch_limit_high) + 2] = 0xFF;
> -wmask[offset + offsetof(CXLDVSECPortExtensions, 
> alt_prefetch_limit_high) + 3] = 0xFF;
> +wmask[offset + offsetof(CXLDVSECPortExt, control)] = 0x0F;
> +wmask[offset + offsetof(CXLDVSECPortExt, control) + 1] = 0x40;
> +wmask[offset + offsetof(CXLDVSECPortExt, alt_bus_base)] = 0xFF;
> +wmask[offset + offsetof(CXLDVSECPortExt, alt_bus_limit)] = 0xFF;
> +wmask[offset + offsetof(CXLDVSECPortExt, alt_memory_base)] = 0xF0;
> +wmask[offset + offsetof(CXLDVSECPortExt, alt_memory_base) + 1] = 
> 0xFF;
> +wmask[offset + offsetof(CXLDVSECPortExt, alt_memory_limit)] = 0xF0;
> +wmask[offset + offsetof(CXLDVSECPortExt, alt_memory_limit) + 1] = 
> 0xFF;
> +wmask[offset + offsetof(CXLDVSECPortExt, alt_prefetch_base)] = 0xF0;
> +wmask[offset + offsetof(CXLDVSECPortExt, alt_prefetch_base) + 1] = 
> 0xFF;
> +wmask[offset + offsetof(CXLDVSECPortExt, alt_prefetch_limit)] = 0xF0;
> +wmask[offset + offsetof(CXLDVSECPortExt, alt_prefetch_limit) + 1] =
> +   

Re: [PATCH v2 2/4] hw/cxl: Use switch statements for read and write of cachemem registers

[Fan Ni]

On Fri, Sep 15, 2023 at 06:04:16PM +0100, Jonathan Cameron wrote:

> Establishing that only register accesses of size 4 and 8 can occur
> using these functions requires looking at their callers. Make it
> easier to see that by using switch statements.
> Assertions are used to enforce that the register storage is of the
> matching size, allowing fixed values to be used for divisors of
> the array indices.
> 
> Suggested-by: Michael Tokarev 
> Signed-off-by: Jonathan Cameron 
> ---

Reviewed-by: Fan Ni 

> v2: Use switch statements. Note we coudl have renamed registers to make
> the sizes visible as is true of some other cases, but that felt odd
> as this is the only size these registers can be, so why call it out.
> The asserts do the same job.
>  hw/cxl/cxl-component-utils.c | 65 +++-
>  1 file changed, 42 insertions(+), 23 deletions(-)
> 
> diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c
> index f3bbf0fd13..7ef3ef2bd6 100644
> --- a/hw/cxl/cxl-component-utils.c
> +++ b/hw/cxl/cxl-component-utils.c
> @@ -67,16 +67,24 @@ static uint64_t cxl_cache_mem_read_reg(void *opaque, 
> hwaddr offset,
>  CXLComponentState *cxl_cstate = opaque;
>  ComponentRegisters *cregs = _cstate->crb;
>  
> -if (size == 8) {
> +switch (size) {
> +case 4:
> +if (cregs->special_ops && cregs->special_ops->read) {
> +return cregs->special_ops->read(cxl_cstate, offset, 4);
> +} else {
> +static_assert(sizeof(*cregs->cache_mem_registers) == 4);
> +return cregs->cache_mem_registers[offset / 4];
> +}
> +case 8:
>  qemu_log_mask(LOG_UNIMP,
>"CXL 8 byte cache mem registers not implemented\n");
>  return 0;
> -}
> -
> -if (cregs->special_ops && cregs->special_ops->read) {
> -return cregs->special_ops->read(cxl_cstate, offset, size);
> -} else {
> -return cregs->cache_mem_registers[offset / 
> sizeof(*cregs->cache_mem_registers)];
> +default:
> +/*
> + * In line with specifiction limitaions on access sizes, this
> + * routine is not called with other sizes.
> + */
> +g_assert_not_reached();
>  }
>  }
>  
> @@ -117,25 +125,36 @@ static void cxl_cache_mem_write_reg(void *opaque, 
> hwaddr offset, uint64_t value,
>  ComponentRegisters *cregs = _cstate->crb;
>  uint32_t mask;
>  
> -if (size == 8) {
> +switch (size) {
> +case 4:
> +static_assert(sizeof(*cregs->cache_mem_regs_write_mask) == 4);
> +static_assert(sizeof(*cregs->cache_mem_registers) == 4);
> +mask = cregs->cache_mem_regs_write_mask[offset / 4];
> +value &= mask;
> +/* RO bits should remain constant. Done by reading existing value */
> +value |= ~mask & cregs->cache_mem_registers[offset / 4];
> +if (cregs->special_ops && cregs->special_ops->write) {
> +cregs->special_ops->write(cxl_cstate, offset, value, size);
> +return;
> +}
> +
> +if (offset >= A_CXL_HDM_DECODER_CAPABILITY &&
> +offset <= A_CXL_HDM_DECODER3_TARGET_LIST_HI) {
> +dumb_hdm_handler(cxl_cstate, offset, value);
> +} else {
> +cregs->cache_mem_registers[offset / 4] = value;
> +}
> +return;
> +case 8:
>  qemu_log_mask(LOG_UNIMP,
>"CXL 8 byte cache mem registers not implemented\n");
>  return;
> -}
> -mask = cregs->cache_mem_regs_write_mask[offset / 
> sizeof(*cregs->cache_mem_regs_write_mask)];
> -value &= mask;
> -/* RO bits should remain constant. Done by reading existing value */
> -value |= ~mask & cregs->cache_mem_registers[offset / 
> sizeof(*cregs->cache_mem_registers)];
> -if (cregs->special_ops && cregs->special_ops->write) {
> -cregs->special_ops->write(cxl_cstate, offset, value, size);
> -return;
> -}
> -
> -if (offset >= A_CXL_HDM_DECODER_CAPABILITY &&
> -offset <= A_CXL_HDM_DECODER3_TARGET_LIST_HI) {
> -dumb_hdm_handler(cxl_cstate, offset, value);
> -} else {
> -cregs->cache_mem_registers[offset / 
> sizeof(*cregs->cache_mem_registers)] = value;
> +default:
> +/*
> + * In line with specifiction limitaions on access sizes, this
> + * routine is not called with other sizes.
> + */
> +g_assert_not_reached();
>  }
>  }
>  
> -- 
> 2.39.2
> 

Re: [PATCH v2 1/4] hw/cxl: Use a switch to explicitly check size in caps_reg_read()

[Fan Ni]

On Fri, Sep 15, 2023 at 06:04:15PM +0100, Jonathan Cameron wrote:

> Bring this read function inline with the others that do
> check for unexpected size values.
> 
> Also reduces line lengths to sub 80 chars.
> 
> Signed-off-by: Jonathan Cameron 
> Reviewed-by: Philippe Mathieu-Daudé 
> ---

Reviewed-by: Fan Ni 

>  hw/cxl/cxl-device-utils.c | 11 +++
>  1 file changed, 7 insertions(+), 4 deletions(-)
> 
> diff --git a/hw/cxl/cxl-device-utils.c b/hw/cxl/cxl-device-utils.c
> index 517f06d869..cd0c45a2ed 100644
> --- a/hw/cxl/cxl-device-utils.c
> +++ b/hw/cxl/cxl-device-utils.c
> @@ -32,10 +32,13 @@ static uint64_t caps_reg_read(void *opaque, hwaddr 
> offset, unsigned size)
>  {
>  CXLDeviceState *cxl_dstate = opaque;
>  
> -if (size == 4) {
> -return cxl_dstate->caps_reg_state32[offset / 
> sizeof(*cxl_dstate->caps_reg_state32)];
> -} else {
> -return cxl_dstate->caps_reg_state64[offset / 
> sizeof(*cxl_dstate->caps_reg_state64)];
> +switch (size) {
> +case 4:
> +return cxl_dstate->caps_reg_state32[offset / size];
> +case 8:
> +return cxl_dstate->caps_reg_state64[offset / size];
> +default:
> +g_assert_not_reached();
>  }
>  }
>  
> -- 
> 2.39.2
> 

Re: [PATCH 3/3] docs/cxl: Cleanout some more aarch64 examples.

[Fan Ni]

On Mon, Sep 18, 2023 at 04:02:59PM +0100, Jonathan Cameron wrote:

> These crossed with the previous fix to get rid of examples
> using aarch64 for which support is not yet upstream.
> 
> Signed-off-by: Jonathan Cameron 

Reviewed-by: Fan Ni 

> ---
>  docs/system/devices/cxl.rst | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/docs/system/devices/cxl.rst b/docs/system/devices/cxl.rst
> index b742120657..6ab5f72473 100644
> --- a/docs/system/devices/cxl.rst
> +++ b/docs/system/devices/cxl.rst
> @@ -313,7 +313,7 @@ A very simple setup with just one directly attached CXL 
> Type 3 Persistent Memory
>  
>  A very simple setup with just one directly attached CXL Type 3 Volatile 
> Memory device::
>  
> -  qemu-system-aarch64 -M virt,gic-version=3,cxl=on -m 4g,maxmem=8G,slots=8 
> -cpu max \
> +  qemu-system-x86_64 -M q35,cxl=on -m 4G,maxmem=8G,slots=8 -smp 4 \
>...
>-object memory-backend-ram,id=vmem0,share=on,size=256M \
>-device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
> @@ -323,7 +323,7 @@ A very simple setup with just one directly attached CXL 
> Type 3 Volatile Memory d
>  
>  The same volatile setup may optionally include an LSA region::
>  
> -  qemu-system-aarch64 -M virt,gic-version=3,cxl=on -m 4g,maxmem=8G,slots=8 
> -cpu max \
> +  qemu-system-x86_64 -M q35,cxl=on -m 4G,maxmem=8G,slots=8 -smp 4 \
>...
>-object memory-backend-ram,id=vmem0,share=on,size=256M \
>-object 
> memory-backend-file,id=cxl-lsa0,share=on,mem-path=/tmp/lsa.raw,size=256M \
> -- 
> 2.39.2
> 

Re: [PATCH 2/3] hw/mem/cxl_type3: Add missing copyright and license notice

[Fan Ni]

On Mon, Sep 18, 2023 at 04:02:58PM +0100, Jonathan Cameron wrote:

> This has been missing from the start. Assume it should match
> with cxl/cxl-component-utils.c as both were part of early
> postings from Ben.
> 
> Suggested-by: Philippe Mathieu-Daudé 
> Signed-off-by: Jonathan Cameron 
> ---

Reviewed-by: Fan Ni 

>  hw/mem/cxl_type3.c | 9 +
>  1 file changed, 9 insertions(+)
> 
> diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
> index c5855d4e7d..ad3f0f6a9d 100644
> --- a/hw/mem/cxl_type3.c
> +++ b/hw/mem/cxl_type3.c
> @@ -1,3 +1,12 @@
> +/*
> + * CXL Type 3 (memory expander) device
> + *
> + * Copyright(C) 2020 Intel Corporation.
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2. See the
> + * COPYING file in the top-level directory.
> + */
> +
>  #include "qemu/osdep.h"
>  #include "qemu/units.h"
>  #include "qemu/error-report.h"
> -- 
> 2.39.2
> 

Re: [PATCH 1/3] hw/cxl: Fix out of bound array access

[Fan Ni]

On Mon, Sep 18, 2023 at 04:02:57PM +0100, Jonathan Cameron wrote:

> From: Dmitry Frolov 
> 
> According to cxl_interleave_ways_enc(), fw->num_targets is allowed to be up
> to 16. This also corresponds to CXL r3.0 spec. So, the fw->target_hbs[]
> array is iterated from 0 to 15. But it is staticaly declared of length 8.

s/staticaly/statically/

Fan
> Thus, out of bound array access may occur.
> 
> Fixes: c28db9e000 ("hw/pci-bridge: Make PCIe and CXL PXB Devices inherit from 
> TYPE_PXB_DEV")
> Signed-off-by: Dmitry Frolov 
> Reviewed-by: Michael Tokarev 
> Link: 
> https://urldefense.com/v3/__https://lore.kernel.org/r/20230913101055.754709-1-frolov@swemel.ru__;!!EwVzqGoTKBqv-0DWAJBm!RYOYJeMCX_mlzCETIKjHDYun2TZCQxC7wF1SxIdUwJ3BYbDOtmDpTaVXXiQgiFwLYI_4JAnU6Asem4T0aHKzwzPJWer7$
>  
> Cc: qemu-sta...@nongnu.org
> Signed-off-by: Jonathan Cameron 
> ---
>  include/hw/cxl/cxl.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/include/hw/cxl/cxl.h b/include/hw/cxl/cxl.h
> index 56c9e7676e..4944725849 100644
> --- a/include/hw/cxl/cxl.h
> +++ b/include/hw/cxl/cxl.h
> @@ -29,7 +29,7 @@ typedef struct PXBCXLDev PXBCXLDev;
>  typedef struct CXLFixedWindow {
>  uint64_t size;
>  char **targets;
> -PXBCXLDev *target_hbs[8];
> +PXBCXLDev *target_hbs[16];
>  uint8_t num_targets;
>  uint8_t enc_int_ways;
>  uint8_t enc_int_gran;
> -- 
> 2.39.2
> 

Re: [PATCH 4/4] docs/cxl: Change to lowercase as others

[Fan Ni]

On Mon, Sep 04, 2023 at 02:28:06PM +0100, Jonathan Cameron wrote:
> From: Li Zhijian 
> 
> Using the same style as elsewhere for topology / topo
> 
> Signed-off-by: Li Zhijian 
> Link: 
> https://urldefense.com/v3/__https://lore.kernel.org/r/20230519085802.2106900-2-lizhijian@cn.fujitsu.com__;!!EwVzqGoTKBqv-0DWAJBm!TWHVrdL5Ys7OOFU_w1CJQ5DC6mxu649kYA9GYDJ182CNPuQqpVkWYsB5mlJpVd_BAAmhxCD4Si2CkMERZI7ZE03kPz2c$
>  
> Signed-off-by: Jonathan Cameron 
> ---

Reviewed-by: Fan Ni 

>  docs/system/devices/cxl.rst | 8 
>  1 file changed, 4 insertions(+), 4 deletions(-)
> 
> diff --git a/docs/system/devices/cxl.rst b/docs/system/devices/cxl.rst
> index f12011e230..b742120657 100644
> --- a/docs/system/devices/cxl.rst
> +++ b/docs/system/devices/cxl.rst
> @@ -157,7 +157,7 @@ responsible for allocating appropriate ranges from within 
> the CFMWs
>  and exposing those via normal memory configurations as would be done
>  for system RAM.
>  
> -Example system Topology. x marks the match in each decoder level::
> +Example system topology. x marks the match in each decoder level::
>  
>|<--SYSTEM PHYSICAL ADDRESS MAP (1)->|
>|__   __   __|
> @@ -187,8 +187,8 @@ Example system Topology. x marks the match in each 
> decoder level::
> ___|___   __|__   __|_   ___|_
> (3)|  Root Port 0  | | Root Port 1 | | Root Port 2| | Root Port 3 |
>|  Appears in   | | Appears in  | | Appears in | | Appear in   |
> -  |  PCI topology | | PCI Topology| | PCI Topo   | | PCI Topo|
> -  |  As 0c:00.0   | | as 0c:01.0  | | as de:00.0 | | as de:01.0  |
> +  |  PCI topology | | PCI topology| | PCI topo   | | PCI topo|
> +  |  as 0c:00.0   | | as 0c:01.0  | | as de:00.0 | | as de:01.0  |
>|___| |_| || |_|
>  |  |   |  |
>  |  |   |  |
> @@ -272,7 +272,7 @@ Example topology involving a switch::
>|  Root Port 0  |
>|  Appears in   |
>|  PCI topology |
> -  |  As 0c:00.0   |
> +  |  as 0c:00.0   |
>|___x___|
>|
>|
> -- 
> 2.39.2
> 

Re: [PATCH 2/4] hw/pci-bridge/cxl_upstream: Fix bandwidth entry base unit for SSLBIS

[Fan Ni]

On Mon, Sep 04, 2023 at 02:28:04PM +0100, Jonathan Cameron wrote:
> From: Dave Jiang 
> 
> According to ACPI spec 6.5 5.2.28.4 System Locality Latency and Bandwidth
> Information Structure, if the "Entry Base Unit" is 1024 for BW and the
> matrix entry has the value of 100, the BW is 100 GB/s. So the
> entry_base_unit should be changed from 1000 to 1024 given the comment notes
> it's 16GB/s for .latency_bandwidth.
> 
> Fixes: 882877fc359d ("hw/pci-bridge/cxl-upstream: Add a CDAT table access 
> DOE")
> Signed-off-by: Dave Jiang 
> Signed-off-by: Jonathan Cameron 

Reviewed-by: Fan Ni 

> ---
>  hw/pci-bridge/cxl_upstream.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/hw/pci-bridge/cxl_upstream.c b/hw/pci-bridge/cxl_upstream.c
> index 9159f48a8c..2b9cf0cc97 100644
> --- a/hw/pci-bridge/cxl_upstream.c
> +++ b/hw/pci-bridge/cxl_upstream.c
> @@ -262,7 +262,7 @@ static int build_cdat_table(CDATSubHeader ***cdat_table, 
> void *priv)
>  .length = sslbis_size,
>  },
>  .data_type = HMATLB_DATA_TYPE_ACCESS_BANDWIDTH,
> -.entry_base_unit = 1000,
> +.entry_base_unit = 1024,
>  },
>  };
>  
> -- 
> 2.39.2
> 

Re: [PATCH 1/4] hw/cxl: Fix CFMW config memory leak

[Fan Ni]

On Mon, Sep 04, 2023 at 02:28:03PM +0100, Jonathan Cameron wrote:

> From: Li Zhijian 
> 
> Allocate targets and targets[n] resources when all sanity checks are
> passed to avoid memory leaks.
> 
> Suggested-by: Philippe Mathieu-Daudé 
> Signed-off-by: Li Zhijian 
> Reviewed-by: Philippe Mathieu-Daudé 
> Signed-off-by: Jonathan Cameron 

Reviewed-by: Fan Ni 

> ---
>  hw/cxl/cxl-host.c | 12 ++--
>  1 file changed, 6 insertions(+), 6 deletions(-)
> 
> diff --git a/hw/cxl/cxl-host.c b/hw/cxl/cxl-host.c
> index 034c7805b3..f0920da956 100644
> --- a/hw/cxl/cxl-host.c
> +++ b/hw/cxl/cxl-host.c
> @@ -39,12 +39,6 @@ static void cxl_fixed_memory_window_config(CXLState 
> *cxl_state,
>  return;
>  }
>  
> -fw->targets = g_malloc0_n(fw->num_targets, sizeof(*fw->targets));
> -for (i = 0, target = object->targets; target; i++, target = 
> target->next) {
> -/* This link cannot be resolved yet, so stash the name for now */
> -fw->targets[i] = g_strdup(target->value);
> -}
> -
>  if (object->size % (256 * MiB)) {
>  error_setg(errp,
> "Size of a CXL fixed memory window must be a multiple of 
> 256MiB");
> @@ -64,6 +58,12 @@ static void cxl_fixed_memory_window_config(CXLState 
> *cxl_state,
>  fw->enc_int_gran = 0;
>  }
>  
> +fw->targets = g_malloc0_n(fw->num_targets, sizeof(*fw->targets));
> +for (i = 0, target = object->targets; target; i++, target = 
> target->next) {
> +/* This link cannot be resolved yet, so stash the name for now */
> +fw->targets[i] = g_strdup(target->value);
> +}
> +
>  cxl_state->fixed_windows = g_list_append(cxl_state->fixed_windows,
>   g_steal_pointer());
>  
> -- 
> 2.39.2
> 

Re: [PATCH v2 2/3] hw/cxl: Add QTG _DSM support for ACPI0017 device

[Fan Ni]

On Mon, Sep 04, 2023 at 05:18:46PM +0100, Jonathan Cameron wrote:

> From: Dave Jiang 
> 
> Add a simple _DSM call support for the ACPI0017 device to return a fake QTG
> ID value of 0 in all cases. The enabling is for _DSM plumbing testing
> from the OS.
> 
> Following edited for readbility only
> 
> Device (CXLM)
> {
> Name (_HID, "ACPI0017")  // _HID: Hardware ID
> ...
> Method (_DSM, 4, Serialized)  // _DSM: Device-Specific Method
> {
> If ((Arg0 == ToUUID ("f365f9a6-a7de-4071-a66a-b40c0b4f8e52")))
> {
> If ((Arg2 == Zero))
> {
> Return (Buffer (One) { 0x01 })
> }
> 
> If ((Arg2 == One))
> {
> Return (Package (0x02)
> {
> Buffer (0x02)
> { 0x01, 0x00 },
> Package (0x01)
> {
> Buffer (0x02)
> { 0x00, 0x00 }
> }
> })
> }
> }
> }
> 
> Signed-off-by: Dave Jiang 
> Signed-off-by: Jonathan Cameron 

Looks good to me. One minor comment inline.
> 
> --
> v2: Minor edit to drop reference to switches in patch description.
> ---
>  include/hw/acpi/cxl.h |  1 +
>  hw/acpi/cxl.c | 57 +++
>  hw/i386/acpi-build.c  |  1 +
>  3 files changed, 59 insertions(+)
> 
> diff --git a/include/hw/acpi/cxl.h b/include/hw/acpi/cxl.h
> index acf4418886..8f22c71530 100644
> --- a/include/hw/acpi/cxl.h
> +++ b/include/hw/acpi/cxl.h
> @@ -25,5 +25,6 @@ void cxl_build_cedt(GArray *table_offsets, GArray 
> *table_data,
>  BIOSLinker *linker, const char *oem_id,
>  const char *oem_table_id, CXLState *cxl_state);
>  void build_cxl_osc_method(Aml *dev);
> +void build_cxl_dsm_method(Aml *dev);
>  
>  #endif
> diff --git a/hw/acpi/cxl.c b/hw/acpi/cxl.c
> index 92b46bc932..5e9039785a 100644
> --- a/hw/acpi/cxl.c
> +++ b/hw/acpi/cxl.c
> @@ -30,6 +30,63 @@
>  #include "qapi/error.h"
>  #include "qemu/uuid.h"
>  
> +void build_cxl_dsm_method(Aml *dev)

Not a concern for now, I think, do we need to check the revision
field?

Fan
> +{
> +Aml *method, *ifctx, *ifctx2;
> +
> +method = aml_method("_DSM", 4, AML_SERIALIZED);
> +{
> +Aml *function, *uuid;
> +
> +uuid = aml_arg(0);
> +function = aml_arg(2);
> +/* CXL spec v3.0 9.17.3.1 *, QTG ID _DSM */
> +ifctx = aml_if(aml_equal(
> +uuid, aml_touuid("F365F9A6-A7DE-4071-A66A-B40C0B4F8E52")));
> +
> +/* Function 0, standard DSM query function */
> +ifctx2 = aml_if(aml_equal(function, aml_int(0)));
> +{
> +uint8_t byte_list[1] = { 0x01 }; /* functions 1 only */
> +
> +aml_append(ifctx2,
> +   aml_return(aml_buffer(sizeof(byte_list), byte_list)));
> +}
> +aml_append(ifctx, ifctx2);
> +
> +/*
> + * Function 1
> + * A return value of {1, {0}} inciate that
> + * max supported QTG ID of 1 and recommended QTG is 0.
> + * The values here are faked to simplify emulation.
> + */
> +ifctx2 = aml_if(aml_equal(function, aml_int(1)));
> +{
> +uint16_t word_list[1] = { 0x01 };
> +uint16_t word_list2[1] = { 0 };
> +uint8_t *byte_list = (uint8_t *)word_list;
> +uint8_t *byte_list2 = (uint8_t *)word_list2;
> +Aml *pak, *pak1;
> +
> +/*
> + * The return package is a package of a WORD and another package.
> + * The embedded package contains 0 or more WORDs for the
> + * recommended QTG IDs.
> + */
> +pak1 = aml_package(1);
> +aml_append(pak1, aml_buffer(sizeof(uint16_t), byte_list2));
> +pak = aml_package(2);
> +aml_append(pak, aml_buffer(sizeof(uint16_t), byte_list));
> +aml_append(pak, pak1);
> +
> +aml_append(ifctx2, aml_return(pak));
> +}
> +aml_append(ifctx, ifctx2);
> +}
> +aml_append(method, ifctx);
> +aml_append(dev, method);
> +}
> +
>  static void cedt_build_chbs(GArray *table_data, PXBCXLDev *cxl)
>  {
>  PXBDev *pxb = PXB_DEV(cxl);
> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> index bb12b0ad43..d3bc5875eb 100644
> --- a/hw/i386/acpi-build.c
> +++ b/hw/i386/acpi-build.c
> @@ -1422,6 +1422,7 @@ static void build_acpi0017(Aml *table)
>  method = aml_method("_STA", 0, AML_NOTSERIALIZED);
>  aml_append(method, aml_return(aml_int(0x01)));
>  aml_append(dev, method);
> +build_cxl_dsm_method(dev);
>  
>  aml_append(scope, dev);
>  aml_append(table, scope);
> -- 
> 2.39.2
> 
> 

Re: [PATCH v2 1/3] tests/acpi: Allow update of DSDT.cxl

[Fan Ni]

On Mon, Sep 04, 2023 at 05:18:45PM +0100, Jonathan Cameron wrote:

> Addition of QTG in following patch requires an update to the test
> data.
> 
> Signed-off-by: Jonathan Cameron 
> ---

Reviewed-by: Fan Ni 

>  tests/qtest/bios-tables-test-allowed-diff.h | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/tests/qtest/bios-tables-test-allowed-diff.h 
> b/tests/qtest/bios-tables-test-allowed-diff.h
> index dfb8523c8b..9ce0f596cc 100644
> --- a/tests/qtest/bios-tables-test-allowed-diff.h
> +++ b/tests/qtest/bios-tables-test-allowed-diff.h
> @@ -1 +1,2 @@
>  /* List of comma-separated changed AML files to ignore */
> +"tests/data/acpi/q35/DSDT.cxl",
> -- 
> 2.39.2
> 
> 

Re: [PATCH v2 3/3] tests/acpi: Update DSDT.cxl with QTG DSM

[Fan Ni]

On Mon, Sep 04, 2023 at 05:18:47PM +0100, Jonathan Cameron wrote:

> Description of change in previous patch.
> 
> Signed-off-by: Jonathan Cameron 

Reviewed-by: Fan Ni 

> ---
>  tests/qtest/bios-tables-test-allowed-diff.h |   1 -
>  tests/data/acpi/q35/DSDT.cxl| Bin 9655 -> 9723 bytes
>  2 files changed, 1 deletion(-)
> 
> diff --git a/tests/qtest/bios-tables-test-allowed-diff.h 
> b/tests/qtest/bios-tables-test-allowed-diff.h
> index 9ce0f596cc..dfb8523c8b 100644
> --- a/tests/qtest/bios-tables-test-allowed-diff.h
> +++ b/tests/qtest/bios-tables-test-allowed-diff.h
> @@ -1,2 +1 @@
>  /* List of comma-separated changed AML files to ignore */
> -"tests/data/acpi/q35/DSDT.cxl",
> diff --git a/tests/data/acpi/q35/DSDT.cxl b/tests/data/acpi/q35/DSDT.cxl
> index 
> ee16a861c2de7b7caaf11d91c50fcdf308815233..d4272e87c00e010a6805b6a276fcc87d9b6ead17
>  100644
> GIT binary patch
> delta 129
> zcmdn){o9+%CDZ1KTP@zG5VY|arrz8vu$o-VwO zA{_C-A}77)2ae;$4D$c@|hs?J;#_4B>ug$~QIw(xNK_XREBoSen5M39-0
> gae?^cEXE~5f=q&}Tuh7%LL7`B1_Q(9{fa-B0lXk1>i_@%
> 
> delta 61
> zcmezEz1^G3CD5k9^g@gANoypGNRo(2Yn<_sbn
> R@xdXE3`-a{Gb{aI1^_O_5bOW|
> 
> -- 
> 2.39.2
> 
> 

Re: [PATCH v3 4/4] hw/cxl: Support 4 HDM decoders at all levels of topology

[Fan Ni]

On Mon, Sep 11, 2023 at 12:43:13PM +0100, Jonathan Cameron wrote:

> Support these decoders in CXL host bridges (pxb-cxl), CXL Switch USP
> and CXL Type 3 end points.
> 
> Signed-off-by: Jonathan Cameron 
> 
> ---

One comment inline, other than that, looks good to me.


> v3: Factor out the hdm_inc changes to previous patch.
> Fix use of encoded hdm count as if it were decoded in cxl-host.
> Minor refactoring to make that path and one in cxl_type3.c
> look more similar.
> ---
>  include/hw/cxl/cxl_component.h | 10 +++-
>  hw/cxl/cxl-component-utils.c   |  7 ++-
>  hw/cxl/cxl-host.c  | 67 
>  hw/mem/cxl_type3.c | 96 +++---
>  4 files changed, 124 insertions(+), 56 deletions(-)
> 
> diff --git a/include/hw/cxl/cxl_component.h b/include/hw/cxl/cxl_component.h
> index 7c864d2044..3c795a6278 100644
> --- a/include/hw/cxl/cxl_component.h
> +++ b/include/hw/cxl/cxl_component.h
> @@ -135,6 +135,10 @@ REG32(CXL_RAS_ERR_HEADER0, CXL_RAS_REGISTERS_OFFSET + 
> 0x18)
>REG32(CXL_HDM_DECODER##n##_TARGET_LIST_LO, 
>   \
>  CXL_HDM_REGISTERS_OFFSET + (0x20 * n) + 0x24)
>   \
>REG32(CXL_HDM_DECODER##n##_TARGET_LIST_HI, 
>   \
> +CXL_HDM_REGISTERS_OFFSET + (0x20 * n) + 0x28)
>   \
> +  REG32(CXL_HDM_DECODER##n##_DPA_SKIP_LO,
>   \
> +CXL_HDM_REGISTERS_OFFSET + (0x20 * n) + 0x24)
>   \
> +  REG32(CXL_HDM_DECODER##n##_DPA_SKIP_HI,
>   \
>  CXL_HDM_REGISTERS_OFFSET + (0x20 * n) + 0x28)
>  
>  REG32(CXL_HDM_DECODER_CAPABILITY, CXL_HDM_REGISTERS_OFFSET)
> @@ -147,9 +151,13 @@ REG32(CXL_HDM_DECODER_GLOBAL_CONTROL, 
> CXL_HDM_REGISTERS_OFFSET + 4)
>  FIELD(CXL_HDM_DECODER_GLOBAL_CONTROL, POISON_ON_ERR_EN, 0, 1)
>  FIELD(CXL_HDM_DECODER_GLOBAL_CONTROL, HDM_DECODER_ENABLE, 1, 1)
>  
> +/* Support 4 decoders at all levels of topology */
> +#define CXL_HDM_DECODER_COUNT 4
> +
>  HDM_DECODER_INIT(0);
> -/* Only used for HDM decoder registers block address increment */
>  HDM_DECODER_INIT(1);
> +HDM_DECODER_INIT(2);
> +HDM_DECODER_INIT(3);
>  
>  /* 8.2.5.13 - CXL Extended Security Capability Structure (Root complex only) 
> */
>  #define EXTSEC_ENTRY_MAX256
> diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c
> index aa011a8f34..3ecdad4a5e 100644
> --- a/hw/cxl/cxl-component-utils.c
> +++ b/hw/cxl/cxl-component-utils.c
> @@ -90,6 +90,9 @@ static void dumb_hdm_handler(CXLComponentState *cxl_cstate, 
> hwaddr offset,
>  
>  switch (offset) {
>  case A_CXL_HDM_DECODER0_CTRL:
> +case A_CXL_HDM_DECODER1_CTRL:
> +case A_CXL_HDM_DECODER2_CTRL:
> +case A_CXL_HDM_DECODER3_CTRL:
>  should_commit = FIELD_EX32(value, CXL_HDM_DECODER0_CTRL, COMMIT);
>  should_uncommit = !should_commit;

So for the commit/uncommit flag, we always check decoder 0 control
register? Or i read it wrong? I thought the commit bit is per control register
thing?

Fan

>  break;
> @@ -129,7 +132,7 @@ static void cxl_cache_mem_write_reg(void *opaque, hwaddr 
> offset, uint64_t value,
>  }
>  
>  if (offset >= A_CXL_HDM_DECODER_CAPABILITY &&
> -offset <= A_CXL_HDM_DECODER0_TARGET_LIST_HI) {
> +offset <= A_CXL_HDM_DECODER3_TARGET_LIST_HI) {
>  dumb_hdm_handler(cxl_cstate, offset, value);
>  } else {
>  cregs->cache_mem_registers[offset / 
> sizeof(*cregs->cache_mem_registers)] = value;
> @@ -209,7 +212,7 @@ static void ras_init_common(uint32_t *reg_state, uint32_t 
> *write_msk)
>  static void hdm_init_common(uint32_t *reg_state, uint32_t *write_msk,
>  enum reg_type type)
>  {
> -int decoder_count = 1;
> +int decoder_count = CXL_HDM_DECODER_COUNT;
>  int hdm_inc = R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_LO;
>  int i;
>  
> diff --git a/hw/cxl/cxl-host.c b/hw/cxl/cxl-host.c
> index 73c5426476..2aa776c79c 100644
> --- a/hw/cxl/cxl-host.c
> +++ b/hw/cxl/cxl-host.c
> @@ -97,35 +97,58 @@ void cxl_fmws_link_targets(CXLState *cxl_state, Error 
> **errp)
>  }
>  }
>  
> -/* TODO: support, multiple hdm decoders */
>  static bool cxl_hdm_find_target(uint32_t *cache_mem, hwaddr addr,
>  uint8_t *target)
>  {
>  int hdm_inc = R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_LO;
> -uint32_t ctrl;
> -uint32_t ig_enc;
> -uint32_t iw_enc;
> -uint32_t target_idx;
> -int i = 0;
> -
> -ctrl = cache_mem[R_CXL_HDM_DECODER0_CTRL + i * hdm_inc];
> -if (!FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED)) {
> -return false;
> -}
> -
> -ig_enc = FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, IG);
> -iw_enc = FIELD_EX32(ctrl, CXL_HDM_DECODER0_CTRL, IW);
> -target_idx = (addr / cxl_decode_ig(ig_enc)) % (1 

Re: [PATCH v3 3/4] hw/cxl: Fix and use same calculation for HDM decoder block size everywhere

[Fan Ni]

On Mon, Sep 11, 2023 at 12:43:12PM +0100, Jonathan Cameron wrote:

> In order to avoid having the size of the per HDM decoder register block
> repeated in lots of places, create the register definitions for HDM
> decoder 1 and use the offset between the first registers in HDM decoder 0 and
> HDM decoder 1 to establish the offset.
> 
> Calculate in each function as this is more obvious and leads to shorter
> line lengths than a single #define which would need a long name
> to be specific enough.
> 
> Note that the code currently only supports one decoder, so the bugs this
> fixes don't actually affect anything. Previously the offset didn't
> take into account that the write_msk etc are 4 byte fields.
> 
> Signed-off-by: Jonathan Cameron 
> 

Reviewed-by: Fan Ni 

> --
> v3:
> New patch to separate this out from the addition of HDM decoders.
> ---
>  include/hw/cxl/cxl_component.h |  2 ++
>  hw/cxl/cxl-component-utils.c   | 19 +++
>  hw/cxl/cxl-host.c  |  4 +++-
>  hw/mem/cxl_type3.c | 24 +++-
>  4 files changed, 31 insertions(+), 18 deletions(-)
> 
> diff --git a/include/hw/cxl/cxl_component.h b/include/hw/cxl/cxl_component.h
> index ef9e033919..7c864d2044 100644
> --- a/include/hw/cxl/cxl_component.h
> +++ b/include/hw/cxl/cxl_component.h
> @@ -148,6 +148,8 @@ REG32(CXL_HDM_DECODER_GLOBAL_CONTROL, 
> CXL_HDM_REGISTERS_OFFSET + 4)
>  FIELD(CXL_HDM_DECODER_GLOBAL_CONTROL, HDM_DECODER_ENABLE, 1, 1)
>  
>  HDM_DECODER_INIT(0);
> +/* Only used for HDM decoder registers block address increment */
> +HDM_DECODER_INIT(1);
>  
>  /* 8.2.5.13 - CXL Extended Security Capability Structure (Root complex only) 
> */
>  #define EXTSEC_ENTRY_MAX256
> diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c
> index 352d0dace2..aa011a8f34 100644
> --- a/hw/cxl/cxl-component-utils.c
> +++ b/hw/cxl/cxl-component-utils.c
> @@ -210,6 +210,7 @@ static void hdm_init_common(uint32_t *reg_state, uint32_t 
> *write_msk,
>  enum reg_type type)
>  {
>  int decoder_count = 1;
> +int hdm_inc = R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_LO;
>  int i;
>  
>  ARRAY_FIELD_DP32(reg_state, CXL_HDM_DECODER_CAPABILITY, DECODER_COUNT,
> @@ -222,19 +223,21 @@ static void hdm_init_common(uint32_t *reg_state, 
> uint32_t *write_msk,
>   HDM_DECODER_ENABLE, 0);
>  write_msk[R_CXL_HDM_DECODER_GLOBAL_CONTROL] = 0x3;
>  for (i = 0; i < decoder_count; i++) {
> -write_msk[R_CXL_HDM_DECODER0_BASE_LO + i * 0x20] = 0xf000;
> -write_msk[R_CXL_HDM_DECODER0_BASE_HI + i * 0x20] = 0x;
> -write_msk[R_CXL_HDM_DECODER0_SIZE_LO + i * 0x20] = 0xf000;
> -write_msk[R_CXL_HDM_DECODER0_SIZE_HI + i * 0x20] = 0x;
> -write_msk[R_CXL_HDM_DECODER0_CTRL + i * 0x20] = 0x13ff;
> +write_msk[R_CXL_HDM_DECODER0_BASE_LO + i * hdm_inc] = 0xf000;
> +write_msk[R_CXL_HDM_DECODER0_BASE_HI + i * hdm_inc] = 0x;
> +write_msk[R_CXL_HDM_DECODER0_SIZE_LO + i * hdm_inc] = 0xf000;
> +write_msk[R_CXL_HDM_DECODER0_SIZE_HI + i * hdm_inc] = 0x;
> +write_msk[R_CXL_HDM_DECODER0_CTRL + i * hdm_inc] = 0x13ff;
>  if (type == CXL2_DEVICE ||
>  type == CXL2_TYPE3_DEVICE ||
>  type == CXL2_LOGICAL_DEVICE) {
> -write_msk[R_CXL_HDM_DECODER0_TARGET_LIST_LO + i * 0x20] = 
> 0xf000;
> +write_msk[R_CXL_HDM_DECODER0_TARGET_LIST_LO + i * hdm_inc] =
> +0xf000;
>  } else {
> -write_msk[R_CXL_HDM_DECODER0_TARGET_LIST_LO + i * 0x20] = 
> 0x;
> +write_msk[R_CXL_HDM_DECODER0_TARGET_LIST_LO + i * hdm_inc] =
> +0x;
>  }
> -write_msk[R_CXL_HDM_DECODER0_TARGET_LIST_HI + i * 0x20] = 0x;
> +write_msk[R_CXL_HDM_DECODER0_TARGET_LIST_HI + i * hdm_inc] = 
> 0x;
>  }
>  }
>  
> diff --git a/hw/cxl/cxl-host.c b/hw/cxl/cxl-host.c
> index f0920da956..73c5426476 100644
> --- a/hw/cxl/cxl-host.c
> +++ b/hw/cxl/cxl-host.c
> @@ -101,12 +101,14 @@ void cxl_fmws_link_targets(CXLState *cxl_state, Error 
> **errp)
>  static bool cxl_hdm_find_target(uint32_t *cache_mem, hwaddr addr,
>  uint8_t *target)
>  {
> +int hdm_inc = R_CXL_HDM_DECODER1_BASE_LO - R_CXL_HDM_DECODER0_BASE_LO;
>  uint32_t ctrl;
>  uint32_t ig_enc;
>  uint32_t iw_enc;
>  uint32_t target_idx;
> +int i = 0;
>  
> -ctrl = cache_mem[R_CXL_HDM_DECODER0_CTRL];
> +ctrl = cache_mem[R_CXL_HDM_DECODER0_CTRL + i * hdm_i

Re: [PATCH v3 2/4] hw/cxl: Add utility functions decoder interleave ways and target count.

[Fan Ni]

On Mon, Sep 11, 2023 at 12:43:11PM +0100, Jonathan Cameron wrote:

> As an encoded version of these key configuration parameters is available
> in a register, provide functions to extract it again so as to avoid
> the need for duplicating the storage.
> 
> Whilst here update the _enc() function to include additional values
> as defined in the CXL 3.0 specification. Whilst they are not
> currently used in the emulation, they may be in future and it is
> easier to compare with the specification if all values are covered.
> 
> Reviewed-by: Philippe Mathieu-Daudé 
> Signed-off-by: Jonathan Cameron 
> ---

LGTM. Only one minor comment inline.

Reviewed-by: Fan Ni 


> v3: No changes, picked up tags.
> v2: Thanks to Philippe Mathieu-Daudé
>  - Expand both enc() and dec() functions to include full set of values
>defined in CXL r3.0
>  - Pushed implementation down into the .c file.
> ---
>  include/hw/cxl/cxl_component.h |  2 ++
>  hw/cxl/cxl-component-utils.c   | 59 ++
>  2 files changed, 55 insertions(+), 6 deletions(-)
> 
> diff --git a/include/hw/cxl/cxl_component.h b/include/hw/cxl/cxl_component.h
> index bdb3881a6b..ef9e033919 100644
> --- a/include/hw/cxl/cxl_component.h
> +++ b/include/hw/cxl/cxl_component.h
> @@ -226,8 +226,10 @@ void cxl_component_create_dvsec(CXLComponentState 
> *cxl_cstate,
>  uint16_t type, uint8_t rev, uint8_t *body);
>  
>  int cxl_decoder_count_enc(int count);
> +int cxl_decoder_count_dec(int enc_cnt);
>  
>  uint8_t cxl_interleave_ways_enc(int iw, Error **errp);
> +int cxl_interleave_ways_dec(uint8_t iw_enc, Error **errp);
>  uint8_t cxl_interleave_granularity_enc(uint64_t gran, Error **errp);
>  
>  hwaddr cxl_decode_ig(int ig);
> diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c
> index ea2d4770ec..352d0dace2 100644
> --- a/hw/cxl/cxl-component-utils.c
> +++ b/hw/cxl/cxl-component-utils.c
> @@ -13,15 +13,45 @@
>  #include "hw/pci/pci.h"
>  #include "hw/cxl/cxl.h"
>  
> +/* CXL r3.0 Section 8.2.4.19.1 CXL HDM Decoder Capability Register */
>  int cxl_decoder_count_enc(int count)
>  {
>  switch (count) {
> -case 1: return 0;
> -case 2: return 1;
> -case 4: return 2;
> -case 6: return 3;
> -case 8: return 4;
> -case 10: return 5;
> +case 1: return 0x0;
> +case 2: return 0x1;
> +case 4: return 0x2;
> +case 6: return 0x3;
> +case 8: return 0x4;
> +case 10: return 0x5;
> +/* Switches and Host Bridges may have more than 10 decoders */
> +case 12: return 0x6;
> +case 14: return 0x7;
> +case 16: return 0x8;
> +case 20: return 0x9;
> +case 24: return 0xa;
> +case 28: return 0xb;
> +case 32: return 0xc;
> +}
> +return 0;
> +}
> +
> +int cxl_decoder_count_dec(int enc_cnt)
> +{
> +switch (enc_cnt) {
> +case 0x0: return 1;
> +case 0x1: return 2;
> +case 0x2: return 4;
> +case 0x3: return 6;
> +case 0x4: return 8;
> +case 0x5: return 10;
> +/* Switches and Host Bridges may have more than 10 decoders */
> +case 0x6: return 12;
> +case 0x7: return 14;
> +case 0x8: return 16;
> +case 0x9: return 20;
> +case 0xa: return 24;
> +case 0xb: return 28;
> +case 0xc: return 32;
>  }
>  return 0;
>  }
> @@ -410,6 +440,23 @@ uint8_t cxl_interleave_ways_enc(int iw, Error **errp)
>  }
>  }
>  

Similar as decoder count dec/enc, maybe we want to add a line of comment below.
/* CXL r3.0 Section 8.2.4.19.7 CXL HDM Decoder n Control Register */

Fan
> +int cxl_interleave_ways_dec(uint8_t iw_enc, Error **errp)
> +{
> +switch (iw_enc) {
> +case 0x0: return 1;
> +case 0x1: return 2;
> +case 0x2: return 4;
> +case 0x3: return 8;
> +case 0x4: return 16;
> +case 0x8: return 3;
> +case 0x9: return 6;
> +case 0xa: return 12;
> +default:
> +error_setg(errp, "Encoded interleave ways: %d not supported", 
> iw_enc);
> +return 0;
> +}
> +}
> +
>  uint8_t cxl_interleave_granularity_enc(uint64_t gran, Error **errp)
>  {
>  switch (gran) {
> -- 
> 2.39.2
> 
> 

Re: [PATCH v3 1/4] hw/cxl: Push cxl_decoder_count_enc() and cxl_decode_ig() into .c

[Fan Ni]

On Mon, Sep 11, 2023 at 12:43:10PM +0100, Jonathan Cameron wrote:

> There is no strong justification for keeping these in the header
> so push them down into the associated cxl-component-utils.c file.
> 
> Suggested-by: Philippe Mathieu-Daudé 
> Reviewed-by: Philippe Mathieu-Daudé 
> Signed-off-by: Jonathan Cameron 
> ---

Reviewed-by: Fan Ni 

>  include/hw/cxl/cxl_component.h | 18 ++
>  hw/cxl/cxl-component-utils.c   | 18 ++
>  2 files changed, 20 insertions(+), 16 deletions(-)
> 
> diff --git a/include/hw/cxl/cxl_component.h b/include/hw/cxl/cxl_component.h
> index 42c7e581a7..bdb3881a6b 100644
> --- a/include/hw/cxl/cxl_component.h
> +++ b/include/hw/cxl/cxl_component.h
> @@ -225,26 +225,12 @@ void cxl_component_create_dvsec(CXLComponentState 
> *cxl_cstate,
>  enum reg_type cxl_dev_type, uint16_t length,
>  uint16_t type, uint8_t rev, uint8_t *body);
>  
> -static inline int cxl_decoder_count_enc(int count)
> -{
> -switch (count) {
> -case 1: return 0;
> -case 2: return 1;
> -case 4: return 2;
> -case 6: return 3;
> -case 8: return 4;
> -case 10: return 5;
> -}
> -return 0;
> -}
> +int cxl_decoder_count_enc(int count);
>  
>  uint8_t cxl_interleave_ways_enc(int iw, Error **errp);
>  uint8_t cxl_interleave_granularity_enc(uint64_t gran, Error **errp);
>  
> -static inline hwaddr cxl_decode_ig(int ig)
> -{
> -return 1ULL << (ig + 8);
> -}
> +hwaddr cxl_decode_ig(int ig);
>  
>  CXLComponentState *cxl_get_hb_cstate(PCIHostState *hb);
>  bool cxl_get_hb_passthrough(PCIHostState *hb);
> diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c
> index 378f1082ce..ea2d4770ec 100644
> --- a/hw/cxl/cxl-component-utils.c
> +++ b/hw/cxl/cxl-component-utils.c
> @@ -13,6 +13,24 @@
>  #include "hw/pci/pci.h"
>  #include "hw/cxl/cxl.h"
>  
> +int cxl_decoder_count_enc(int count)
> +{
> +switch (count) {
> +case 1: return 0;
> +case 2: return 1;
> +case 4: return 2;
> +case 6: return 3;
> +case 8: return 4;
> +case 10: return 5;
> +}
> +return 0;
> +}
> +
> +hwaddr cxl_decode_ig(int ig)
> +{
> +return 1ULL << (ig + 8);
> +}
> +
>  static uint64_t cxl_cache_mem_read_reg(void *opaque, hwaddr offset,
> unsigned size)
>  {
> -- 
> 2.39.2
> 
> 

Re: [Qemu PATCH v2 7/9] hw/cxl/cxl-mailbox-utils: Add mailbox commands to support add/release dynamic capacity response

[Fan Ni]

On Fri, Sep 08, 2023 at 01:00:16PM +, J?rgen Hansen wrote:
> On 7/25/23 20:39, Fan Ni wrote:
> > From: Fan Ni 
> >
> > Per CXL spec 3.0, two mailbox commands are implemented:
> > Add Dynamic Capacity Response (Opcode 4802h) 8.2.9.8.9.3, and
> > Release Dynamic Capacity (Opcode 4803h) 8.2.9.8.9.4.
> >
> > Signed-off-by: Fan Ni 
> > ---
> >   hw/cxl/cxl-mailbox-utils.c  | 253 
> >   include/hw/cxl/cxl_device.h |   3 +-
> >   2 files changed, 255 insertions(+), 1 deletion(-)
> >
> > diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
> > index 3d25a9697e..1e4944da95 100644
> > --- a/hw/cxl/cxl-mailbox-utils.c
> > +++ b/hw/cxl/cxl-mailbox-utils.c
> > @@ -84,6 +84,8 @@ enum {
> >   DCD_CONFIG  = 0x48, /*r3.0: 8.2.9.8.9*/
> >   #define GET_DC_CONFIG  0x0
> >   #define GET_DYN_CAP_EXT_LIST   0x1
> > +#define ADD_DYN_CAP_RSP0x2
> > +#define RELEASE_DYN_CAP0x3
> >   PHYSICAL_SWITCH = 0x51
> >   #define IDENTIFY_SWITCH_DEVICE  0x0
> >   };
> > @@ -1086,6 +1088,251 @@ static CXLRetCode 
> > cmd_dcd_get_dyn_cap_ext_list(struct cxl_cmd *cmd,
> >   return CXL_MBOX_SUCCESS;
> >   }
> >
> > +/*
> > + * Check whether the bits at addr between [nr, nr+size) are all set,
> > + * return 1 if all 1s, else return 0
> > + */
> > +static inline int test_bits(const unsigned long *addr, int nr, int size)
> > +{
> > +unsigned long res = find_next_zero_bit(addr, size + nr, nr);
> > +
> > +return (res >= nr + size) ? 1 : 0;
> > +}
> > +
> > +/*
> > + * Find dynamic capacity region id based on dpa range [dpa, dpa+len)
> > + */
> > +static uint8_t find_region_id(struct CXLType3Dev *dev, uint64_t dpa,
> > +uint64_t len)
> > +{
> > +int8_t i = dev->dc.num_regions - 1;
> > +
> > +while (i > 0 && dpa < dev->dc.regions[i].base) {
> > +i--;
> > +}
> > +
> > +if (dpa < dev->dc.regions[i].base
> > +|| dpa + len > dev->dc.regions[i].base + 
> > dev->dc.regions[i].len) {
> > +return dev->dc.num_regions;
> > +}
> > +
> > +return i;
> > +}
> > +
> > +static void insert_extent_to_extent_list(CXLDCDExtentList *list, uint64_t 
> > dpa,
> > +uint64_t len, uint8_t *tag, uint16_t shared_seq)
> > +{
> > +CXLDCD_Extent *extent;
> > +extent = g_new0(CXLDCD_Extent, 1);
> > +extent->start_dpa = dpa;
> > +extent->len = len;
> > +if (tag) {
> > +memcpy(extent->tag, tag, 0x10);
> > +} else {
> > +memset(extent->tag, 0, 0x10);
> > +}
> > +extent->shared_seq = shared_seq;
> > +
> > +QTAILQ_INSERT_TAIL(list, extent, node);
> > +}
> > +
> > +typedef struct updated_dc_extent_list_in_pl {
> > +uint32_t num_entries_updated;
> > +uint8_t rsvd[4];
> > +struct { /* r3.0: Table 8-130 */
> > +uint64_t start_dpa;
> > +uint64_t len;
> > +uint8_t rsvd[8];
> > +} QEMU_PACKED updated_entries[];
> > +} QEMU_PACKED updated_dc_extent_list_in_pl;
> > +
> > +/*
> > + * The function only check the input extent list against itself.
> > + */
> > +static CXLRetCode detect_malformed_extent_list(CXLType3Dev *dev,
> > +const updated_dc_extent_list_in_pl *in)
> > +{
> > +unsigned long *blk_bitmap;
> > +uint64_t min_block_size = dev->dc.regions[0].block_size;
> > +struct CXLDCD_Region *region = >dc.regions[0];
> > +uint32_t i;
> > +uint64_t dpa, len;
> > +uint8_t rid;
> > +CXLRetCode ret;
> > +
> > +for (i = 1; i < dev->dc.num_regions; i++) {
> > +region = >dc.regions[i];
> > +if (min_block_size > region->block_size) {
> > +min_block_size = region->block_size;
> > +}
> > +}
> > +
> > +blk_bitmap = bitmap_new((region->len + region->base
> > +- dev->dc.regions[0].base) / min_block_size);
> > +
> > +for (i = 0; i < in->num_entries_updated; i++) {
> > +dpa = in->updated_entries[i].start_dpa;
> > +len = in->updated_entries[i].len;
> > +
> > +rid = find_region_id(dev, dpa, len);
> > +if (rid == dev->dc.num_regions) {
> > +  

Re: [Qemu PATCH v2 6/9] hw/mem/cxl_type3: Add DC extent list representative and get DC extent list mailbox support

[Fan Ni]

On Fri, Sep 08, 2023 at 01:12:45PM +, J?rgen Hansen wrote:
> On 7/25/23 20:39, Fan Ni wrote:
> > From: Fan Ni 
> >
> > Add dynamic capacity extent list representative to the definition of
> > CXLType3Dev and add get DC extent list mailbox command per
> > CXL.spec.3.0:.8.2.9.8.9.2.
> >
> > Signed-off-by: Fan Ni 
> > ---
> >   hw/cxl/cxl-mailbox-utils.c  | 71 +
> >   hw/mem/cxl_type3.c  |  1 +
> >   include/hw/cxl/cxl_device.h | 23 
> >   3 files changed, 95 insertions(+)
> >
> > diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
> > index 0511b8e6f7..3d25a9697e 100644
> > --- a/hw/cxl/cxl-mailbox-utils.c
> > +++ b/hw/cxl/cxl-mailbox-utils.c
> > @@ -83,6 +83,7 @@ enum {
> >   #define CLEAR_POISON   0x2
> >   DCD_CONFIG  = 0x48, /*r3.0: 8.2.9.8.9*/
> >   #define GET_DC_CONFIG  0x0
> > +#define GET_DYN_CAP_EXT_LIST   0x1
> >   PHYSICAL_SWITCH = 0x51
> >   #define IDENTIFY_SWITCH_DEVICE  0x0
> >   };
> > @@ -1018,6 +1019,73 @@ static CXLRetCode cmd_dcd_get_dyn_cap_config(struct 
> > cxl_cmd *cmd,
> >   return CXL_MBOX_SUCCESS;
> >   }
> >
> > +/*
> > + * cxl spec 3.0: 8.2.9.8.9.2
> > + * Get Dynamic Capacity Extent List (Opcode 4810h)
> > + */
> > +static CXLRetCode cmd_dcd_get_dyn_cap_ext_list(struct cxl_cmd *cmd,
> > +CXLDeviceState *cxl_dstate,
> > +uint16_t *len)
> > +{
> > +struct get_dyn_cap_ext_list_in_pl {
> > +uint32_t extent_cnt;
> > +uint32_t start_extent_id;
> > +} QEMU_PACKED;
> > +
> > +struct get_dyn_cap_ext_list_out_pl {
> > +uint32_t count;
> > +uint32_t total_extents;
> > +uint32_t generation_num;
> > +uint8_t rsvd[4];
> > +CXLDCExtent_raw records[];
> > +} QEMU_PACKED;
> > +
> > +struct get_dyn_cap_ext_list_in_pl *in = (void *)cmd->payload;
> > +struct get_dyn_cap_ext_list_out_pl *out = (void *)cmd->payload;
> > +struct CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev,
> > +cxl_dstate);
> > +uint16_t record_count = 0, i = 0, record_done = 0;
> > +CXLDCDExtentList *extent_list = >dc.extents;
> > +CXLDCD_Extent *ent;
> > +uint16_t out_pl_len;
> > +uint32_t start_extent_id = in->start_extent_id;
> > +
> > +if (start_extent_id > ct3d->dc.total_extent_count) {
> > +return CXL_MBOX_INVALID_INPUT;
> > +}
> > +
> > +record_count = MIN(in->extent_cnt,
> > +ct3d->dc.total_extent_count - start_extent_id);
> > +
> > +out_pl_len = sizeof(*out) + record_count * sizeof(out->records[0]);
> > +/* May need more processing here in the future */
> > +assert(out_pl_len <= CXL_MAILBOX_MAX_PAYLOAD_SIZE);
> > +
> > +memset(out, 0, out_pl_len);
> > +stl_le_p(>count, record_count);
> > +stl_le_p(>total_extents, ct3d->dc.total_extent_count);
> > +stl_le_p(>generation_num, ct3d->dc.ext_list_gen_seq);
> > +
> > +if (record_count > 0) {
> > +QTAILQ_FOREACH(ent, extent_list, node) {
> > +if (i++ < start_extent_id) {
> > +continue;
> > +}
> > +stq_le_p(>records[record_done].start_dpa, ent->start_dpa);
> > +stq_le_p(>records[record_done].len, ent->len);
> > +memcpy(>records[record_done].tag, ent->tag, 0x10);
> > +stw_le_p(>records[record_done].shared_seq, 
> > ent->shared_seq);
> > +record_done++;
> > +if (record_done == record_count) {
> > +break;
> > +}
> > +}
> > +}
> > +
> > +*len = out_pl_len;
> > +return CXL_MBOX_SUCCESS;
> > +}
> > +
> >   #define IMMEDIATE_CONFIG_CHANGE (1 << 1)
> >   #define IMMEDIATE_DATA_CHANGE (1 << 2)
> >   #define IMMEDIATE_POLICY_CHANGE (1 << 3)
> > @@ -1058,6 +1126,9 @@ static struct cxl_cmd cxl_cmd_set[256][256] = {
> >   cmd_media_clear_poison, 72, 0 },
> >   [DCD_CONFIG][GET_DC_CONFIG] = { "DCD_GET_DC_CONFIG",
> >   cmd_dcd_get_dyn_cap_config, 2, 0 },
> > +[DCD_CONFIG][GET_DYN_CAP_EXT_LIST] = {
> > +"DCD_GET_DYNAMIC_CAPACITY_EXTENT_LIST", 
> > cmd_dcd_get_dyn_cap_ext_list,
> > +8, 0 },

Re: [Qemu PATCH v2 9/9] hw/mem/cxl_type3: Add dpa range validation for accesses to dc regions

[Fan Ni]

On Fri, Aug 25, 2023 at 12:42:56PM +0100, Jonathan Cameron wrote:
> On Thu, 24 Aug 2023 13:49:00 -0700
> Fan Ni  wrote:
>
> > On Mon, Aug 07, 2023 at 09:53:42AM +0100, Jonathan Cameron wrote:
> > > On Tue, 25 Jul 2023 18:39:56 +
> > > Fan Ni  wrote:
> > >
> > > > From: Fan Ni 
> > > >
> > > > Not all dpa range in the dc regions is valid to access until an extent
> > > > covering the range has been added. Add a bitmap for each region to
> > > > record whether a dc block in the region has been backed by dc extent.
> > > > For the bitmap, a bit in the bitmap represents a dc block. When a dc
> > > > extent is added, all the bits of the blocks in the extent will be set,
> > > > which will be cleared when the extent is released.
> > > >
> > > > Signed-off-by: Fan Ni 
> > > Hi Fan,
> > >
> > > A few of the bits of feedback apply broadly across the series.  Given I'm
> > > rebasing this anyway to give myself something to test I'll tidy things up
> > > (feel free to disagree with and revert any changes !)
> > > and push a tree out in next day or two.  I'll message when I've done so.
> > >
> > > Jonathan
> >
> > Hi Jonathan,
> > I tried DCD with your branch "cxl-2023-08-07", and noticed the
> > following,
> > 1. You made some changes to the bitmap functionality, now it is only
> > used to validate extents when adding/releasing dc extents. My original
> > thought of adding the bitmap is to 1) validating extents for extent
> > add/release as you do; 2) Add validating when doing read/write to the dc
> > regions since some address region may not have valid extent added yet.
> > Do you think 2) is not necessary?
>
> Change wasn't intentional. I probably just messed up the rebase!

Just double checked the code. The logic is still there, but in another
patch in the series, so no issue and ignore my previous question.
Sorry for the confusion.

>
> >
> > 2. Your change introduced a bug in the code.
> > https://gitlab.com/jic23/qemu/-/blob/cxl-2023-08-07/hw/cxl/cxl-mailbox-utils.c?ref_type=heads#L1394
> > ct3d->dc.num_regions should be ct3d->dc.num_regions-1.
> Thanks.  Given I might forget about about it, if you want to incorporate that 
> in
> your next version that would be great. I might remember to fix it in the 
> meantime!
>
> Jonathan
>

My code does not have this. It seems you added the lastregion variable
to record the last region, while I use the following logic to iterate
the regions and record last region automatically while collecting
min_block_size.

+for (i = 1; i < dev->dc.num_regions; i++) {
+region = >dc.regions[i];
+if (min_block_size > region->block_size) {
+min_block_size = region->block_size;
+}
+}
+
+blk_bitmap = bitmap_new((region->len + region->base
+- dev->dc.regions[0].base) / min_block_size);


Fan

> >
> > Thanks,
> > Fan
> >
> > >
> > > > ---
> > > >  hw/mem/cxl_type3.c  | 155 
> > > >  include/hw/cxl/cxl_device.h |   1 +
> > > >  2 files changed, 156 insertions(+)
> > > >
> > > > diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
> > > > index 41a828598a..51943a36fc 100644
> > > > --- a/hw/mem/cxl_type3.c
> > > > +++ b/hw/mem/cxl_type3.c
> > > > @@ -787,13 +787,37 @@ static int cxl_create_dc_regions(CXLType3Dev 
> > > > *ct3d)
> > > >  /* dsmad_handle is set when creating cdat table entries */
> > > >  region->flags = 0;
> > > >
> > > > +region->blk_bitmap = bitmap_new(region->len / 
> > > > region->block_size);
> > >
> > > In common with many allocators in qemu if this fails it calls abort()
> > > internally so no need to handle potential errors.
> > >
> > > > +if (!region->blk_bitmap) {
> > > > +break;
> > > > +}
> > > > +
> > > >  region_base += region->len;
> > > >  }
> > > > +
> > > > +if (i < ct3d->dc.num_regions) {
> > > > +while (--i >= 0) {
> > > > +g_free(ct3d->dc.regions[i].blk_bitmap);
> > > > +}
> > > > +return -1;
> > > > +}
> > > > +
> > > >  QTAILQ_INIT(>dc.extents);
> > > >
>

Re: [Qemu PATCH v2 9/9] hw/mem/cxl_type3: Add dpa range validation for accesses to dc regions

[Fan Ni]

On Mon, Aug 07, 2023 at 09:53:42AM +0100, Jonathan Cameron wrote:
> On Tue, 25 Jul 2023 18:39:56 +
> Fan Ni  wrote:
>
> > From: Fan Ni 
> >
> > Not all dpa range in the dc regions is valid to access until an extent
> > covering the range has been added. Add a bitmap for each region to
> > record whether a dc block in the region has been backed by dc extent.
> > For the bitmap, a bit in the bitmap represents a dc block. When a dc
> > extent is added, all the bits of the blocks in the extent will be set,
> > which will be cleared when the extent is released.
> >
> > Signed-off-by: Fan Ni 
> Hi Fan,
>
> A few of the bits of feedback apply broadly across the series.  Given I'm
> rebasing this anyway to give myself something to test I'll tidy things up
> (feel free to disagree with and revert any changes !)
> and push a tree out in next day or two.  I'll message when I've done so.
>
> Jonathan

Hi Jonathan,
I tried DCD with your branch "cxl-2023-08-07", and noticed the
following,
1. You made some changes to the bitmap functionality, now it is only
used to validate extents when adding/releasing dc extents. My original
thought of adding the bitmap is to 1) validating extents for extent
add/release as you do; 2) Add validating when doing read/write to the dc
regions since some address region may not have valid extent added yet.
Do you think 2) is not necessary?

2. Your change introduced a bug in the code.
https://gitlab.com/jic23/qemu/-/blob/cxl-2023-08-07/hw/cxl/cxl-mailbox-utils.c?ref_type=heads#L1394
ct3d->dc.num_regions should be ct3d->dc.num_regions-1.

Thanks,
Fan

>
> > ---
> >  hw/mem/cxl_type3.c  | 155 
> >  include/hw/cxl/cxl_device.h |   1 +
> >  2 files changed, 156 insertions(+)
> >
> > diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
> > index 41a828598a..51943a36fc 100644
> > --- a/hw/mem/cxl_type3.c
> > +++ b/hw/mem/cxl_type3.c
> > @@ -787,13 +787,37 @@ static int cxl_create_dc_regions(CXLType3Dev *ct3d)
> >  /* dsmad_handle is set when creating cdat table entries */
> >  region->flags = 0;
> >
> > +region->blk_bitmap = bitmap_new(region->len / region->block_size);
>
> In common with many allocators in qemu if this fails it calls abort()
> internally so no need to handle potential errors.
>
> > +if (!region->blk_bitmap) {
> > +break;
> > +}
> > +
> >  region_base += region->len;
> >  }
> > +
> > +if (i < ct3d->dc.num_regions) {
> > +while (--i >= 0) {
> > +g_free(ct3d->dc.regions[i].blk_bitmap);
> > +}
> > +return -1;
> > +}
> > +
> >  QTAILQ_INIT(>dc.extents);
> >
> >  return 0;
> >  }
> >
> > +static void cxl_destroy_dc_regions(CXLType3Dev *ct3d)
> > +{
> > +int i;
> > +struct CXLDCD_Region *region;
> > +
> > +for (i = 0; i < ct3d->dc.num_regions; i++) {
> > +region = >dc.regions[i];
> > +g_free(region->blk_bitmap);
> > +}
> > +}
> > +
> >  static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
> >  {
> >  DeviceState *ds = DEVICE(ct3d);
> > @@ -1021,6 +1045,7 @@ err_free_special_ops:
> >  g_free(regs->special_ops);
> >  err_address_space_free:
> >  if (ct3d->dc.host_dc) {
> > +cxl_destroy_dc_regions(ct3d);
> >  address_space_destroy(>dc.host_dc_as);
> >  }
> >  if (ct3d->hostpmem) {
> > @@ -1043,6 +1068,7 @@ static void ct3_exit(PCIDevice *pci_dev)
> >  spdm_sock_fini(ct3d->doe_spdm.socket);
> >  g_free(regs->special_ops);
> >  if (ct3d->dc.host_dc) {
> > +cxl_destroy_dc_regions(ct3d);
> >  address_space_destroy(>dc.host_dc_as);
> >  }
> >  if (ct3d->hostpmem) {
> > @@ -1053,6 +1079,110 @@ static void ct3_exit(PCIDevice *pci_dev)
> >  }
> >  }
> >
> > +/*
> > + * This function will marked the dpa range [dpa, dap + len) to be backed 
> > and
> > + * accessible, this happens when a dc extent is added and accepted by the
> > + * host.
> > + */
> > +static void set_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa,
> > +uint64_t len)
> > +{
> > +int i;
> > +CXLDCD_Region *region = >dc.regions[0];
> > +
> > +if (dpa < region->base
> > +|| dpa >= region->base + ct3d->dc.total_capacity)
>

Re: CXL volatile memory is not listed

[Fan Ni]

On Fri, Aug 11, 2023 at 07:52:25AM +0530, Maverickk 78 wrote:
> Thanks Fan,
>
> cxl create-region works like a charm :)
>
> Since this gets listed as "System Ram(kmem)", I guess the kernel
> treats it as regular memory and
> allocates it to the applications when needed?
> or is there an extra effort needed to make it available for
> applications on the host?
>

Yes. Once it is onlined, you can use it as regular memory.
CXL memory will serve as a zero-CPU memory-only NUMA node.
You can check it with numactl -H.

To use the cxl memory with an app, you can use
numactl --membind=numa_id app_name
#numa_id is the dedicated numa node where cxl memory sits.

One thing to notes, kvm will not work correctly with Qemu emulation when
you try to use cxl memory for an application, so do not enable kvm.

Fan

> On Thu, 10 Aug 2023 at 22:03, Fan Ni  wrote:
> >
> > On Wed, Aug 09, 2023 at 04:21:47AM +0530, Maverickk 78 wrote:
> > > Hello,
> > >
> > > I am running qemu-system-x86_64
> > >
> > > qemu-system-x86_64 --version
> > > QEMU emulator version 8.0.92 (v8.1.0-rc2-80-g0450cf0897)
> > >
> > > qemu-system-x86_64 \
> > > -m 2G,slots=4,maxmem=4G \
> > > -smp 4 \
> > > -machine type=q35,accel=kvm,cxl=on \
> > > -enable-kvm \
> > > -nographic \
> > > -device pxb-cxl,id=cxl.0,bus=pcie.0,bus_nr=52 \
> > > -device cxl-rp,id=rp0,bus=cxl.0,chassis=0,port=0,slot=0 \
> > > -object memory-backend-file,id=mem0,mem-path=/tmp/mem0,size=1G,share=true 
> > > \
> > > -device cxl-type3,bus=rp0,volatile-memdev=mem0,id=cxl-mem0 \
> > > -M cxl-fmw.0.targets.0=cxl.0,cxl-fmw.0.size=1G
> > >
> > >
> > > I was expecting the CXL memory to be listed in "System Ram", the lsmem
> > > shows only 2G memory which is System RAM, it's not listing the CXL
> > > memory.
> > >
> > > Do I need to pass any particular parameter in the kernel command line?
> > >
> > > Is there any documentation available? I followed the inputs provided in
> > >
> > > https://lore.kernel.org/linux-mm/y+csoehvlkudn...@kroah.com/T/
> > >
> > > Is there any documentation/blog listed?
> >
> > If I remember it correctly, for volatile cxl memory, we need to create a
> > region and then it will be discovered as system memory and shows up.
> >
> > Try to create a region with "cxl create-region".
> >
> > Fan
> > >

Re: CXL volatile memory is not listed

[Fan Ni]

On Wed, Aug 09, 2023 at 04:21:47AM +0530, Maverickk 78 wrote:
> Hello,
>
> I am running qemu-system-x86_64
>
> qemu-system-x86_64 --version
> QEMU emulator version 8.0.92 (v8.1.0-rc2-80-g0450cf0897)
>
> qemu-system-x86_64 \
> -m 2G,slots=4,maxmem=4G \
> -smp 4 \
> -machine type=q35,accel=kvm,cxl=on \
> -enable-kvm \
> -nographic \
> -device pxb-cxl,id=cxl.0,bus=pcie.0,bus_nr=52 \
> -device cxl-rp,id=rp0,bus=cxl.0,chassis=0,port=0,slot=0 \
> -object memory-backend-file,id=mem0,mem-path=/tmp/mem0,size=1G,share=true \
> -device cxl-type3,bus=rp0,volatile-memdev=mem0,id=cxl-mem0 \
> -M cxl-fmw.0.targets.0=cxl.0,cxl-fmw.0.size=1G
>
>
> I was expecting the CXL memory to be listed in "System Ram", the lsmem
> shows only 2G memory which is System RAM, it's not listing the CXL
> memory.
>
> Do I need to pass any particular parameter in the kernel command line?
>
> Is there any documentation available? I followed the inputs provided in
>
> https://lore.kernel.org/linux-mm/y+csoehvlkudn...@kroah.com/T/
>
> Is there any documentation/blog listed?

If I remember it correctly, for volatile cxl memory, we need to create a
region and then it will be discovered as system memory and shows up.

Try to create a region with "cxl create-region".

Fan
>

[Qemu PATCH v2 2/9] hw/cxl/cxl-mailbox-utils: Add dynamic capacity region representative and mailbox command support

[Fan Ni]

From: Fan Ni 

Per cxl spec 3.0, add dynamic capacity region representative based on
Table 8-126 and extend the cxl type3 device definition to include dc region
information. Also, based on info in 8.2.9.8.9.1, add 'Get Dynamic Capacity
Configuration' mailbox support.

Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c  | 72 +
 hw/mem/cxl_type3.c  |  6 
 include/hw/cxl/cxl_device.h | 17 +
 3 files changed, 95 insertions(+)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index b013e30314..0fe9f3eb5d 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -81,6 +81,8 @@ enum {
 #define GET_POISON_LIST0x0
 #define INJECT_POISON  0x1
 #define CLEAR_POISON   0x2
+DCD_CONFIG  = 0x48, /*r3.0: 8.2.9.8.9*/
+#define GET_DC_CONFIG  0x0
 PHYSICAL_SWITCH = 0x51
 #define IDENTIFY_SWITCH_DEVICE  0x0
 };
@@ -939,6 +941,71 @@ static CXLRetCode cmd_media_clear_poison(struct cxl_cmd 
*cmd,
 return CXL_MBOX_SUCCESS;
 }
 
+/*
+ * cxl spec 3.0: 8.2.9.8.9.1
+ * Get Dynamic Capacity Configuration
+ **/
+static CXLRetCode cmd_dcd_get_dyn_cap_config(struct cxl_cmd *cmd,
+CXLDeviceState *cxl_dstate,
+uint16_t *len)
+{
+struct get_dyn_cap_config_in_pl {
+uint8_t region_cnt;
+uint8_t start_region_id;
+} QEMU_PACKED;
+
+struct get_dyn_cap_config_out_pl {
+uint8_t num_regions;
+uint8_t rsvd1[7];
+struct {
+uint64_t base;
+uint64_t decode_len;
+uint64_t region_len;
+uint64_t block_size;
+uint32_t dsmadhandle;
+uint8_t flags;
+uint8_t rsvd2[3];
+} QEMU_PACKED records[];
+} QEMU_PACKED;
+
+struct get_dyn_cap_config_in_pl *in = (void *)cmd->payload;
+struct get_dyn_cap_config_out_pl *out = (void *)cmd->payload;
+struct CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev,
+cxl_dstate);
+uint16_t record_count = 0, i;
+uint16_t out_pl_len;
+uint8_t start_region_id = in->start_region_id;
+
+if (start_region_id >= ct3d->dc.num_regions) {
+return CXL_MBOX_INVALID_INPUT;
+}
+
+record_count = MIN(ct3d->dc.num_regions - in->start_region_id,
+in->region_cnt);
+
+out_pl_len = sizeof(*out) + record_count * sizeof(out->records[0]);
+assert(out_pl_len <= CXL_MAILBOX_MAX_PAYLOAD_SIZE);
+
+memset(out, 0, out_pl_len);
+out->num_regions = record_count;
+for (i = 0; i < record_count; i++) {
+stq_le_p(>records[i].base,
+ct3d->dc.regions[start_region_id + i].base);
+stq_le_p(>records[i].decode_len,
+ct3d->dc.regions[start_region_id + i].decode_len);
+stq_le_p(>records[i].region_len,
+ct3d->dc.regions[start_region_id + i].len);
+stq_le_p(>records[i].block_size,
+ct3d->dc.regions[start_region_id + i].block_size);
+stl_le_p(>records[i].dsmadhandle,
+ct3d->dc.regions[start_region_id + i].dsmadhandle);
+out->records[i].flags = ct3d->dc.regions[start_region_id + i].flags;
+}
+
+*len = out_pl_len;
+return CXL_MBOX_SUCCESS;
+}
+
 #define IMMEDIATE_CONFIG_CHANGE (1 << 1)
 #define IMMEDIATE_DATA_CHANGE (1 << 2)
 #define IMMEDIATE_POLICY_CHANGE (1 << 3)
@@ -977,6 +1044,8 @@ static struct cxl_cmd cxl_cmd_set[256][256] = {
 cmd_media_inject_poison, 8, 0 },
 [MEDIA_AND_POISON][CLEAR_POISON] = { "MEDIA_AND_POISON_CLEAR_POISON",
 cmd_media_clear_poison, 72, 0 },
+[DCD_CONFIG][GET_DC_CONFIG] = { "DCD_GET_DC_CONFIG",
+cmd_dcd_get_dyn_cap_config, 2, 0 },
 };
 
 static struct cxl_cmd cxl_cmd_set_sw[256][256] = {
@@ -1164,6 +1233,9 @@ void cxl_initialize_mailbox(CXLDeviceState *cxl_dstate, 
bool switch_cci)
 }
 for (int set = 0; set < 256; set++) {
 for (int cmd = 0; cmd < 256; cmd++) {
+if (!cxl_dstate->is_dcd && set == DCD_CONFIG) {
+continue;
+}
 if (cxl_dstate->cxl_cmd_set[set][cmd].handler) {
 struct cxl_cmd *c = _dstate->cxl_cmd_set[set][cmd];
 struct cel_log *log =
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 862107c5ef..4d68824dfe 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -1046,6 +1046,12 @@ static void ct3d_reset(DeviceState *dev)
 uint32_t *reg_state = ct3d->cxl_cstate.crb.cache_mem_registers;
 uint32_t *write_msk = ct3d->cxl_cstate.crb.cache_mem_regs_write_mask;
 
+if (ct3d->dc.num_regions) {
+ct3d->cxl_dstate.is_dcd = true;
+} else {
+ct3d->cxl_dstate.is_dcd = false;
+}
+
 cxl_component_register_init_common(reg_state, write_msk, 
CXL2_TYPE3_DE

[Qemu PATCH v2 9/9] hw/mem/cxl_type3: Add dpa range validation for accesses to dc regions

[Fan Ni]

From: Fan Ni 

Not all dpa range in the dc regions is valid to access until an extent
covering the range has been added. Add a bitmap for each region to
record whether a dc block in the region has been backed by dc extent.
For the bitmap, a bit in the bitmap represents a dc block. When a dc
extent is added, all the bits of the blocks in the extent will be set,
which will be cleared when the extent is released.

Signed-off-by: Fan Ni 
---
 hw/mem/cxl_type3.c  | 155 
 include/hw/cxl/cxl_device.h |   1 +
 2 files changed, 156 insertions(+)

diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 41a828598a..51943a36fc 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -787,13 +787,37 @@ static int cxl_create_dc_regions(CXLType3Dev *ct3d)
 /* dsmad_handle is set when creating cdat table entries */
 region->flags = 0;
 
+region->blk_bitmap = bitmap_new(region->len / region->block_size);
+if (!region->blk_bitmap) {
+break;
+}
+
 region_base += region->len;
 }
+
+if (i < ct3d->dc.num_regions) {
+while (--i >= 0) {
+g_free(ct3d->dc.regions[i].blk_bitmap);
+}
+return -1;
+}
+
 QTAILQ_INIT(>dc.extents);
 
 return 0;
 }
 
+static void cxl_destroy_dc_regions(CXLType3Dev *ct3d)
+{
+int i;
+struct CXLDCD_Region *region;
+
+for (i = 0; i < ct3d->dc.num_regions; i++) {
+region = >dc.regions[i];
+g_free(region->blk_bitmap);
+}
+}
+
 static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
 {
 DeviceState *ds = DEVICE(ct3d);
@@ -1021,6 +1045,7 @@ err_free_special_ops:
 g_free(regs->special_ops);
 err_address_space_free:
 if (ct3d->dc.host_dc) {
+cxl_destroy_dc_regions(ct3d);
 address_space_destroy(>dc.host_dc_as);
 }
 if (ct3d->hostpmem) {
@@ -1043,6 +1068,7 @@ static void ct3_exit(PCIDevice *pci_dev)
 spdm_sock_fini(ct3d->doe_spdm.socket);
 g_free(regs->special_ops);
 if (ct3d->dc.host_dc) {
+cxl_destroy_dc_regions(ct3d);
 address_space_destroy(>dc.host_dc_as);
 }
 if (ct3d->hostpmem) {
@@ -1053,6 +1079,110 @@ static void ct3_exit(PCIDevice *pci_dev)
 }
 }
 
+/*
+ * This function will marked the dpa range [dpa, dap + len) to be backed and
+ * accessible, this happens when a dc extent is added and accepted by the
+ * host.
+ */
+static void set_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa,
+uint64_t len)
+{
+int i;
+CXLDCD_Region *region = >dc.regions[0];
+
+if (dpa < region->base
+|| dpa >= region->base + ct3d->dc.total_capacity)
+return;
+
+/*
+ * spec 3.0 9.13.3: Regions are used in increasing-DPA order, with
+ * Region 0 being used for the lowest DPA of Dynamic Capacity and
+ * Region 7 for the highest DPA.
+ * So we check from the last region to find where the dpa belongs.
+ * access across multiple regions is not allowed.
+ **/
+for (i = ct3d->dc.num_regions - 1; i >= 0; i--) {
+region = >dc.regions[i];
+if (dpa >= region->base) {
+break;
+}
+}
+
+bitmap_set(region->blk_bitmap, (dpa - region->base) / region->block_size,
+len / region->block_size);
+}
+
+/*
+ * This function check whether a dpa range [dpa, dpa + len) has been backed
+ * with dc extents, used when validating read/write to dc regions
+ */
+static bool test_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa,
+uint64_t len)
+{
+int i;
+CXLDCD_Region *region = >dc.regions[0];
+uint64_t nbits;
+long nr;
+
+if (dpa < region->base
+|| dpa >= region->base + ct3d->dc.total_capacity)
+return false;
+
+/*
+ * spec 3.0 9.13.3: Regions are used in increasing-DPA order, with
+ * Region 0 being used for the lowest DPA of Dynamic Capacity and
+ * Region 7 for the highest DPA.
+ * So we check from the last region to find where the dpa belongs.
+ * access across multiple regions is not allowed.
+ */
+for (i = ct3d->dc.num_regions - 1; i >= 0; i--) {
+region = >dc.regions[i];
+if (dpa >= region->base) {
+break;
+}
+}
+
+nr = (dpa - region->base) / region->block_size;
+nbits = len / region->block_size;
+return find_next_zero_bit(region->blk_bitmap, nbits, nr) >= nr + nbits;
+}
+
+/*
+ * This function will marked the dpa range [dpa, dap + len) to be unbacked and
+ * inaccessible, this happens when a dc extent is added and accepted by the
+ * host.
+ */
+static void clear_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa,
+uint64_t len)
+{
+int i;
+CXLDCD_Region *region = >dc.regions[0];
+uint64_t nbits;
+long nr;
+
+if (dpa &l

[Qemu PATCH v2 3/9] include/hw/cxl/cxl_device: Rename mem_size as static_mem_size for type3 memory devices

[Fan Ni]

From: Fan Ni 

Rename mem_size as static_mem_size for type3 memdev to cover static RAM and
pmem capacity, preparing for the introduction of dynamic capacity to support
dynamic capacity devices.

Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c  | 5 +++--
 hw/mem/cxl_type3.c  | 8 
 include/hw/cxl/cxl_device.h | 2 +-
 3 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 0fe9f3eb5d..dd5ea95af8 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -540,7 +540,8 @@ static CXLRetCode cmd_identify_memory_device(struct cxl_cmd 
*cmd,
 
 snprintf(id->fw_revision, 0x10, "BWFW VERSION %02d", 0);
 
-stq_le_p(>total_capacity, cxl_dstate->mem_size / 
CXL_CAPACITY_MULTIPLIER);
+stq_le_p(>total_capacity,
+cxl_dstate->static_mem_size / CXL_CAPACITY_MULTIPLIER);
 stq_le_p(>persistent_capacity, cxl_dstate->pmem_size / 
CXL_CAPACITY_MULTIPLIER);
 stq_le_p(>volatile_capacity, cxl_dstate->vmem_size / 
CXL_CAPACITY_MULTIPLIER);
 stl_le_p(>lsa_size, cvc->get_lsa_size(ct3d));
@@ -879,7 +880,7 @@ static CXLRetCode cmd_media_clear_poison(struct cxl_cmd 
*cmd,
 struct clear_poison_pl *in = (void *)cmd->payload;
 
 dpa = ldq_le_p(>dpa);
-if (dpa + CXL_CACHE_LINE_SIZE > cxl_dstate->mem_size) {
+if (dpa + CXL_CACHE_LINE_SIZE > cxl_dstate->static_mem_size) {
 return CXL_MBOX_INVALID_PA;
 }
 
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 4d68824dfe..3d7acffcb7 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -748,7 +748,7 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error 
**errp)
 }
 address_space_init(>hostvmem_as, vmr, v_name);
 ct3d->cxl_dstate.vmem_size = memory_region_size(vmr);
-ct3d->cxl_dstate.mem_size += memory_region_size(vmr);
+ct3d->cxl_dstate.static_mem_size += memory_region_size(vmr);
 g_free(v_name);
 }
 
@@ -771,7 +771,7 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error 
**errp)
 }
 address_space_init(>hostpmem_as, pmr, p_name);
 ct3d->cxl_dstate.pmem_size = memory_region_size(pmr);
-ct3d->cxl_dstate.mem_size += memory_region_size(pmr);
+ct3d->cxl_dstate.static_mem_size += memory_region_size(pmr);
 g_free(p_name);
 }
 
@@ -984,7 +984,7 @@ static int cxl_type3_hpa_to_as_and_dpa(CXLType3Dev *ct3d,
 return -EINVAL;
 }
 
-if (*dpa_offset > ct3d->cxl_dstate.mem_size) {
+if (*dpa_offset > ct3d->cxl_dstate.static_mem_size) {
 return -EINVAL;
 }
 
@@ -1148,7 +1148,7 @@ static bool set_cacheline(CXLType3Dev *ct3d, uint64_t 
dpa_offset, uint8_t *data)
 return false;
 }
 
-if (dpa_offset + CXL_CACHE_LINE_SIZE > ct3d->cxl_dstate.mem_size) {
+if (dpa_offset + CXL_CACHE_LINE_SIZE > ct3d->cxl_dstate.static_mem_size) {
 return false;
 }
 
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index dae39da438..503c344326 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -209,7 +209,7 @@ typedef struct cxl_device_state {
 } timestamp;
 
 /* memory region size, HDM */
-uint64_t mem_size;
+uint64_t static_mem_size;
 uint64_t pmem_size;
 uint64_t vmem_size;
 bool is_dcd;
-- 
2.25.1

[Qemu PATCH v2 5/9] hw/mem/cxl_type3: Add host backend and address space handling for DC regions

[Fan Ni]

From: Fan Ni 

Add (file/memory backed) host backend, all the dynamic capacity regions
will share a single, large enough host backend. Set up address space for
DC regions to support read/write operations to dynamic capacity for DCD.

With the change, following supports are added:
1. add a new property to type3 device "nonvolatile-dc-memdev" to point to host
   memory backend for dynamic capacity;
2. add namespace for dynamic capacity for read/write support;
3. create cdat entries for each dynamic capacity region;
4. fix dvsec range registers to include DC regions.

Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c  |  19 +++-
 hw/mem/cxl_type3.c  | 203 +---
 include/hw/cxl/cxl_device.h |   4 +
 3 files changed, 185 insertions(+), 41 deletions(-)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index dd5ea95af8..0511b8e6f7 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -388,9 +388,11 @@ static CXLRetCode cmd_firmware_update_get_info(struct 
cxl_cmd *cmd,
 char fw_rev4[0x10];
 } QEMU_PACKED *fw_info;
 QEMU_BUILD_BUG_ON(sizeof(*fw_info) != 0x50);
+CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
 
 if ((cxl_dstate->vmem_size < CXL_CAPACITY_MULTIPLIER) ||
-(cxl_dstate->pmem_size < CXL_CAPACITY_MULTIPLIER)) {
+(cxl_dstate->pmem_size < CXL_CAPACITY_MULTIPLIER) ||
+(ct3d->dc.total_capacity < CXL_CAPACITY_MULTIPLIER)) {
 return CXL_MBOX_INTERNAL_ERROR;
 }
 
@@ -531,7 +533,8 @@ static CXLRetCode cmd_identify_memory_device(struct cxl_cmd 
*cmd,
 CXLType3Class *cvc = CXL_TYPE3_GET_CLASS(ct3d);
 
 if ((!QEMU_IS_ALIGNED(cxl_dstate->vmem_size, CXL_CAPACITY_MULTIPLIER)) ||
-(!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER))) {
+(!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER)) ||
+(!QEMU_IS_ALIGNED(ct3d->dc.total_capacity, CXL_CAPACITY_MULTIPLIER))) {
 return CXL_MBOX_INTERNAL_ERROR;
 }
 
@@ -566,9 +569,11 @@ static CXLRetCode cmd_ccls_get_partition_info(struct 
cxl_cmd *cmd,
 uint64_t next_pmem;
 } QEMU_PACKED *part_info = (void *)cmd->payload;
 QEMU_BUILD_BUG_ON(sizeof(*part_info) != 0x20);
+CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
 
 if ((!QEMU_IS_ALIGNED(cxl_dstate->vmem_size, CXL_CAPACITY_MULTIPLIER)) ||
-(!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER))) {
+(!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER)) ||
+(!QEMU_IS_ALIGNED(ct3d->dc.total_capacity, CXL_CAPACITY_MULTIPLIER))) {
 return CXL_MBOX_INTERNAL_ERROR;
 }
 
@@ -880,7 +885,13 @@ static CXLRetCode cmd_media_clear_poison(struct cxl_cmd 
*cmd,
 struct clear_poison_pl *in = (void *)cmd->payload;
 
 dpa = ldq_le_p(>dpa);
-if (dpa + CXL_CACHE_LINE_SIZE > cxl_dstate->static_mem_size) {
+if (dpa + CXL_CACHE_LINE_SIZE >= cxl_dstate->static_mem_size
+&& ct3d->dc.num_regions == 0) {
+return CXL_MBOX_INVALID_PA;
+}
+
+if (ct3d->dc.num_regions && dpa + CXL_CACHE_LINE_SIZE >=
+cxl_dstate->static_mem_size + ct3d->dc.total_capacity) {
 return CXL_MBOX_INVALID_PA;
 }
 
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index b29bb2309a..76bbd9f785 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -20,6 +20,7 @@
 #include "hw/pci/spdm.h"
 
 #define DWORD_BYTE 4
+#define CXL_CAPACITY_MULTIPLIER   (256 * MiB)
 
 /* Default CDAT entries for a memory region */
 enum {
@@ -33,8 +34,8 @@ enum {
 };
 
 static int ct3_build_cdat_entries_for_mr(CDATSubHeader **cdat_table,
- int dsmad_handle, MemoryRegion *mr,
- bool is_pmem, uint64_t dpa_base)
+int dsmad_handle, uint8_t flags,
+uint64_t dpa_base, uint64_t size)
 {
 g_autofree CDATDsmas *dsmas = NULL;
 g_autofree CDATDslbis *dslbis0 = NULL;
@@ -53,9 +54,9 @@ static int ct3_build_cdat_entries_for_mr(CDATSubHeader 
**cdat_table,
 .length = sizeof(*dsmas),
 },
 .DSMADhandle = dsmad_handle,
-.flags = is_pmem ? CDAT_DSMAS_FLAG_NV : 0,
+.flags = flags,
 .DPA_base = dpa_base,
-.DPA_length = memory_region_size(mr),
+.DPA_length = size,
 };
 
 /* For now, no memory side cache, plausiblish numbers */
@@ -137,9 +138,9 @@ static int ct3_build_cdat_entries_for_mr(CDATSubHeader 
**cdat_table,
  * NV: Reserved - the non volatile from DSMAS matters
  * V: EFI_MEMORY_SP
  */
-.EFI_memory_type_attr = is_pmem ? 2 : 1,
+.EFI_memory_type_attr = flags ? 2 : 1,
 .DPA_offset = 0,
-.DPA_length = memory_region_size(mr),
+.DPA_length = size,

[Qemu PATCH v2 8/9] hw/cxl/events: Add qmp interfaces to add/release dynamic capacity extents

[Fan Ni]

From: Fan Ni 

Since fabric manager emulation is not supported yet, the change implements
the functions to add/release dynamic capacity extents as QMP interfaces.

1. Add dynamic capacity extents:

For example, the command to add two continuous extents (each is 128MB long)
to region 0 (starting at dpa offset 0 and 128MB) looks like below:

{ "execute": "qmp_capabilities" }

{ "execute": "cxl-add-dynamic-capacity-event",
  "arguments": {
  "path": "/machine/peripheral/cxl-dcd0",
  "extents": [
  {
  "region-id": 0,
  "dpa": 0,
  "len": 128
  },
  {
  "region-id": 0,
  "dpa": 128,
  "len": 128
  }
  ]
  }
}

2. Release dynamic capacity extents:

For example, the command to release an extent of size 128MB from region 0
(starting at dpa offset 128MB) look like below:

{ "execute": "cxl-release-dynamic-capacity-event",
  "arguments": {
  "path": "/machine/peripheral/cxl-dcd0",
  "extents": [
  {
  "region-id": 0,
  "dpa": 128,
  "len": 128
  }
  ]
  }
}

Signed-off-by: Fan Ni 
---
 hw/mem/cxl_type3.c  | 145 
 hw/mem/cxl_type3_stubs.c|   6 ++
 include/hw/cxl/cxl_events.h |  16 
 qapi/cxl.json   |  49 
 4 files changed, 216 insertions(+)

diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index f1170b8047..41a828598a 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -1817,6 +1817,151 @@ void qmp_cxl_inject_memory_module_event(const char 
*path, CxlEventLog log,
 }
 }
 
+static const QemuUUID dynamic_capacity_uuid = {
+.data = UUID(0xca95afa7, 0xf183, 0x4018, 0x8c, 0x2f,
+0x95, 0x26, 0x8e, 0x10, 0x1a, 0x2a),
+};
+
+/*
+ * cxl r3.0: Table 8-47
+ * 00h: add capacity
+ * 01h: release capacity
+ * 02h: forced capacity release
+ * 03h: region configuration updated
+ * 04h: Add capacity response
+ * 05h: capacity released
+ */
+enum DC_Event_Type {
+DC_EVENT_ADD_CAPACITY,
+DC_EVENT_RELEASE_CAPACITY,
+DC_EVENT_FORCED_RELEASE_CAPACITY,
+DC_EVENT_REGION_CONFIG_UPDATED,
+DC_EVENT_ADD_CAPACITY_RSP,
+DC_EVENT_CAPACITY_RELEASED,
+DC_EVENT_NUM
+};
+
+#define MEM_BLK_SIZE_MB 128
+static void qmp_cxl_process_dynamic_capacity_event(const char *path,
+CxlEventLog log, enum DC_Event_Type type,
+uint16_t hid, CXLDCExtentRecordList *records, Error **errp)
+{
+Object *obj = object_resolve_path(path, NULL);
+CXLEventDynamicCapacity dCap;
+CXLEventRecordHdr *hdr = 
+CXLDeviceState *cxlds;
+CXLType3Dev *dcd;
+uint8_t flags = 1 << CXL_EVENT_TYPE_INFO;
+uint32_t num_extents = 0;
+CXLDCExtentRecordList *list = records;
+CXLDCExtent_raw *extents;
+uint64_t dpa, len;
+uint8_t rid = 0;
+int i;
+
+if (!obj) {
+error_setg(errp, "Unable to resolve path");
+return;
+}
+if (!object_dynamic_cast(obj, TYPE_CXL_TYPE3)) {
+error_setg(errp, "Path not point to a valid CXL type3 device");
+return;
+}
+
+dcd = CXL_TYPE3(obj);
+cxlds = >cxl_dstate;
+memset(, 0, sizeof(dCap));
+
+if (!dcd->dc.num_regions) {
+error_setg(errp, "No dynamic capacity support from the device");
+return;
+}
+
+while (list) {
+dpa = list->value->dpa * 1024 * 1024;
+len = list->value->len * 1024 * 1024;
+rid = list->value->region_id;
+
+if (rid >= dcd->dc.num_regions) {
+error_setg(errp, "region id is too large");
+return;
+}
+
+if (dpa % dcd->dc.regions[rid].block_size
+|| len % dcd->dc.regions[rid].block_size) {
+error_setg(errp, "dpa or len is not aligned to region block size");
+return;
+}
+
+if (dpa + len > dcd->dc.regions[rid].decode_len * 256 * 1024 * 1024) {
+error_setg(errp, "extent range is beyond the region end");
+return;
+}
+
+num_extents++;
+list = list->next;
+}
+
+i = 0;
+list = records;
+extents = g_new0(CXLDCExtent_raw, num_extents);
+while (list) {
+dpa = list->value->dpa * 1024 * 1024;
+len = list->value->len * 1024 * 1024;
+rid = list->value->region_id;
+
+extents[i].start_dpa = dpa + dcd->dc.regions[rid].base;
+extents[i].len = len;
+memset(extents[i].tag, 0, 0x10);
+extents[i].shared_seq = 0;
+
+list = list->next;
+i++;
+}
+
+/*
+ * 8.2.9.1.5
+ * All Dynamic Capacity event records shall set the Event Record
+ * Severi

[Qemu PATCH v2 0/9] Enabling DCD emulation support in Qemu

[Fan Ni]

v1[1]->v2:

1. fix a regression issue reported by Ira[2]:
2. fix a compile warning due to uninitialized 'rip' in qmp processing function.


[1] 
https://lore.kernel.org/linux-cxl/20230724162313.34196-1-fan...@samsung.com/T/#t
[2] 
https://lore.kernel.org/linux-cxl/64bfe7b090843_12757b2945b@iweiny-mobl.notmuch/T/#m09983a3dbaa9135a850e345d86714bf2ab957ef6

Fan Ni (9):
  hw/cxl/cxl-mailbox-utils: Add dc_event_log_size field to output
payload of identify memory device command
  hw/cxl/cxl-mailbox-utils: Add dynamic capacity region representative
and mailbox command support
  include/hw/cxl/cxl_device: Rename mem_size as static_mem_size for
type3 memory devices
  hw/mem/cxl_type3: Add support to create DC regions to type3 memory
devices
  hw/mem/cxl_type3: Add host backend and address space handling for DC
regions
  hw/mem/cxl_type3: Add DC extent list representative and get DC extent
list mailbox support
  hw/cxl/cxl-mailbox-utils: Add mailbox commands to support add/release
dynamic capacity response
  hw/cxl/events: Add qmp interfaces to add/release dynamic capacity
extents
  hw/mem/cxl_type3: Add dpa range validation for accesses to dc regions

 hw/cxl/cxl-mailbox-utils.c  | 424 +++-
 hw/mem/cxl_type3.c  | 545 +---
 hw/mem/cxl_type3_stubs.c|   6 +
 include/hw/cxl/cxl_device.h |  50 +++-
 include/hw/cxl/cxl_events.h |  16 ++
 qapi/cxl.json   |  49 
 6 files changed, 1044 insertions(+), 46 deletions(-)

-- 
2.25.1

[Qemu PATCH v2 7/9] hw/cxl/cxl-mailbox-utils: Add mailbox commands to support add/release dynamic capacity response

[Fan Ni]

From: Fan Ni 

Per CXL spec 3.0, two mailbox commands are implemented:
Add Dynamic Capacity Response (Opcode 4802h) 8.2.9.8.9.3, and
Release Dynamic Capacity (Opcode 4803h) 8.2.9.8.9.4.

Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c  | 253 
 include/hw/cxl/cxl_device.h |   3 +-
 2 files changed, 255 insertions(+), 1 deletion(-)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 3d25a9697e..1e4944da95 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -84,6 +84,8 @@ enum {
 DCD_CONFIG  = 0x48, /*r3.0: 8.2.9.8.9*/
 #define GET_DC_CONFIG  0x0
 #define GET_DYN_CAP_EXT_LIST   0x1
+#define ADD_DYN_CAP_RSP0x2
+#define RELEASE_DYN_CAP0x3
 PHYSICAL_SWITCH = 0x51
 #define IDENTIFY_SWITCH_DEVICE  0x0
 };
@@ -1086,6 +1088,251 @@ static CXLRetCode cmd_dcd_get_dyn_cap_ext_list(struct 
cxl_cmd *cmd,
 return CXL_MBOX_SUCCESS;
 }
 
+/*
+ * Check whether the bits at addr between [nr, nr+size) are all set,
+ * return 1 if all 1s, else return 0
+ */
+static inline int test_bits(const unsigned long *addr, int nr, int size)
+{
+unsigned long res = find_next_zero_bit(addr, size + nr, nr);
+
+return (res >= nr + size) ? 1 : 0;
+}
+
+/*
+ * Find dynamic capacity region id based on dpa range [dpa, dpa+len)
+ */
+static uint8_t find_region_id(struct CXLType3Dev *dev, uint64_t dpa,
+uint64_t len)
+{
+int8_t i = dev->dc.num_regions - 1;
+
+while (i > 0 && dpa < dev->dc.regions[i].base) {
+i--;
+}
+
+if (dpa < dev->dc.regions[i].base
+|| dpa + len > dev->dc.regions[i].base + dev->dc.regions[i].len) {
+return dev->dc.num_regions;
+}
+
+return i;
+}
+
+static void insert_extent_to_extent_list(CXLDCDExtentList *list, uint64_t dpa,
+uint64_t len, uint8_t *tag, uint16_t shared_seq)
+{
+CXLDCD_Extent *extent;
+extent = g_new0(CXLDCD_Extent, 1);
+extent->start_dpa = dpa;
+extent->len = len;
+if (tag) {
+memcpy(extent->tag, tag, 0x10);
+} else {
+memset(extent->tag, 0, 0x10);
+}
+extent->shared_seq = shared_seq;
+
+QTAILQ_INSERT_TAIL(list, extent, node);
+}
+
+typedef struct updated_dc_extent_list_in_pl {
+uint32_t num_entries_updated;
+uint8_t rsvd[4];
+struct { /* r3.0: Table 8-130 */
+uint64_t start_dpa;
+uint64_t len;
+uint8_t rsvd[8];
+} QEMU_PACKED updated_entries[];
+} QEMU_PACKED updated_dc_extent_list_in_pl;
+
+/*
+ * The function only check the input extent list against itself.
+ */
+static CXLRetCode detect_malformed_extent_list(CXLType3Dev *dev,
+const updated_dc_extent_list_in_pl *in)
+{
+unsigned long *blk_bitmap;
+uint64_t min_block_size = dev->dc.regions[0].block_size;
+struct CXLDCD_Region *region = >dc.regions[0];
+uint32_t i;
+uint64_t dpa, len;
+uint8_t rid;
+CXLRetCode ret;
+
+for (i = 1; i < dev->dc.num_regions; i++) {
+region = >dc.regions[i];
+if (min_block_size > region->block_size) {
+min_block_size = region->block_size;
+}
+}
+
+blk_bitmap = bitmap_new((region->len + region->base
+- dev->dc.regions[0].base) / min_block_size);
+
+for (i = 0; i < in->num_entries_updated; i++) {
+dpa = in->updated_entries[i].start_dpa;
+len = in->updated_entries[i].len;
+
+rid = find_region_id(dev, dpa, len);
+if (rid == dev->dc.num_regions) {
+ret = CXL_MBOX_INVALID_PA;
+goto out;
+}
+
+region = >dc.regions[rid];
+if (dpa % region->block_size || len % region->block_size) {
+ret = CXL_MBOX_INVALID_EXTENT_LIST;
+goto out;
+}
+/* the dpa range already covered by some other extents in the list */
+if (test_bits(blk_bitmap, dpa / min_block_size, len / min_block_size)) 
{
+ret = CXL_MBOX_INVALID_EXTENT_LIST;
+goto out;
+}
+bitmap_set(blk_bitmap, dpa / min_block_size, len / min_block_size);
+   }
+
+ret = CXL_MBOX_SUCCESS;
+
+out:
+g_free(blk_bitmap);
+return ret;
+}
+
+/*
+ * cxl spec 3.0: 8.2.9.8.9.3
+ * Add Dynamic Capacity Response (opcode 4802h)
+ * Assume an extent is added only after the response is processed successfully
+ * TODO: for better extent list validation, a better solution would be
+ * maintaining a pending extent list and use it to verify the extent list in
+ * the response.
+ */
+static CXLRetCode cmd_dcd_add_dyn_cap_rsp(struct cxl_cmd *cmd,
+CXLDeviceState *cxl_dstate, uint16_t *len_unused)
+{
+updated_dc_extent_list_in_pl *in = (void *)cmd->payload;
+struct CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev,
+cxl_dstate);
+CXLDCDExtentLi

[Qemu PATCH v2 6/9] hw/mem/cxl_type3: Add DC extent list representative and get DC extent list mailbox support

[Fan Ni]

From: Fan Ni 

Add dynamic capacity extent list representative to the definition of
CXLType3Dev and add get DC extent list mailbox command per
CXL.spec.3.0:.8.2.9.8.9.2.

Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c  | 71 +
 hw/mem/cxl_type3.c  |  1 +
 include/hw/cxl/cxl_device.h | 23 
 3 files changed, 95 insertions(+)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 0511b8e6f7..3d25a9697e 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -83,6 +83,7 @@ enum {
 #define CLEAR_POISON   0x2
 DCD_CONFIG  = 0x48, /*r3.0: 8.2.9.8.9*/
 #define GET_DC_CONFIG  0x0
+#define GET_DYN_CAP_EXT_LIST   0x1
 PHYSICAL_SWITCH = 0x51
 #define IDENTIFY_SWITCH_DEVICE  0x0
 };
@@ -1018,6 +1019,73 @@ static CXLRetCode cmd_dcd_get_dyn_cap_config(struct 
cxl_cmd *cmd,
 return CXL_MBOX_SUCCESS;
 }
 
+/*
+ * cxl spec 3.0: 8.2.9.8.9.2
+ * Get Dynamic Capacity Extent List (Opcode 4810h)
+ */
+static CXLRetCode cmd_dcd_get_dyn_cap_ext_list(struct cxl_cmd *cmd,
+CXLDeviceState *cxl_dstate,
+uint16_t *len)
+{
+struct get_dyn_cap_ext_list_in_pl {
+uint32_t extent_cnt;
+uint32_t start_extent_id;
+} QEMU_PACKED;
+
+struct get_dyn_cap_ext_list_out_pl {
+uint32_t count;
+uint32_t total_extents;
+uint32_t generation_num;
+uint8_t rsvd[4];
+CXLDCExtent_raw records[];
+} QEMU_PACKED;
+
+struct get_dyn_cap_ext_list_in_pl *in = (void *)cmd->payload;
+struct get_dyn_cap_ext_list_out_pl *out = (void *)cmd->payload;
+struct CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev,
+cxl_dstate);
+uint16_t record_count = 0, i = 0, record_done = 0;
+CXLDCDExtentList *extent_list = >dc.extents;
+CXLDCD_Extent *ent;
+uint16_t out_pl_len;
+uint32_t start_extent_id = in->start_extent_id;
+
+if (start_extent_id > ct3d->dc.total_extent_count) {
+return CXL_MBOX_INVALID_INPUT;
+}
+
+record_count = MIN(in->extent_cnt,
+ct3d->dc.total_extent_count - start_extent_id);
+
+out_pl_len = sizeof(*out) + record_count * sizeof(out->records[0]);
+/* May need more processing here in the future */
+assert(out_pl_len <= CXL_MAILBOX_MAX_PAYLOAD_SIZE);
+
+memset(out, 0, out_pl_len);
+stl_le_p(>count, record_count);
+stl_le_p(>total_extents, ct3d->dc.total_extent_count);
+stl_le_p(>generation_num, ct3d->dc.ext_list_gen_seq);
+
+if (record_count > 0) {
+QTAILQ_FOREACH(ent, extent_list, node) {
+if (i++ < start_extent_id) {
+continue;
+}
+stq_le_p(>records[record_done].start_dpa, ent->start_dpa);
+stq_le_p(>records[record_done].len, ent->len);
+memcpy(>records[record_done].tag, ent->tag, 0x10);
+stw_le_p(>records[record_done].shared_seq, ent->shared_seq);
+record_done++;
+if (record_done == record_count) {
+break;
+}
+}
+}
+
+*len = out_pl_len;
+return CXL_MBOX_SUCCESS;
+}
+
 #define IMMEDIATE_CONFIG_CHANGE (1 << 1)
 #define IMMEDIATE_DATA_CHANGE (1 << 2)
 #define IMMEDIATE_POLICY_CHANGE (1 << 3)
@@ -1058,6 +1126,9 @@ static struct cxl_cmd cxl_cmd_set[256][256] = {
 cmd_media_clear_poison, 72, 0 },
 [DCD_CONFIG][GET_DC_CONFIG] = { "DCD_GET_DC_CONFIG",
 cmd_dcd_get_dyn_cap_config, 2, 0 },
+[DCD_CONFIG][GET_DYN_CAP_EXT_LIST] = {
+"DCD_GET_DYNAMIC_CAPACITY_EXTENT_LIST", cmd_dcd_get_dyn_cap_ext_list,
+8, 0 },
 };
 
 static struct cxl_cmd cxl_cmd_set_sw[256][256] = {
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 76bbd9f785..f1170b8047 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -789,6 +789,7 @@ static int cxl_create_dc_regions(CXLType3Dev *ct3d)
 
 region_base += region->len;
 }
+QTAILQ_INIT(>dc.extents);
 
 return 0;
 }
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index 1c99b05a66..3a338b3b37 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -385,6 +385,25 @@ typedef QLIST_HEAD(, CXLPoison) CXLPoisonList;
 
 #define DCD_MAX_REGION_NUM 8
 
+typedef struct CXLDCD_Extent_raw {
+uint64_t start_dpa;
+uint64_t len;
+uint8_t tag[0x10];
+uint16_t shared_seq;
+uint8_t rsvd[0x6];
+} QEMU_PACKED CXLDCExtent_raw;
+
+typedef struct CXLDCD_Extent {
+uint64_t start_dpa;
+uint64_t len;
+uint8_t tag[0x10];
+uint16_t shared_seq;
+uint8_t rsvd[0x6];
+
+QTAILQ_ENTRY(CXLDCD_Extent) node;
+} CXLDCD_Extent;
+typedef QTAILQ_HEAD(, CXLDCD_Extent) CXLDCDExtentList;
+
 typedef struct CXLDCD_Region {
 uint64_t base;
 uint64_t decode_len; /* in multiples 

[Qemu PATCH v2 1/9] hw/cxl/cxl-mailbox-utils: Add dc_event_log_size field to output payload of identify memory device command

[Fan Ni]

From: Fan Ni 

Based on CXL spec 3.0 Table 8-94 (Identify Memory Device Output
Payload), dynamic capacity event log size should be part of
output of the Identify command.
Add dc_event_log_size to the output payload for the host to get the info.

Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index ad7a6116e4..b013e30314 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -21,6 +21,8 @@
 #include "sysemu/hostmem.h"
 
 #define CXL_CAPACITY_MULTIPLIER   (256 * MiB)
+/* Experimental value: dynamic capacity event log size */
+#define CXL_DC_EVENT_LOG_SIZE 8
 
 /*
  * How to add a new command, example. The command set FOO, with cmd BAR.
@@ -519,8 +521,9 @@ static CXLRetCode cmd_identify_memory_device(struct cxl_cmd 
*cmd,
 uint16_t inject_poison_limit;
 uint8_t poison_caps;
 uint8_t qos_telemetry_caps;
+uint16_t dc_event_log_size;
 } QEMU_PACKED *id;
-QEMU_BUILD_BUG_ON(sizeof(*id) != 0x43);
+QEMU_BUILD_BUG_ON(sizeof(*id) != 0x45);
 
 CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
 CXLType3Class *cvc = CXL_TYPE3_GET_CLASS(ct3d);
@@ -543,6 +546,7 @@ static CXLRetCode cmd_identify_memory_device(struct cxl_cmd 
*cmd,
 st24_le_p(id->poison_list_max_mer, 256);
 /* No limit - so limited by main poison record limit */
 stw_le_p(>inject_poison_limit, 0);
+stw_le_p(>dc_event_log_size, CXL_DC_EVENT_LOG_SIZE);
 
 *len = sizeof(*id);
 return CXL_MBOX_SUCCESS;
-- 
2.25.1

[Qemu PATCH v2 4/9] hw/mem/cxl_type3: Add support to create DC regions to type3 memory devices

[Fan Ni]

From: Fan Ni 

With the change, when setting up memory for type3 memory device, we can
create DC regions
A property 'num-dc-regions' is added to ct3_props to allow users to pass the
number of DC regions to create. To make it easier, other region parameters
like region base, length, and block size are hard coded. If needed,
these parameters can be added easily.

With the change, we can create DC regions with proper kernel side
support as below:

region=$(cat /sys/bus/cxl/devices/decoder0.0/create_dc_region)
echo $region> /sys/bus/cxl/devices/decoder0.0/create_dc_region
echo 256 > /sys/bus/cxl/devices/$region/interleave_granularity
echo 1 > /sys/bus/cxl/devices/$region/interleave_ways

echo "dc0" >/sys/bus/cxl/devices/decoder2.0/mode
echo 0x4000 >/sys/bus/cxl/devices/decoder2.0/dpa_size

echo 0x4000 > /sys/bus/cxl/devices/$region/size
echo  "decoder2.0" > /sys/bus/cxl/devices/$region/target0
echo 1 > /sys/bus/cxl/devices/$region/commit
echo $region > /sys/bus/cxl/drivers/cxl_region/bind

Signed-off-by: Fan Ni 
---
 hw/mem/cxl_type3.c | 33 +
 1 file changed, 33 insertions(+)

diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 3d7acffcb7..b29bb2309a 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -707,6 +707,34 @@ static void ct3d_reg_write(void *opaque, hwaddr offset, 
uint64_t value,
 }
 }
 
+/*
+ * Create a dc region to test "Get Dynamic Capacity Configuration" command.
+ */
+static int cxl_create_dc_regions(CXLType3Dev *ct3d)
+{
+int i;
+uint64_t region_base = (ct3d->hostvmem ? ct3d->hostvmem->size : 0)
++ (ct3d->hostpmem ? ct3d->hostpmem->size : 0);
+uint64_t region_len = (uint64_t)2 * 1024 * 1024 * 1024;
+uint64_t decode_len = 4; /* 4*256MB */
+uint64_t blk_size = 2 * 1024 * 1024;
+struct CXLDCD_Region *region;
+
+for (i = 0; i < ct3d->dc.num_regions; i++) {
+region = >dc.regions[i];
+region->base = region_base;
+region->decode_len = decode_len;
+region->len = region_len;
+region->block_size = blk_size;
+/* dsmad_handle is set when creating cdat table entries */
+region->flags = 0;
+
+region_base += region->len;
+}
+
+return 0;
+}
+
 static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
 {
 DeviceState *ds = DEVICE(ct3d);
@@ -775,6 +803,10 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error 
**errp)
 g_free(p_name);
 }
 
+if (cxl_create_dc_regions(ct3d)) {
+return false;
+}
+
 return true;
 }
 
@@ -1068,6 +1100,7 @@ static Property ct3_props[] = {
 DEFINE_PROP_UINT64("sn", CXLType3Dev, sn, UI64_NULL),
 DEFINE_PROP_STRING("cdat", CXLType3Dev, cxl_cstate.cdat.filename),
 DEFINE_PROP_UINT16("spdm", CXLType3Dev, spdm_port, 0),
+DEFINE_PROP_UINT8("num-dc-regions", CXLType3Dev, dc.num_regions, 0),
 DEFINE_PROP_END_OF_LIST(),
 };
 
-- 
2.25.1

Re: [Qemu RFC 0/7] Early enabling of DCD emulation in Qemu

[Fan Ni]

On Tue, Jul 25, 2023 at 08:18:08AM -0700, Ira Weiny wrote:

> Fan Ni wrote:
> > On Thu, May 11, 2023 at 05:56:40PM +0000, Fan Ni wrote:
> > 
> > FYI.
> > 
> > I have updated the patch series and sent out again.
> > 
> > I suggested anyone who are interested in DCD and using this patch series to
> > use the new series. Quite a few things has been fixed.
> > 
> > https://urldefense.com/v3/__https://lore.kernel.org/linux-cxl/20230724162313.34196-1-fan...@samsung.com/T/*t__;Iw!!EwVzqGoTKBqv-0DWAJBm!V8kDTpT5yLUAyWm3sFm7XIgN0QdNUyQYZd9vYLHjUVkMkhDT14F8avgNBh23KPAtsS_dGm2LZuHJ102mgIg$
> >  
> > 
> > Also, if you want to use the code repo directly, you can try
> > 
> > https://urldefense.com/v3/__https://protect2.fireeye.com/v1/url?k=0600fd6b-678be820-06017624-74fe485fb305-f529279062b02b73=1=5df65010-e62f-40a1-9a21-609eb1400921=https*3A*2F*2Fgithub.com*2Fmoking*2Fqemu-dcd-preview-latest*2Ftree*2Fdcd-dev__;JSUlJSUlJQ!!EwVzqGoTKBqv-0DWAJBm!V8kDTpT5yLUAyWm3sFm7XIgN0QdNUyQYZd9vYLHjUVkMkhDT14F8avgNBh23KPAtsS_dGm2LZuHJpB0SmNs$
> >  
> 
> Thanks for the branch!
> 
> I took a quick look and I don't see a resolution to the problem I
> mentioned with non DCD devices being supported.[1]
> 
> [1] 
> https://urldefense.com/v3/__https://lore.kernel.org/all/6483946e8152f_f1132294a2@iweiny-mobl.notmuch/__;!!EwVzqGoTKBqv-0DWAJBm!V8kDTpT5yLUAyWm3sFm7XIgN0QdNUyQYZd9vYLHjUVkMkhDT14F8avgNBh23KPAtsS_dGm2LZuHJ4geSEAg$
>  
> 
> Did you fix this in a different way?  If I don't add DC to my mem devices they
> don't get probed properly.  I'm still looking into this with your new branch,
> but I don't think DC commands should be in the CEL if the device does not
> support it.
> 
> Also I get a build warning on this branch I had to fix[3] as my build is
> treating warnings as errors.[2]
> 
> I don't think this fix is technically necessary as 'list' should never be NULL
> that I can see.  But might be nice to check or just use my fix.
> 
> I'll try and get to a review once I get the DCD stuff out on the list again.
> 
> Ira
> 

Oh, I missed your previous comments, let me look into it and fix
accordingly and send out a new version.

Btw, when I did the DCD test with the last DCD kernel code, I found
some issue there.

When I add a DCD extent for the first time, it will be recognized as
system RAM automatically and show up with command lsmem.

However, when I release it and try to re-add the same extent again.
The adding seems normal and the device will show up under /dev/ as
dax0.X. But it will not show up with lsmem command and I have to use
daxctl reconfigure command to turn it to system ram and then it can
show up with lsmem command. I would expect the behavior for the
first add and second add be the same.

Fan.


> 
> [2]
> ../hw/mem/cxl_type3.c: In function 
> ‘qmp_cxl_process_dynamic_capacity_event.constprop’:
> ../hw/mem/cxl_type3.c:2063:28: error: ‘rid’ may be used uninitialized 
> [-Werror=maybe-uninitialized]
>  2063 | dCap.updated_region_id = rid;
>   | ~~~^
> ../hw/mem/cxl_type3.c:1987:13: note: ‘rid’ was declared here
>  1987 | uint8_t rid;
>   | ^~~
> cc1: all warnings being treated as errors
> 
> [3]
> 
> diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
> index e67328780407..d25e6064f6c9 100644
> --- a/hw/mem/cxl_type3.c
> +++ b/hw/mem/cxl_type3.c
> @@ -1984,7 +1984,7 @@ static void 
> qmp_cxl_process_dynamic_capacity_event(const char *path,
>  CXLDCExtentRecordList *list = records;
>  CXLDCExtent_raw *extents;
>  uint64_t dpa, len;
> -uint8_t rid;
> +uint8_t rid = 0;
>  int i;
>  
>  if (!obj) {

Re: [Qemu RFC 0/7] Early enabling of DCD emulation in Qemu

[Fan Ni]

On Thu, May 11, 2023 at 05:56:40PM +, Fan Ni wrote:

FYI.

I have updated the patch series and sent out again.

I suggested anyone who are interested in DCD and using this patch series to
use the new series. Quite a few things has been fixed.

https://lore.kernel.org/linux-cxl/20230724162313.34196-1-fan...@samsung.com/T/#t

Also, if you want to use the code repo directly, you can try

https://github.com/moking/qemu-dcd-preview-latest/tree/dcd-dev

Fan


> Since the early draft of DCD support in kernel is out
> (https://lore.kernel.org/linux-cxl/20230417164126.GA1904906@bgt-140510-bm03/T/#t),
> this patch series provide dcd emulation in qemu so people who are interested
> can have an early try. It is noted that the patch series may need to be 
> updated
> accordingly if the kernel side implementation changes.
> 
> To support DCD emulation, the patch series add DCD related mailbox command
> support (CXL Spec 3.0: 8.2.9.8.9), and extend the cxl type3 memory device
> with dynamic capacity extent and region representative.
> To support read/write to the dynamic capacity of the device, a host backend
> is provided and necessary check mechnism is added to ensure the dynamic
> capacity accessed is backed with active dc extents.
> Currently FM related mailbox commands (cxl spec 3.0: 7.6.7.6) is not supported
> , but we add two qmp interfaces for adding/releasing dynamic capacity extents.
> Also, the support for multiple hosts sharing the same DCD case is missing.
> 
> Things we can try with the patch series together with kernel dcd code:
> 1. Create DC regions to cover the address range of the dynamic capacity
> regions.
> 2. Add/release dynamic capacity extents to the device and notify the
> kernel.
> 3. Test kernel side code to accept added dc extents and create dax devices,
> and release dc extents and notify the device
> 4. Online the memory range backed with dc extents and let application use
> them.
> 
> The patch series is based on Jonathan's local qemu branch:
> https://gitlab.com/jic23/qemu/-/tree/cxl-2023-02-28
> 
> Simple tests peformed with the patch series:
> 1 Install cxl modules:
> 
> modprobe -a cxl_acpi cxl_core cxl_pci cxl_port cxl_mem
> 
> 2 Create dc regions:
> 
> region=$(cat /sys/bus/cxl/devices/decoder0.0/create_dc_region)
> echo $region> /sys/bus/cxl/devices/decoder0.0/create_dc_region
> echo 256 > /sys/bus/cxl/devices/$region/interleave_granularity
> echo 1 > /sys/bus/cxl/devices/$region/interleave_ways
> echo "dc" >/sys/bus/cxl/devices/decoder2.0/mode
> echo 0x1000 >/sys/bus/cxl/devices/decoder2.0/dpa_size
> echo 0x1000 > /sys/bus/cxl/devices/$region/size
> echo  "decoder2.0" > /sys/bus/cxl/devices/$region/target0
> echo 1 > /sys/bus/cxl/devices/$region/commit
> echo $region > /sys/bus/cxl/drivers/cxl_region/bind
> 
> /home/fan/cxl/tools-and-scripts# cxl list
> [
>   {
> "memdevs":[
>   {
> "memdev":"mem0",
> "pmem_size":536870912,
> "ram_size":0,
> "serial":0,
> "host":":0d:00.0"
>   }
> ]
>   },
>   {
> "regions":[
>   {
> "region":"region0",
> "resource":45365592064,
> "size":268435456,
> "interleave_ways":1,
> "interleave_granularity":256,
> "decode_state":"commit"
>   }
> ]
>   }
> ]
> 
> 3 Add two dc extents (128MB each) through qmp interface
> 
> { "execute": "qmp_capabilities" }
> 
> { "execute": "cxl-add-dynamic-capacity-event",
>   "arguments": {
>"path": "/machine/peripheral/cxl-pmem0",
>   "region-id" : 0,
>"num-extent": 2,
>   "dpa":0,
>   "extent-len": 128
>   }
> }
> 
> /home/fan/cxl/tools-and-scripts# lsmem
> RANGE  SIZE   STATE REMOVABLE   BLOCK
> 0x-0x7fff2G  online   yes0-15
> 0x0001-0x00027fff6G  online   yes   32-79
> 0x000a9000-0x000a9fff  256M offline   338-339
> 
> Memory block size:   128M
> Total online memory:   8G
> Total offline memory:256M
> 
> 
> 4.Online the momory with 'daxctl online-memory dax0.0' to online the memory
> 
> /home/fan/cxl/ndctl# ./build/daxctl/daxctl online-memory dax0.0
> [  230.730553] Fallback order for Node 0: 0 1
> [  230.730825] Fallback order for Node 1: 1

Re: [Qemu PATCH 0/9] Enabling DCD emulation support in Qemu

[Fan Ni]

On Mon, Jul 24, 2023 at 09:57:48AM +0100, Jonathan Cameron wrote:

Hi Ira & Jonathan,

Thanks for helping check. I resend the patch series and now they
look normal now. Although I am still not sure what happened to my
previous sending.

The resend is here:
https://lore.kernel.org/linux-cxl/20230724162313.34196-1-fan...@samsung.com/T/#t

Thanks,
Fan



> On Sat, 22 Jul 2023 21:52:06 -0700
> Ira Weiny  wrote:
> 
> > nifan@ wrote:
> > > From: Fan Ni 
> > > 
> > > The patch series provides dynamic capacity device (DCD) emulation in 
> > > Qemu.  
> > 
> > I don't the patches on the list.
> > 
> > https://lore.kernel.org/all/sg2pr06mb33976bb3f9c47cbe08f02d09b2...@sg2pr06mb3397.apcprd06.prod.outlook.com/
> > 
> > Did they get sent?
> Something odd going on...
> They are in 
> https://lists.nongnu.org/archive/html/qemu-devel/2023-07/msg04105.html
> though threading is broken.
> Also seems to have made it to qemu-devel on lore
> https://lore.kernel.org/all/sg2pr06mb3397f3e74a083607f7492fa4b2...@sg2pr06mb3397.apcprd06.prod.outlook.com/
> 
> Might be a backlog somewhere that will take a while to drain...
> 
> Fan, perhaps just resend the lot and make sure the threading works so we can 
> find them more easily.
> 
> Jonathan
> 
> 
> > 
> > Ira
> > 
> > > More specifically, it provides the following functionalities:
> > > 1. Extended type3 memory device to support DC regions and extents.
> > > 2. Implemented DCD related mailbox command support in CXL r3.0: 8.2.9.8.9.
> > > 3. ADD QMP interfaces for adding and releasing DC extents to simulate FM
> > > functions for DCD described in cxl r3.0: 7.6.7.6.5 and 7.6.7.6.6.
> > > 4. Add new ct3d properties for DCD devices (host backend, number of dc
> > > regions, etc.)
> > > 5. Add read/write support from/to DC regions of the device.
> > > 6. Add mechanism to validate accessed to DC region address space.
> > > 
> > > A more detailed description can be found from the previously posted 
> > > RFC[1].
> > > 
> > > Compared to the previously posted RFC[1], following changes have been 
> > > made:
> > > 1. Rebased the code on top of Jonathan's branch
> > > https://gitlab.com/jic23/qemu/-/tree/cxl-2023-05-25.
> > > 2. Extracted the rename of mem_size to a separated patch.(Jonathan)
> > > 3. Reordered the patch series to improve its readability.(Jonathan)
> > > 4. Split the validation of accesses to DC region address space as a 
> > > separate
> > > patch.
> > > 5. Redesigned the QMP interfaces for adding and releasing DC extents to 
> > > make
> > > them easier to understand and act like existing QMP interfaces (like the
> > > interface for cxl-inject-uncorrectable-errors). (Jonathan)
> > > 6. Updated dvsec range register setting to support DCD devices without 
> > > static
> > > capacity.
> > > 7. Fixed other issues mentioned in the comments (Jonathan 
> > > Fontenot).
> > > 8. Fixed the format issues and checked with checkpatch.pl under qemu code 
> > > dir.
> > > 
> > > 
> > > The code is tested with the DCD patch series at the kernel side[2]. The 
> > > test
> > > is similar to those mentioned in the cover letter of [1].
> > > 
> > > 
> > > [1]: 
> > > https://lore.kernel.org/all/20230511175609.2091136-1-fan...@samsung.com/
> > > [2]: 
> > > https://lore.kernel.org/linux-cxl/649da378c28a3_968bb29420@iweiny-mobl.notmuch/T/#t
> > > 
> > > Fan Ni (9):
> > >   hw/cxl/cxl-mailbox-utils: Add dc_event_log_size field to output
> > > payload of identify memory device command
> > >   hw/cxl/cxl-mailbox-utils: Add dynamic capacity region representative
> > > and mailbox command support
> > >   include/hw/cxl/cxl_device: Rename mem_size as static_mem_size for
> > > type3 memory devices
> > >   hw/mem/cxl_type3: Add support to create DC regions to type3 memory
> > > devices
> > >   hw/mem/cxl_type3: Add host backend and address space handling for DC
> > > regions
> > >   hw/mem/cxl_type3: Add DC extent list representative and get DC extent
> > > list mailbox support
> > >   hw/cxl/cxl-mailbox-utils: Add mailbox commands to support add/release
> > > dynamic capacity response
> > >   hw/cxl/events: Add qmp interfaces to add/release dynamic capacity
> > > extents
> > >   hw/mem/cxl_type3: Add dpa range validation for accesses to dc regions
> > > 
> > >  hw/cxl/cxl-mailbox-utils.c  | 421 +++-
> > >  hw/mem/cxl_type3.c  | 539 +---
> > >  hw/mem/cxl_type3_stubs.c|   6 +
> > >  include/hw/cxl/cxl_device.h |  49 +++-
> > >  include/hw/cxl/cxl_events.h |  16 ++
> > >  qapi/cxl.json   |  49 
> > >  6 files changed, 1034 insertions(+), 46 deletions(-)
> > > 
> > > -- 
> > > 2.39.2
> > >   
> > 
> > 
> > 
> 
> 

[Qemu PATCH RESEND 1/9] hw/cxl/cxl-mailbox-utils: Add dc_event_log_size field to output payload of identify memory device command

[Fan Ni]

From: Fan Ni 

Based on CXL spec 3.0 Table 8-94 (Identify Memory Device Output
Payload), dynamic capacity event log size should be part of
output of the Identify command.
Add dc_event_log_size to the output payload for the host to get the info.

Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index ad7a6116e4..b013e30314 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -21,6 +21,8 @@
 #include "sysemu/hostmem.h"
 
 #define CXL_CAPACITY_MULTIPLIER   (256 * MiB)
+/* Experimental value: dynamic capacity event log size */
+#define CXL_DC_EVENT_LOG_SIZE 8
 
 /*
  * How to add a new command, example. The command set FOO, with cmd BAR.
@@ -519,8 +521,9 @@ static CXLRetCode cmd_identify_memory_device(struct cxl_cmd 
*cmd,
 uint16_t inject_poison_limit;
 uint8_t poison_caps;
 uint8_t qos_telemetry_caps;
+uint16_t dc_event_log_size;
 } QEMU_PACKED *id;
-QEMU_BUILD_BUG_ON(sizeof(*id) != 0x43);
+QEMU_BUILD_BUG_ON(sizeof(*id) != 0x45);
 
 CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
 CXLType3Class *cvc = CXL_TYPE3_GET_CLASS(ct3d);
@@ -543,6 +546,7 @@ static CXLRetCode cmd_identify_memory_device(struct cxl_cmd 
*cmd,
 st24_le_p(id->poison_list_max_mer, 256);
 /* No limit - so limited by main poison record limit */
 stw_le_p(>inject_poison_limit, 0);
+stw_le_p(>dc_event_log_size, CXL_DC_EVENT_LOG_SIZE);
 
 *len = sizeof(*id);
 return CXL_MBOX_SUCCESS;
-- 
2.25.1

[Qemu PATCH RESEND 8/9] hw/cxl/events: Add qmp interfaces to add/release dynamic capacity extents

[Fan Ni]

From: Fan Ni 

Since fabric manager emulation is not supported yet, the change implements
the functions to add/release dynamic capacity extents as QMP interfaces.

1. Add dynamic capacity extents:

For example, the command to add two continuous extents (each is 128MB long)
to region 0 (starting at dpa offset 0 and 128MB) looks like below:

{ "execute": "qmp_capabilities" }

{ "execute": "cxl-add-dynamic-capacity-event",
  "arguments": {
  "path": "/machine/peripheral/cxl-dcd0",
  "extents": [
  {
  "region-id": 0,
  "dpa": 0,
  "len": 128
  },
  {
  "region-id": 0,
  "dpa": 128,
  "len": 128
  }
  ]
  }
}

2. Release dynamic capacity extents:

For example, the command to release an extent of size 128MB from region 0
(starting at dpa offset 128MB) look like below:

{ "execute": "cxl-release-dynamic-capacity-event",
  "arguments": {
  "path": "/machine/peripheral/cxl-dcd0",
  "extents": [
  {
  "region-id": 0,
  "dpa": 128,
  "len": 128
  }
  ]
  }
}

Signed-off-by: Fan Ni 
---
 hw/mem/cxl_type3.c  | 145 
 hw/mem/cxl_type3_stubs.c|   6 ++
 include/hw/cxl/cxl_events.h |  16 
 qapi/cxl.json   |  49 
 4 files changed, 216 insertions(+)

diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 608063ac52..cb1f9182e6 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -1811,6 +1811,151 @@ void qmp_cxl_inject_memory_module_event(const char 
*path, CxlEventLog log,
 }
 }
 
+static const QemuUUID dynamic_capacity_uuid = {
+.data = UUID(0xca95afa7, 0xf183, 0x4018, 0x8c, 0x2f,
+0x95, 0x26, 0x8e, 0x10, 0x1a, 0x2a),
+};
+
+/*
+ * cxl r3.0: Table 8-47
+ * 00h: add capacity
+ * 01h: release capacity
+ * 02h: forced capacity release
+ * 03h: region configuration updated
+ * 04h: Add capacity response
+ * 05h: capacity released
+ */
+enum DC_Event_Type {
+DC_EVENT_ADD_CAPACITY,
+DC_EVENT_RELEASE_CAPACITY,
+DC_EVENT_FORCED_RELEASE_CAPACITY,
+DC_EVENT_REGION_CONFIG_UPDATED,
+DC_EVENT_ADD_CAPACITY_RSP,
+DC_EVENT_CAPACITY_RELEASED,
+DC_EVENT_NUM
+};
+
+#define MEM_BLK_SIZE_MB 128
+static void qmp_cxl_process_dynamic_capacity_event(const char *path,
+CxlEventLog log, enum DC_Event_Type type,
+uint16_t hid, CXLDCExtentRecordList *records, Error **errp)
+{
+Object *obj = object_resolve_path(path, NULL);
+CXLEventDynamicCapacity dCap;
+CXLEventRecordHdr *hdr = 
+CXLDeviceState *cxlds;
+CXLType3Dev *dcd;
+uint8_t flags = 1 << CXL_EVENT_TYPE_INFO;
+uint32_t num_extents = 0;
+CXLDCExtentRecordList *list = records;
+CXLDCExtent_raw *extents;
+uint64_t dpa, len;
+uint8_t rid;
+int i;
+
+if (!obj) {
+error_setg(errp, "Unable to resolve path");
+return;
+}
+if (!object_dynamic_cast(obj, TYPE_CXL_TYPE3)) {
+error_setg(errp, "Path not point to a valid CXL type3 device");
+return;
+}
+
+dcd = CXL_TYPE3(obj);
+cxlds = >cxl_dstate;
+memset(, 0, sizeof(dCap));
+
+if (!dcd->dc.num_regions) {
+error_setg(errp, "No dynamic capacity support from the device");
+return;
+}
+
+while (list) {
+dpa = list->value->dpa * 1024 * 1024;
+len = list->value->len * 1024 * 1024;
+rid = list->value->region_id;
+
+if (rid >= dcd->dc.num_regions) {
+error_setg(errp, "region id is too large");
+return;
+}
+
+if (dpa % dcd->dc.regions[rid].block_size
+|| len % dcd->dc.regions[rid].block_size) {
+error_setg(errp, "dpa or len is not aligned to region block size");
+return;
+}
+
+if (dpa + len > dcd->dc.regions[rid].decode_len * 256 * 1024 * 1024) {
+error_setg(errp, "extent range is beyond the region end");
+return;
+}
+
+num_extents++;
+list = list->next;
+}
+
+i = 0;
+list = records;
+extents = g_new0(CXLDCExtent_raw, num_extents);
+while (list) {
+dpa = list->value->dpa * 1024 * 1024;
+len = list->value->len * 1024 * 1024;
+rid = list->value->region_id;
+
+extents[i].start_dpa = dpa + dcd->dc.regions[rid].base;
+extents[i].len = len;
+memset(extents[i].tag, 0, 0x10);
+extents[i].shared_seq = 0;
+
+list = list->next;
+i++;
+}
+
+/*
+ * 8.2.9.1.5
+ * All Dynamic Capacity event records shall set the Event Record
+ * Severity fie

[Qemu PATCH RESEND 4/9] hw/mem/cxl_type3: Add support to create DC regions to type3 memory devices

[Fan Ni]

From: Fan Ni 

With the change, when setting up memory for type3 memory device, we can
create DC regions
A property 'num-dc-regions' is added to ct3_props to allow users to pass the
number of DC regions to create. To make it easier, other region parameters
like region base, length, and block size are hard coded. If needed,
these parameters can be added easily.

With the change, we can create DC regions with proper kernel side
support as below:

region=$(cat /sys/bus/cxl/devices/decoder0.0/create_dc_region)
echo $region> /sys/bus/cxl/devices/decoder0.0/create_dc_region
echo 256 > /sys/bus/cxl/devices/$region/interleave_granularity
echo 1 > /sys/bus/cxl/devices/$region/interleave_ways

echo "dc0" >/sys/bus/cxl/devices/decoder2.0/mode
echo 0x4000 >/sys/bus/cxl/devices/decoder2.0/dpa_size

echo 0x4000 > /sys/bus/cxl/devices/$region/size
echo  "decoder2.0" > /sys/bus/cxl/devices/$region/target0
echo 1 > /sys/bus/cxl/devices/$region/commit
echo $region > /sys/bus/cxl/drivers/cxl_region/bind

Signed-off-by: Fan Ni 
---
 hw/mem/cxl_type3.c | 33 +
 1 file changed, 33 insertions(+)

diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 237b544b9c..27b5920f7d 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -707,6 +707,34 @@ static void ct3d_reg_write(void *opaque, hwaddr offset, 
uint64_t value,
 }
 }
 
+/*
+ * Create a dc region to test "Get Dynamic Capacity Configuration" command.
+ */
+static int cxl_create_dc_regions(CXLType3Dev *ct3d)
+{
+int i;
+uint64_t region_base = (ct3d->hostvmem ? ct3d->hostvmem->size : 0)
++ (ct3d->hostpmem ? ct3d->hostpmem->size : 0);
+uint64_t region_len = (uint64_t)2 * 1024 * 1024 * 1024;
+uint64_t decode_len = 4; /* 4*256MB */
+uint64_t blk_size = 2 * 1024 * 1024;
+struct CXLDCD_Region *region;
+
+for (i = 0; i < ct3d->dc.num_regions; i++) {
+region = >dc.regions[i];
+region->base = region_base;
+region->decode_len = decode_len;
+region->len = region_len;
+region->block_size = blk_size;
+/* dsmad_handle is set when creating cdat table entries */
+region->flags = 0;
+
+region_base += region->len;
+}
+
+return 0;
+}
+
 static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
 {
 DeviceState *ds = DEVICE(ct3d);
@@ -775,6 +803,10 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error 
**errp)
 g_free(p_name);
 }
 
+if (cxl_create_dc_regions(ct3d)) {
+return false;
+}
+
 return true;
 }
 
@@ -1062,6 +1094,7 @@ static Property ct3_props[] = {
 DEFINE_PROP_UINT64("sn", CXLType3Dev, sn, UI64_NULL),
 DEFINE_PROP_STRING("cdat", CXLType3Dev, cxl_cstate.cdat.filename),
 DEFINE_PROP_UINT16("spdm", CXLType3Dev, spdm_port, 0),
+DEFINE_PROP_UINT8("num-dc-regions", CXLType3Dev, dc.num_regions, 0),
 DEFINE_PROP_END_OF_LIST(),
 };
 
-- 
2.25.1

[Qemu PATCH RESEND 5/9] hw/mem/cxl_type3: Add host backend and address space handling for DC regions

[Fan Ni]

From: Fan Ni 

Add (file/memory backed) host backend, all the dynamic capacity regions
will share a single, large enough host backend. Set up address space for
DC regions to support read/write operations to dynamic capacity for DCD.

With the change, following supports are added:
1. add a new property to type3 device "nonvolatile-dc-memdev" to point to host
   memory backend for dynamic capacity;
2. add namespace for dynamic capacity for read/write support;
3. create cdat entries for each dynamic capacity region;
4. fix dvsec range registers to include DC regions.

Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c  |  19 +++-
 hw/mem/cxl_type3.c  | 203 +---
 include/hw/cxl/cxl_device.h |   4 +
 3 files changed, 185 insertions(+), 41 deletions(-)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 59d57ae245..c497298a1d 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -388,9 +388,11 @@ static CXLRetCode cmd_firmware_update_get_info(struct 
cxl_cmd *cmd,
 char fw_rev4[0x10];
 } QEMU_PACKED *fw_info;
 QEMU_BUILD_BUG_ON(sizeof(*fw_info) != 0x50);
+CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
 
 if ((cxl_dstate->vmem_size < CXL_CAPACITY_MULTIPLIER) ||
-(cxl_dstate->pmem_size < CXL_CAPACITY_MULTIPLIER)) {
+(cxl_dstate->pmem_size < CXL_CAPACITY_MULTIPLIER) ||
+(ct3d->dc.total_capacity < CXL_CAPACITY_MULTIPLIER)) {
 return CXL_MBOX_INTERNAL_ERROR;
 }
 
@@ -531,7 +533,8 @@ static CXLRetCode cmd_identify_memory_device(struct cxl_cmd 
*cmd,
 CXLType3Class *cvc = CXL_TYPE3_GET_CLASS(ct3d);
 
 if ((!QEMU_IS_ALIGNED(cxl_dstate->vmem_size, CXL_CAPACITY_MULTIPLIER)) ||
-(!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER))) {
+(!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER)) ||
+(!QEMU_IS_ALIGNED(ct3d->dc.total_capacity, CXL_CAPACITY_MULTIPLIER))) {
 return CXL_MBOX_INTERNAL_ERROR;
 }
 
@@ -566,9 +569,11 @@ static CXLRetCode cmd_ccls_get_partition_info(struct 
cxl_cmd *cmd,
 uint64_t next_pmem;
 } QEMU_PACKED *part_info = (void *)cmd->payload;
 QEMU_BUILD_BUG_ON(sizeof(*part_info) != 0x20);
+CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
 
 if ((!QEMU_IS_ALIGNED(cxl_dstate->vmem_size, CXL_CAPACITY_MULTIPLIER)) ||
-(!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER))) {
+(!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER)) ||
+(!QEMU_IS_ALIGNED(ct3d->dc.total_capacity, CXL_CAPACITY_MULTIPLIER))) {
 return CXL_MBOX_INTERNAL_ERROR;
 }
 
@@ -880,7 +885,13 @@ static CXLRetCode cmd_media_clear_poison(struct cxl_cmd 
*cmd,
 struct clear_poison_pl *in = (void *)cmd->payload;
 
 dpa = ldq_le_p(>dpa);
-if (dpa + CXL_CACHE_LINE_SIZE > cxl_dstate->static_mem_size) {
+if (dpa + CXL_CACHE_LINE_SIZE >= cxl_dstate->static_mem_size
+&& ct3d->dc.num_regions == 0) {
+return CXL_MBOX_INVALID_PA;
+}
+
+if (ct3d->dc.num_regions && dpa + CXL_CACHE_LINE_SIZE >=
+cxl_dstate->static_mem_size + ct3d->dc.total_capacity) {
 return CXL_MBOX_INVALID_PA;
 }
 
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 27b5920f7d..af1d919be3 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -20,6 +20,7 @@
 #include "hw/pci/spdm.h"
 
 #define DWORD_BYTE 4
+#define CXL_CAPACITY_MULTIPLIER   (256 * MiB)
 
 /* Default CDAT entries for a memory region */
 enum {
@@ -33,8 +34,8 @@ enum {
 };
 
 static int ct3_build_cdat_entries_for_mr(CDATSubHeader **cdat_table,
- int dsmad_handle, MemoryRegion *mr,
- bool is_pmem, uint64_t dpa_base)
+int dsmad_handle, uint8_t flags,
+uint64_t dpa_base, uint64_t size)
 {
 g_autofree CDATDsmas *dsmas = NULL;
 g_autofree CDATDslbis *dslbis0 = NULL;
@@ -53,9 +54,9 @@ static int ct3_build_cdat_entries_for_mr(CDATSubHeader 
**cdat_table,
 .length = sizeof(*dsmas),
 },
 .DSMADhandle = dsmad_handle,
-.flags = is_pmem ? CDAT_DSMAS_FLAG_NV : 0,
+.flags = flags,
 .DPA_base = dpa_base,
-.DPA_length = memory_region_size(mr),
+.DPA_length = size,
 };
 
 /* For now, no memory side cache, plausiblish numbers */
@@ -137,9 +138,9 @@ static int ct3_build_cdat_entries_for_mr(CDATSubHeader 
**cdat_table,
  * NV: Reserved - the non volatile from DSMAS matters
  * V: EFI_MEMORY_SP
  */
-.EFI_memory_type_attr = is_pmem ? 2 : 1,
+.EFI_memory_type_attr = flags ? 2 : 1,
 .DPA_offset = 0,
-.DPA_length = memory_region_size(mr),
+.DPA_length = size,

[Qemu PATCH RESEND 3/9] include/hw/cxl/cxl_device: Rename mem_size as static_mem_size for type3 memory devices

[Fan Ni]

From: Fan Ni 

Rename mem_size as static_mem_size for type3 memdev to cover static RAM and
pmem capacity, preparing for the introduction of dynamic capacity to support
dynamic capacity devices.

Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c  | 5 +++--
 hw/mem/cxl_type3.c  | 8 
 include/hw/cxl/cxl_device.h | 2 +-
 3 files changed, 8 insertions(+), 7 deletions(-)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 41e43ec231..59d57ae245 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -540,7 +540,8 @@ static CXLRetCode cmd_identify_memory_device(struct cxl_cmd 
*cmd,
 
 snprintf(id->fw_revision, 0x10, "BWFW VERSION %02d", 0);
 
-stq_le_p(>total_capacity, cxl_dstate->mem_size / 
CXL_CAPACITY_MULTIPLIER);
+stq_le_p(>total_capacity,
+cxl_dstate->static_mem_size / CXL_CAPACITY_MULTIPLIER);
 stq_le_p(>persistent_capacity, cxl_dstate->pmem_size / 
CXL_CAPACITY_MULTIPLIER);
 stq_le_p(>volatile_capacity, cxl_dstate->vmem_size / 
CXL_CAPACITY_MULTIPLIER);
 stl_le_p(>lsa_size, cvc->get_lsa_size(ct3d));
@@ -879,7 +880,7 @@ static CXLRetCode cmd_media_clear_poison(struct cxl_cmd 
*cmd,
 struct clear_poison_pl *in = (void *)cmd->payload;
 
 dpa = ldq_le_p(>dpa);
-if (dpa + CXL_CACHE_LINE_SIZE > cxl_dstate->mem_size) {
+if (dpa + CXL_CACHE_LINE_SIZE > cxl_dstate->static_mem_size) {
 return CXL_MBOX_INVALID_PA;
 }
 
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 862107c5ef..237b544b9c 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -748,7 +748,7 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error 
**errp)
 }
 address_space_init(>hostvmem_as, vmr, v_name);
 ct3d->cxl_dstate.vmem_size = memory_region_size(vmr);
-ct3d->cxl_dstate.mem_size += memory_region_size(vmr);
+ct3d->cxl_dstate.static_mem_size += memory_region_size(vmr);
 g_free(v_name);
 }
 
@@ -771,7 +771,7 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error 
**errp)
 }
 address_space_init(>hostpmem_as, pmr, p_name);
 ct3d->cxl_dstate.pmem_size = memory_region_size(pmr);
-ct3d->cxl_dstate.mem_size += memory_region_size(pmr);
+ct3d->cxl_dstate.static_mem_size += memory_region_size(pmr);
 g_free(p_name);
 }
 
@@ -984,7 +984,7 @@ static int cxl_type3_hpa_to_as_and_dpa(CXLType3Dev *ct3d,
 return -EINVAL;
 }
 
-if (*dpa_offset > ct3d->cxl_dstate.mem_size) {
+if (*dpa_offset > ct3d->cxl_dstate.static_mem_size) {
 return -EINVAL;
 }
 
@@ -1142,7 +1142,7 @@ static bool set_cacheline(CXLType3Dev *ct3d, uint64_t 
dpa_offset, uint8_t *data)
 return false;
 }
 
-if (dpa_offset + CXL_CACHE_LINE_SIZE > ct3d->cxl_dstate.mem_size) {
+if (dpa_offset + CXL_CACHE_LINE_SIZE > ct3d->cxl_dstate.static_mem_size) {
 return false;
 }
 
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index bf564f4a0b..a32ee6d6ba 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -209,7 +209,7 @@ typedef struct cxl_device_state {
 } timestamp;
 
 /* memory region size, HDM */
-uint64_t mem_size;
+uint64_t static_mem_size;
 uint64_t pmem_size;
 uint64_t vmem_size;
 
-- 
2.25.1

[Qemu PATCH RESEND 6/9] hw/mem/cxl_type3: Add DC extent list representative and get DC extent list mailbox support

[Fan Ni]

From: Fan Ni 

Add dynamic capacity extent list representative to the definition of
CXLType3Dev and add get DC extent list mailbox command per
CXL.spec.3.0:.8.2.9.8.9.2.

Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c  | 71 +
 hw/mem/cxl_type3.c  |  1 +
 include/hw/cxl/cxl_device.h | 23 
 3 files changed, 95 insertions(+)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index c497298a1d..754ab68b78 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -83,6 +83,7 @@ enum {
 #define CLEAR_POISON   0x2
 DCD_CONFIG  = 0x48, /*r3.0: 8.2.9.8.9*/
 #define GET_DC_CONFIG  0x0
+#define GET_DYN_CAP_EXT_LIST   0x1
 PHYSICAL_SWITCH = 0x51
 #define IDENTIFY_SWITCH_DEVICE  0x0
 };
@@ -1018,6 +1019,73 @@ static CXLRetCode cmd_dcd_get_dyn_cap_config(struct 
cxl_cmd *cmd,
 return CXL_MBOX_SUCCESS;
 }
 
+/*
+ * cxl spec 3.0: 8.2.9.8.9.2
+ * Get Dynamic Capacity Extent List (Opcode 4810h)
+ */
+static CXLRetCode cmd_dcd_get_dyn_cap_ext_list(struct cxl_cmd *cmd,
+CXLDeviceState *cxl_dstate,
+uint16_t *len)
+{
+struct get_dyn_cap_ext_list_in_pl {
+uint32_t extent_cnt;
+uint32_t start_extent_id;
+} QEMU_PACKED;
+
+struct get_dyn_cap_ext_list_out_pl {
+uint32_t count;
+uint32_t total_extents;
+uint32_t generation_num;
+uint8_t rsvd[4];
+CXLDCExtent_raw records[];
+} QEMU_PACKED;
+
+struct get_dyn_cap_ext_list_in_pl *in = (void *)cmd->payload;
+struct get_dyn_cap_ext_list_out_pl *out = (void *)cmd->payload;
+struct CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev,
+cxl_dstate);
+uint16_t record_count = 0, i = 0, record_done = 0;
+CXLDCDExtentList *extent_list = >dc.extents;
+CXLDCD_Extent *ent;
+uint16_t out_pl_len;
+uint32_t start_extent_id = in->start_extent_id;
+
+if (start_extent_id > ct3d->dc.total_extent_count) {
+return CXL_MBOX_INVALID_INPUT;
+}
+
+record_count = MIN(in->extent_cnt,
+ct3d->dc.total_extent_count - start_extent_id);
+
+out_pl_len = sizeof(*out) + record_count * sizeof(out->records[0]);
+/* May need more processing here in the future */
+assert(out_pl_len <= CXL_MAILBOX_MAX_PAYLOAD_SIZE);
+
+memset(out, 0, out_pl_len);
+stl_le_p(>count, record_count);
+stl_le_p(>total_extents, ct3d->dc.total_extent_count);
+stl_le_p(>generation_num, ct3d->dc.ext_list_gen_seq);
+
+if (record_count > 0) {
+QTAILQ_FOREACH(ent, extent_list, node) {
+if (i++ < start_extent_id) {
+continue;
+}
+stq_le_p(>records[record_done].start_dpa, ent->start_dpa);
+stq_le_p(>records[record_done].len, ent->len);
+memcpy(>records[record_done].tag, ent->tag, 0x10);
+stw_le_p(>records[record_done].shared_seq, ent->shared_seq);
+record_done++;
+if (record_done == record_count) {
+break;
+}
+}
+}
+
+*len = out_pl_len;
+return CXL_MBOX_SUCCESS;
+}
+
 #define IMMEDIATE_CONFIG_CHANGE (1 << 1)
 #define IMMEDIATE_DATA_CHANGE (1 << 2)
 #define IMMEDIATE_POLICY_CHANGE (1 << 3)
@@ -1058,6 +1126,9 @@ static struct cxl_cmd cxl_cmd_set[256][256] = {
 cmd_media_clear_poison, 72, 0 },
 [DCD_CONFIG][GET_DC_CONFIG] = { "DCD_GET_DC_CONFIG",
 cmd_dcd_get_dyn_cap_config, 2, 0 },
+[DCD_CONFIG][GET_DYN_CAP_EXT_LIST] = {
+"DCD_GET_DYNAMIC_CAPACITY_EXTENT_LIST", cmd_dcd_get_dyn_cap_ext_list,
+8, 0 },
 };
 
 static struct cxl_cmd cxl_cmd_set_sw[256][256] = {
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index af1d919be3..608063ac52 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -789,6 +789,7 @@ static int cxl_create_dc_regions(CXLType3Dev *ct3d)
 
 region_base += region->len;
 }
+QTAILQ_INIT(>dc.extents);
 
 return 0;
 }
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index ddb24271a8..a9cfe4e904 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -384,6 +384,25 @@ typedef QLIST_HEAD(, CXLPoison) CXLPoisonList;
 
 #define DCD_MAX_REGION_NUM 8
 
+typedef struct CXLDCD_Extent_raw {
+uint64_t start_dpa;
+uint64_t len;
+uint8_t tag[0x10];
+uint16_t shared_seq;
+uint8_t rsvd[0x6];
+} QEMU_PACKED CXLDCExtent_raw;
+
+typedef struct CXLDCD_Extent {
+uint64_t start_dpa;
+uint64_t len;
+uint8_t tag[0x10];
+uint16_t shared_seq;
+uint8_t rsvd[0x6];
+
+QTAILQ_ENTRY(CXLDCD_Extent) node;
+} CXLDCD_Extent;
+typedef QTAILQ_HEAD(, CXLDCD_Extent) CXLDCDExtentList;
+
 typedef struct CXLDCD_Region {
 uint64_t base;
 uint64_t decode_len; /* in multiples 

[Qemu PATCH RESEND 7/9] hw/cxl/cxl-mailbox-utils: Add mailbox commands to support add/release dynamic capacity response

[Fan Ni]

From: Fan Ni 

Per CXL spec 3.0, two mailbox commands are implemented:
Add Dynamic Capacity Response (Opcode 4802h) 8.2.9.8.9.3, and
Release Dynamic Capacity (Opcode 4803h) 8.2.9.8.9.4.

Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c  | 253 
 include/hw/cxl/cxl_device.h |   3 +-
 2 files changed, 255 insertions(+), 1 deletion(-)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 754ab68b78..d547385ba7 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -84,6 +84,8 @@ enum {
 DCD_CONFIG  = 0x48, /*r3.0: 8.2.9.8.9*/
 #define GET_DC_CONFIG  0x0
 #define GET_DYN_CAP_EXT_LIST   0x1
+#define ADD_DYN_CAP_RSP0x2
+#define RELEASE_DYN_CAP0x3
 PHYSICAL_SWITCH = 0x51
 #define IDENTIFY_SWITCH_DEVICE  0x0
 };
@@ -1086,6 +1088,251 @@ static CXLRetCode cmd_dcd_get_dyn_cap_ext_list(struct 
cxl_cmd *cmd,
 return CXL_MBOX_SUCCESS;
 }
 
+/*
+ * Check whether the bits at addr between [nr, nr+size) are all set,
+ * return 1 if all 1s, else return 0
+ */
+static inline int test_bits(const unsigned long *addr, int nr, int size)
+{
+unsigned long res = find_next_zero_bit(addr, size + nr, nr);
+
+return (res >= nr + size) ? 1 : 0;
+}
+
+/*
+ * Find dynamic capacity region id based on dpa range [dpa, dpa+len)
+ */
+static uint8_t find_region_id(struct CXLType3Dev *dev, uint64_t dpa,
+uint64_t len)
+{
+int8_t i = dev->dc.num_regions - 1;
+
+while (i > 0 && dpa < dev->dc.regions[i].base) {
+i--;
+}
+
+if (dpa < dev->dc.regions[i].base
+|| dpa + len > dev->dc.regions[i].base + dev->dc.regions[i].len) {
+return dev->dc.num_regions;
+}
+
+return i;
+}
+
+static void insert_extent_to_extent_list(CXLDCDExtentList *list, uint64_t dpa,
+uint64_t len, uint8_t *tag, uint16_t shared_seq)
+{
+CXLDCD_Extent *extent;
+extent = g_new0(CXLDCD_Extent, 1);
+extent->start_dpa = dpa;
+extent->len = len;
+if (tag) {
+memcpy(extent->tag, tag, 0x10);
+} else {
+memset(extent->tag, 0, 0x10);
+}
+extent->shared_seq = shared_seq;
+
+QTAILQ_INSERT_TAIL(list, extent, node);
+}
+
+typedef struct updated_dc_extent_list_in_pl {
+uint32_t num_entries_updated;
+uint8_t rsvd[4];
+struct { /* r3.0: Table 8-130 */
+uint64_t start_dpa;
+uint64_t len;
+uint8_t rsvd[8];
+} QEMU_PACKED updated_entries[];
+} QEMU_PACKED updated_dc_extent_list_in_pl;
+
+/*
+ * The function only check the input extent list against itself.
+ */
+static CXLRetCode detect_malformed_extent_list(CXLType3Dev *dev,
+const updated_dc_extent_list_in_pl *in)
+{
+unsigned long *blk_bitmap;
+uint64_t min_block_size = dev->dc.regions[0].block_size;
+struct CXLDCD_Region *region = >dc.regions[0];
+uint32_t i;
+uint64_t dpa, len;
+uint8_t rid;
+CXLRetCode ret;
+
+for (i = 1; i < dev->dc.num_regions; i++) {
+region = >dc.regions[i];
+if (min_block_size > region->block_size) {
+min_block_size = region->block_size;
+}
+}
+
+blk_bitmap = bitmap_new((region->len + region->base
+- dev->dc.regions[0].base) / min_block_size);
+
+for (i = 0; i < in->num_entries_updated; i++) {
+dpa = in->updated_entries[i].start_dpa;
+len = in->updated_entries[i].len;
+
+rid = find_region_id(dev, dpa, len);
+if (rid == dev->dc.num_regions) {
+ret = CXL_MBOX_INVALID_PA;
+goto out;
+}
+
+region = >dc.regions[rid];
+if (dpa % region->block_size || len % region->block_size) {
+ret = CXL_MBOX_INVALID_EXTENT_LIST;
+goto out;
+}
+/* the dpa range already covered by some other extents in the list */
+if (test_bits(blk_bitmap, dpa / min_block_size, len / min_block_size)) 
{
+ret = CXL_MBOX_INVALID_EXTENT_LIST;
+goto out;
+}
+bitmap_set(blk_bitmap, dpa / min_block_size, len / min_block_size);
+   }
+
+ret = CXL_MBOX_SUCCESS;
+
+out:
+g_free(blk_bitmap);
+return ret;
+}
+
+/*
+ * cxl spec 3.0: 8.2.9.8.9.3
+ * Add Dynamic Capacity Response (opcode 4802h)
+ * Assume an extent is added only after the response is processed successfully
+ * TODO: for better extent list validation, a better solution would be
+ * maintaining a pending extent list and use it to verify the extent list in
+ * the response.
+ */
+static CXLRetCode cmd_dcd_add_dyn_cap_rsp(struct cxl_cmd *cmd,
+CXLDeviceState *cxl_dstate, uint16_t *len_unused)
+{
+updated_dc_extent_list_in_pl *in = (void *)cmd->payload;
+struct CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev,
+cxl_dstate);
+CXLDCDExtentLi

[Qemu PATCH RESEND 2/9] hw/cxl/cxl-mailbox-utils: Add dynamic capacity region representative and mailbox command support

[Fan Ni]

From: Fan Ni 

Per cxl spec 3.0, add dynamic capacity region representative based on
Table 8-126 and extend the cxl type3 device definition to include dc region
information. Also, based on info in 8.2.9.8.9.1, add 'Get Dynamic Capacity
Configuration' mailbox support.

Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c  | 69 +
 include/hw/cxl/cxl_device.h | 16 +
 2 files changed, 85 insertions(+)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index b013e30314..41e43ec231 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -81,6 +81,8 @@ enum {
 #define GET_POISON_LIST0x0
 #define INJECT_POISON  0x1
 #define CLEAR_POISON   0x2
+DCD_CONFIG  = 0x48, /*r3.0: 8.2.9.8.9*/
+#define GET_DC_CONFIG  0x0
 PHYSICAL_SWITCH = 0x51
 #define IDENTIFY_SWITCH_DEVICE  0x0
 };
@@ -939,6 +941,71 @@ static CXLRetCode cmd_media_clear_poison(struct cxl_cmd 
*cmd,
 return CXL_MBOX_SUCCESS;
 }
 
+/*
+ * cxl spec 3.0: 8.2.9.8.9.1
+ * Get Dynamic Capacity Configuration
+ **/
+static CXLRetCode cmd_dcd_get_dyn_cap_config(struct cxl_cmd *cmd,
+CXLDeviceState *cxl_dstate,
+uint16_t *len)
+{
+struct get_dyn_cap_config_in_pl {
+uint8_t region_cnt;
+uint8_t start_region_id;
+} QEMU_PACKED;
+
+struct get_dyn_cap_config_out_pl {
+uint8_t num_regions;
+uint8_t rsvd1[7];
+struct {
+uint64_t base;
+uint64_t decode_len;
+uint64_t region_len;
+uint64_t block_size;
+uint32_t dsmadhandle;
+uint8_t flags;
+uint8_t rsvd2[3];
+} QEMU_PACKED records[];
+} QEMU_PACKED;
+
+struct get_dyn_cap_config_in_pl *in = (void *)cmd->payload;
+struct get_dyn_cap_config_out_pl *out = (void *)cmd->payload;
+struct CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev,
+cxl_dstate);
+uint16_t record_count = 0, i;
+uint16_t out_pl_len;
+uint8_t start_region_id = in->start_region_id;
+
+if (start_region_id >= ct3d->dc.num_regions) {
+return CXL_MBOX_INVALID_INPUT;
+}
+
+record_count = MIN(ct3d->dc.num_regions - in->start_region_id,
+in->region_cnt);
+
+out_pl_len = sizeof(*out) + record_count * sizeof(out->records[0]);
+assert(out_pl_len <= CXL_MAILBOX_MAX_PAYLOAD_SIZE);
+
+memset(out, 0, out_pl_len);
+out->num_regions = record_count;
+for (i = 0; i < record_count; i++) {
+stq_le_p(>records[i].base,
+ct3d->dc.regions[start_region_id + i].base);
+stq_le_p(>records[i].decode_len,
+ct3d->dc.regions[start_region_id + i].decode_len);
+stq_le_p(>records[i].region_len,
+ct3d->dc.regions[start_region_id + i].len);
+stq_le_p(>records[i].block_size,
+ct3d->dc.regions[start_region_id + i].block_size);
+stl_le_p(>records[i].dsmadhandle,
+ct3d->dc.regions[start_region_id + i].dsmadhandle);
+out->records[i].flags = ct3d->dc.regions[start_region_id + i].flags;
+}
+
+*len = out_pl_len;
+return CXL_MBOX_SUCCESS;
+}
+
 #define IMMEDIATE_CONFIG_CHANGE (1 << 1)
 #define IMMEDIATE_DATA_CHANGE (1 << 2)
 #define IMMEDIATE_POLICY_CHANGE (1 << 3)
@@ -977,6 +1044,8 @@ static struct cxl_cmd cxl_cmd_set[256][256] = {
 cmd_media_inject_poison, 8, 0 },
 [MEDIA_AND_POISON][CLEAR_POISON] = { "MEDIA_AND_POISON_CLEAR_POISON",
 cmd_media_clear_poison, 72, 0 },
+[DCD_CONFIG][GET_DC_CONFIG] = { "DCD_GET_DC_CONFIG",
+cmd_dcd_get_dyn_cap_config, 2, 0 },
 };
 
 static struct cxl_cmd cxl_cmd_set_sw[256][256] = {
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index cd7f28dba8..bf564f4a0b 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -382,6 +382,17 @@ typedef struct CXLPoison {
 typedef QLIST_HEAD(, CXLPoison) CXLPoisonList;
 #define CXL_POISON_LIST_LIMIT 256
 
+#define DCD_MAX_REGION_NUM 8
+
+typedef struct CXLDCD_Region {
+uint64_t base;
+uint64_t decode_len; /* in multiples of 256MB */
+uint64_t len;
+uint64_t block_size;
+uint32_t dsmadhandle;
+uint8_t flags;
+} CXLDCD_Region;
+
 struct CXLType3Dev {
 /* Private */
 PCIDevice parent_obj;
@@ -413,6 +424,11 @@ struct CXLType3Dev {
 unsigned int poison_list_cnt;
 bool poison_list_overflowed;
 uint64_t poison_list_overflow_ts;
+
+struct dynamic_capacity {
+uint8_t num_regions; /* 0-8 regions */
+struct CXLDCD_Region regions[DCD_MAX_REGION_NUM];
+} dc;
 };
 
 #define TYPE_CXL_TYPE3 "cxl-type3"
-- 
2.25.1

[Qemu PATCH RESEND 9/9] hw/mem/cxl_type3: Add dpa range validation for accesses to dc regions

[Fan Ni]

From: Fan Ni 

Not all dpa range in the dc regions is valid to access until an extent
covering the range has been added. Add a bitmap for each region to
record whether a dc block in the region has been backed by dc extent.
For the bitmap, a bit in the bitmap represents a dc block. When a dc
extent is added, all the bits of the blocks in the extent will be set,
which will be cleared when the extent is released.

Signed-off-by: Fan Ni 
---
 hw/mem/cxl_type3.c  | 155 
 include/hw/cxl/cxl_device.h |   1 +
 2 files changed, 156 insertions(+)

diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index cb1f9182e6..e673287804 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -787,13 +787,37 @@ static int cxl_create_dc_regions(CXLType3Dev *ct3d)
 /* dsmad_handle is set when creating cdat table entries */
 region->flags = 0;
 
+region->blk_bitmap = bitmap_new(region->len / region->block_size);
+if (!region->blk_bitmap) {
+break;
+}
+
 region_base += region->len;
 }
+
+if (i < ct3d->dc.num_regions) {
+while (--i >= 0) {
+g_free(ct3d->dc.regions[i].blk_bitmap);
+}
+return -1;
+}
+
 QTAILQ_INIT(>dc.extents);
 
 return 0;
 }
 
+static void cxl_destroy_dc_regions(CXLType3Dev *ct3d)
+{
+int i;
+struct CXLDCD_Region *region;
+
+for (i = 0; i < ct3d->dc.num_regions; i++) {
+region = >dc.regions[i];
+g_free(region->blk_bitmap);
+}
+}
+
 static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
 {
 DeviceState *ds = DEVICE(ct3d);
@@ -1021,6 +1045,7 @@ err_free_special_ops:
 g_free(regs->special_ops);
 err_address_space_free:
 if (ct3d->dc.host_dc) {
+cxl_destroy_dc_regions(ct3d);
 address_space_destroy(>dc.host_dc_as);
 }
 if (ct3d->hostpmem) {
@@ -1043,6 +1068,7 @@ static void ct3_exit(PCIDevice *pci_dev)
 spdm_sock_fini(ct3d->doe_spdm.socket);
 g_free(regs->special_ops);
 if (ct3d->dc.host_dc) {
+cxl_destroy_dc_regions(ct3d);
 address_space_destroy(>dc.host_dc_as);
 }
 if (ct3d->hostpmem) {
@@ -1053,6 +1079,110 @@ static void ct3_exit(PCIDevice *pci_dev)
 }
 }
 
+/*
+ * This function will marked the dpa range [dpa, dap + len) to be backed and
+ * accessible, this happens when a dc extent is added and accepted by the
+ * host.
+ */
+static void set_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa,
+uint64_t len)
+{
+int i;
+CXLDCD_Region *region = >dc.regions[0];
+
+if (dpa < region->base
+|| dpa >= region->base + ct3d->dc.total_capacity)
+return;
+
+/*
+ * spec 3.0 9.13.3: Regions are used in increasing-DPA order, with
+ * Region 0 being used for the lowest DPA of Dynamic Capacity and
+ * Region 7 for the highest DPA.
+ * So we check from the last region to find where the dpa belongs.
+ * access across multiple regions is not allowed.
+ **/
+for (i = ct3d->dc.num_regions - 1; i >= 0; i--) {
+region = >dc.regions[i];
+if (dpa >= region->base) {
+break;
+}
+}
+
+bitmap_set(region->blk_bitmap, (dpa - region->base) / region->block_size,
+len / region->block_size);
+}
+
+/*
+ * This function check whether a dpa range [dpa, dpa + len) has been backed
+ * with dc extents, used when validating read/write to dc regions
+ */
+static bool test_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa,
+uint64_t len)
+{
+int i;
+CXLDCD_Region *region = >dc.regions[0];
+uint64_t nbits;
+long nr;
+
+if (dpa < region->base
+|| dpa >= region->base + ct3d->dc.total_capacity)
+return false;
+
+/*
+ * spec 3.0 9.13.3: Regions are used in increasing-DPA order, with
+ * Region 0 being used for the lowest DPA of Dynamic Capacity and
+ * Region 7 for the highest DPA.
+ * So we check from the last region to find where the dpa belongs.
+ * access across multiple regions is not allowed.
+ */
+for (i = ct3d->dc.num_regions - 1; i >= 0; i--) {
+region = >dc.regions[i];
+if (dpa >= region->base) {
+break;
+}
+}
+
+nr = (dpa - region->base) / region->block_size;
+nbits = len / region->block_size;
+return find_next_zero_bit(region->blk_bitmap, nbits, nr) >= nr + nbits;
+}
+
+/*
+ * This function will marked the dpa range [dpa, dap + len) to be unbacked and
+ * inaccessible, this happens when a dc extent is added and accepted by the
+ * host.
+ */
+static void clear_region_block_backed(CXLType3Dev *ct3d, uint64_t dpa,
+uint64_t len)
+{
+int i;
+CXLDCD_Region *region = >dc.regions[0];
+uint64_t nbits;
+long nr;
+
+if (dpa &l

[Qemu PATCH RESEND 0/9] Enabling DCD emulation support in Qemu

[Fan Ni]

The patch series provides dynamic capacity device (DCD) emulation in Qemu.
More specifically, it provides the following functionalities:
1. Extended type3 memory device to support DC regions and extents.
2. Implemented DCD related mailbox command support in CXL r3.0: 8.2.9.8.9.
3. ADD QMP interfaces for adding and releasing DC extents to simulate FM
functions for DCD described in cxl r3.0: 7.6.7.6.5 and 7.6.7.6.6.
4. Add new ct3d properties for DCD devices (host backend, number of dc
regions, etc.)
5. Add read/write support from/to DC regions of the device.
6. Add mechanism to validate accessed to DC region address space.

A more detailed description can be found from the previously posted RFC[1].

Compared to the previously posted RFC[1], following changes have been made:
1. Rebased the code on top of Jonathan's branch
https://gitlab.com/jic23/qemu/-/tree/cxl-2023-05-25.
2. Extracted the rename of mem_size to a separated patch.(Jonathan)
3. Reordered the patch series to improve its readability.(Jonathan)
4. Split the validation of accesses to DC region address space as a separate
patch.
5. Redesigned the QMP interfaces for adding and releasing DC extents to make
them easier to understand and act like existing QMP interfaces (like the
interface for cxl-inject-uncorrectable-errors). (Jonathan)
6. Updated dvsec range register setting to support DCD devices without static
capacity.
7. Fixed issues mentioned in the comments (Jonathan Fontenot).
8. Fixed the format issue and checked with checkpatch.pl under qemu code dir.


The code is tested with the DCD patch series at the kernel side[2]. The test
is similar to those mentioned in the cover letter of [1].


[1]: https://lore.kernel.org/all/20230511175609.2091136-1-fan...@samsung.com/
[2]: 
https://lore.kernel.org/linux-cxl/649da378c28a3_968bb29420@iweiny-mobl.notmuch/T/#t

Fan Ni (9):
  hw/cxl/cxl-mailbox-utils: Add dc_event_log_size field to output
payload of identify memory device command
  hw/cxl/cxl-mailbox-utils: Add dynamic capacity region representative
and mailbox command support
  include/hw/cxl/cxl_device: Rename mem_size as static_mem_size for
type3 memory devices
  hw/mem/cxl_type3: Add support to create DC regions to type3 memory
devices
  hw/mem/cxl_type3: Add host backend and address space handling for DC
regions
  hw/mem/cxl_type3: Add DC extent list representative and get DC extent
list mailbox support
  hw/cxl/cxl-mailbox-utils: Add mailbox commands to support add/release
dynamic capacity response
  hw/cxl/events: Add qmp interfaces to add/release dynamic capacity
extents
  hw/mem/cxl_type3: Add dpa range validation for accesses to dc regions

 hw/cxl/cxl-mailbox-utils.c  | 421 +++-
 hw/mem/cxl_type3.c  | 539 +---
 hw/mem/cxl_type3_stubs.c|   6 +
 include/hw/cxl/cxl_device.h |  49 +++-
 include/hw/cxl/cxl_events.h |  16 ++
 qapi/cxl.json   |  49 
 6 files changed, 1034 insertions(+), 46 deletions(-)

-- 
2.25.1

Re: [Qemu RFC 0/7] Early enabling of DCD emulation in Qemu

[Fan Ni]

On Wed, Jun 07, 2023 at 06:13:01PM +, Shesha Bhushan Sreenivasamurthy wrote:
> Hi Fan,
>I am implementing DCD FMAPI commands and planning to start pushing changes 
> to the below branch. That requires the contributions you have made. Can your 
> changes be pushed to the below branch ?
> 
> https://urldefense.com/v3/__https://gitlab.com/jic23/qemu/-/tree/cxl-2023-05-25__;!!EwVzqGoTKBqv-0DWAJBm!Vt5uIqwW-L4c4gh02ulI4M762JNQ3_aE9k9lb6QlwE2xm6T23ic7ig7Y77i1VN7l_RX_ySIQhre_z7Q0JA$
>  

Can you push changes to the branch directly? I think it is Jonathan's private
branch. However, I can fork the branch and rebase my patch series atop and
share with you the new repo if that helps you move forward your
work.
Let me know your thought.

Fan

> 
> 
> From: Fan Ni 
> Sent: Monday, June 5, 2023 10:51 AM
> To: Ira Weiny 
> Cc: qemu-devel@nongnu.org ; 
> jonathan.came...@huawei.com ; 
> linux-...@vger.kernel.org ; 
> gregory.pr...@memverge.com ; 
> hch...@avery-design.com.tw ; 
> cbr...@avery-design.com ; dan.j.willi...@intel.com 
> ; Adam Manzanares ; 
> d...@stgolabs.net ; nmtadam.sams...@gmail.com 
> ; ni...@outlook.com 
> Subject: Re: [Qemu RFC 0/7] Early enabling of DCD emulation in Qemu 
>  
> On Mon, Jun 05, 2023 at 10:35:48AM -0700, Ira Weiny wrote:
> > Fan Ni wrote:
> > > Since the early draft of DCD support in kernel is out
> > > (https://urldefense.com/v3/__https://lore.kernel.org/linux-cxl/20230417164126.GA1904906@bgt-140510-bm03/T/*t__;Iw!!EwVzqGoTKBqv-0DWAJBm!RHzXPIcSiGsqUciUIH6HnlG_W--4L5CHfvcOIeUFdwKFhAujXuFDxjymmpCdOu7SLr61rww7lr21LzAGNOk$
> > >  ),
> > > this patch series provide dcd emulation in qemu so people who are 
> > > interested
> > > can have an early try. It is noted that the patch series may need to be 
> > > updated
> > > accordingly if the kernel side implementation changes.
> > 
> > Fan,
> > 
> > Do you have a git tree we can pull this from which is updated to a more
> > recent CXL branch from Jonathan?
> > 
> > Thanks,
> > Ira
> 
> Hi Ira,
> 
> I have a git tree of the patch series based on Jonathan's branch
> cxl-2023-02-28: 
> https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_moking_qemu-2Ddev_tree_dcd-2Drfe=DwIFAg=nKjWec2b6R0mOyPaz7xtfQ=Zta64bwn4nurTRpD4LY2OGr8KklkMRPn7Z_Qy0o4unU=w6dicn5kXEG4Imk6TpICIjdA6KJ-xt84dtHui-Y0fv5H13bijtzEvjxECKE5MHYf=3yeO9RN5FY3gPfO2y19X057YeqRTTQTQNfNA-Gfir_Q=
>  .
> 
> That may be not new enough to include some of the recent patches, but I can
> rebase it to a newer branch if you can tell me which branch you want to use.
> 
> Thanks,
> Fan
> 
> > 
> > > 
> > > To support DCD emulation, the patch series add DCD related mailbox command
> > > support (CXL Spec 3.0: 8.2.9.8.9), and extend the cxl type3 memory device
> > > with dynamic capacity extent and region representative.
> > > To support read/write to the dynamic capacity of the device, a host 
> > > backend
> > > is provided and necessary check mechnism is added to ensure the dynamic
> > > capacity accessed is backed with active dc extents.
> > > Currently FM related mailbox commands (cxl spec 3.0: 7.6.7.6) is not 
> > > supported
> > > , but we add two qmp interfaces for adding/releasing dynamic capacity 
> > > extents.
> > > Also, the support for multiple hosts sharing the same DCD case is missing.
> > > 
> > > Things we can try with the patch series together with kernel dcd code:
> > > 1. Create DC regions to cover the address range of the dynamic capacity
> > > regions.
> > > 2. Add/release dynamic capacity extents to the device and notify the
> > > kernel.
> > > 3. Test kernel side code to accept added dc extents and create dax 
> > > devices,
> > > and release dc extents and notify the device
> > > 4. Online the memory range backed with dc extents and let application use
> > > them.
> > > 
> > > The patch series is based on Jonathan's local qemu branch:
> > > https://urldefense.com/v3/__https://gitlab.com/jic23/qemu/-/tree/cxl-2023-02-28__;!!EwVzqGoTKBqv-0DWAJBm!RHzXPIcSiGsqUciUIH6HnlG_W--4L5CHfvcOIeUFdwKFhAujXuFDxjymmpCdOu7SLr61rww7lr21OO3UHEM$
> > >  
> > > 
> > > Simple tests peformed with the patch series:
> > > 1 Install cxl modules:
> > > 
> > > modprobe -a cxl_acpi cxl_core cxl_pci cxl_port cxl_mem
> > > 
> > > 2 Create dc regions:
> > > 
> > > region=$(cat /sys/bus/cxl/devices/decoder0.0/create_dc_region)
> > > echo $region> /sys/bus/cxl/devices/decoder0.0/create_dc_region
> > > ec

Re: [Qemu RFC 0/7] Early enabling of DCD emulation in Qemu

[Fan Ni]

On Mon, Jun 05, 2023 at 10:35:48AM -0700, Ira Weiny wrote:
> Fan Ni wrote:
> > Since the early draft of DCD support in kernel is out
> > (https://urldefense.com/v3/__https://lore.kernel.org/linux-cxl/20230417164126.GA1904906@bgt-140510-bm03/T/*t__;Iw!!EwVzqGoTKBqv-0DWAJBm!RHzXPIcSiGsqUciUIH6HnlG_W--4L5CHfvcOIeUFdwKFhAujXuFDxjymmpCdOu7SLr61rww7lr21LzAGNOk$
> >  ),
> > this patch series provide dcd emulation in qemu so people who are interested
> > can have an early try. It is noted that the patch series may need to be 
> > updated
> > accordingly if the kernel side implementation changes.
> 
> Fan,
> 
> Do you have a git tree we can pull this from which is updated to a more
> recent CXL branch from Jonathan?
> 
> Thanks,
> Ira

Hi Ira,

I have a git tree of the patch series based on Jonathan's branch
cxl-2023-02-28: https://github.com/moking/qemu-dev/tree/dcd-rfe.

That may be not new enough to include some of the recent patches, but I can
rebase it to a newer branch if you can tell me which branch you want to use.

Thanks,
Fan

> 
> > 
> > To support DCD emulation, the patch series add DCD related mailbox command
> > support (CXL Spec 3.0: 8.2.9.8.9), and extend the cxl type3 memory device
> > with dynamic capacity extent and region representative.
> > To support read/write to the dynamic capacity of the device, a host backend
> > is provided and necessary check mechnism is added to ensure the dynamic
> > capacity accessed is backed with active dc extents.
> > Currently FM related mailbox commands (cxl spec 3.0: 7.6.7.6) is not 
> > supported
> > , but we add two qmp interfaces for adding/releasing dynamic capacity 
> > extents.
> > Also, the support for multiple hosts sharing the same DCD case is missing.
> > 
> > Things we can try with the patch series together with kernel dcd code:
> > 1. Create DC regions to cover the address range of the dynamic capacity
> > regions.
> > 2. Add/release dynamic capacity extents to the device and notify the
> > kernel.
> > 3. Test kernel side code to accept added dc extents and create dax devices,
> > and release dc extents and notify the device
> > 4. Online the memory range backed with dc extents and let application use
> > them.
> > 
> > The patch series is based on Jonathan's local qemu branch:
> > https://urldefense.com/v3/__https://gitlab.com/jic23/qemu/-/tree/cxl-2023-02-28__;!!EwVzqGoTKBqv-0DWAJBm!RHzXPIcSiGsqUciUIH6HnlG_W--4L5CHfvcOIeUFdwKFhAujXuFDxjymmpCdOu7SLr61rww7lr21OO3UHEM$
> >  
> > 
> > Simple tests peformed with the patch series:
> > 1 Install cxl modules:
> > 
> > modprobe -a cxl_acpi cxl_core cxl_pci cxl_port cxl_mem
> > 
> > 2 Create dc regions:
> > 
> > region=$(cat /sys/bus/cxl/devices/decoder0.0/create_dc_region)
> > echo $region> /sys/bus/cxl/devices/decoder0.0/create_dc_region
> > echo 256 > /sys/bus/cxl/devices/$region/interleave_granularity
> > echo 1 > /sys/bus/cxl/devices/$region/interleave_ways
> > echo "dc" >/sys/bus/cxl/devices/decoder2.0/mode
> > echo 0x1000 >/sys/bus/cxl/devices/decoder2.0/dpa_size
> > echo 0x1000 > /sys/bus/cxl/devices/$region/size
> > echo  "decoder2.0" > /sys/bus/cxl/devices/$region/target0
> > echo 1 > /sys/bus/cxl/devices/$region/commit
> > echo $region > /sys/bus/cxl/drivers/cxl_region/bind
> > 
> > /home/fan/cxl/tools-and-scripts# cxl list
> > [
> >   {
> > "memdevs":[
> >   {
> > "memdev":"mem0",
> > "pmem_size":536870912,
> > "ram_size":0,
> > "serial":0,
> > "host":":0d:00.0"
> >   }
> > ]
> >   },
> >   {
> > "regions":[
> >   {
> > "region":"region0",
> > "resource":45365592064,
> > "size":268435456,
> > "interleave_ways":1,
> > "interleave_granularity":256,
> > "decode_state":"commit"
> >   }
> > ]
> >   }
> > ]
> > 
> > 3 Add two dc extents (128MB each) through qmp interface
> > 
> > { "execute": "qmp_capabilities" }
> > 
> > { "execute": "cxl-add-dynamic-capacity-event",
> > "arguments": {
> >  "path": "/machine/peripheral/cxl-pmem0",
> > "region-id" : 0,
> >  "num-extent":

Re: [PATCH v7 6/7] hw/cxl/events: Add injection of DRAM events

[Fan Ni]

The 05/22/2023 16:09, Jonathan Cameron wrote:
> Defined in CXL r3.0 8.2.9.2.1.2 DRAM Event Record, this event
> provides information related to DRAM devices.
> 
> Example injection command in QMP:
> 
> { "execute": "cxl-inject-dram-event",
> "arguments": {
> "path": "/machine/peripheral/cxl-mem0",
> "log": "informational",
> "flags": 1,
> "dpa": 1000,
> "descriptor": 3,
> "type": 3,
> "transaction-type": 192,
> "channel": 3,
> "rank": 17,
> "nibble-mask": 37421234,
> "bank-group": 7,
> "bank": 11,
> "row": 2,
> "column": 77,
> "correction-mask": [33, 44, 55,66]
> }}
> 
> Reviewed-by: Ira Weiny 
> Signed-off-by: Jonathan Cameron 
> 

Reviewed-by: Fan Ni 

> ---
> v7: Additional documentation, plus rename physaddr to dpa reduce
> confusion
> ---
>  qapi/cxl.json   |  63 
>  include/hw/cxl/cxl_events.h |  23 +++
>  hw/mem/cxl_type3.c  | 116 
>  hw/mem/cxl_type3_stubs.c|  13 
>  4 files changed, 215 insertions(+)
> 
> diff --git a/qapi/cxl.json b/qapi/cxl.json
> index 7700e26a0d..ce9adcbc55 100644
> --- a/qapi/cxl.json
> +++ b/qapi/cxl.json
> @@ -84,6 +84,69 @@
>  '*channel': 'uint8', '*rank': 'uint8',
>  '*device': 'uint32', '*component-id': 'str' } }
>  
> +##
> +# @cxl-inject-dram-event:
> +#
> +# Inject an event record for a DRAM Event (CXL r3.0 8.2.9.2.1.2)
> +# This event type is reported via one of the event logs specified via
> +# the log parameter.
> +#
> +# @path: CXL type 3 device canonical QOM path
> +#
> +# @log: Event log to add the event to
> +#
> +# @flags: Event Record Flags. See CXL r3.0 Table 8-42 Common Event
> +# Record Format, Event Record Flags for subfield definitions.
> +#
> +# @dpa: Device Physical Address (relative to @path device). Note lower
> +#   bits include some flags. See CXL r3.0 Table 8-44 DRAM Event
> +#   Record, Physical Address.
> +#
> +# @descriptor: Memory Event Descriptor with additional memory
> +#  event information. See CXL r3.0 Table 8-44 DRAM Event
> +#  Record, Memory Event Descriptor for bit definitions.
> +#
> +# @type: Type of memory event that occurred. See CXL r3.0 Table 8-44
> +#DRAM Event Record, Memory Event Type for possible values.
> +#
> +# @transaction-type: Type of first transaction that caused the event
> +#to occur. See CXL r3.0 Table 8-44 DRAM Event
> +#Record, Transaction Type for possible values.
> +#
> +# @channel: The channel of the memory event location.  A channel is
> +#   an interface that can be independently accessed for a
> +#   transaction.
> +#
> +# @rank: The rank of the memory event location. A rank is a set of
> +#memory devices on a channel that together execute a
> +#transaction.
> +#
> +# @nibble-mask: Identify one or more nibbles that the error affects
> +#
> +# @bank-group: Bank group of the memory event location, incorporating
> +#  a number of Banks.
> +#
> +# @bank: Bank of the memory event location. A single bank is accessed
> +#per read or write of the memory.
> +#
> +# @row: Row address within the DRAM.
> +#
> +# @column: Column address within the DRAM.
> +#
> +# @correction-mask: Bits within each nibble. Used in order of bits set
> +#   in the nibble-mask.  Up to 4 nibbles may be covered.
> +#
> +# Since: 8.1
> +##
> +{ 'command': 'cxl-inject-dram-event',
> +  'data': { 'path': 'str', 'log': 'CxlEventLog', 'flags': 'uint8',
> +'dpa': 'uint64', 'descriptor': 'uint8',
> +'type': 'uint8', 'transaction-type': 'uint8',
> +'*channel': 'uint8', '*rank': 'uint8', '*nibble-mask': 'uint32',
> +'*bank-group': 'uint8', '*bank': 'uint8', '*row': 'uint32',
> +'*column': 'uint16', '*correction-mask': [ 'uint64' ]
> +   }}
> +
>  ##
>  # @cxl-inject-poison:
>  #
> diff --git a/include/hw/cxl/cxl_events.h b/include/hw/cxl/cxl_events.h
> index b189193f4c..a39e30d973 100644
> --- a/include/hw/cxl/cxl_events.h
> +++ b/include/hw/cxl/cxl_events.h
> @@ -123,4 +123,27 @@ typedef struct CXLEventGenMedia {
>  uint8_t reserved[CXL_EVENT_GEN_MED_RES_SIZE];
>  } QEMU_PACKED CXLEventGenMedia;
>  
> +/*
> + * DRAM Event Rec

Re: [PATCH v7 7/7] hw/cxl/events: Add injection of Memory Module Events

[Fan Ni]

The 05/22/2023 16:09, Jonathan Cameron wrote:
> These events include a copy of the device health information at the
> time of the event. Actually using the emulated device health would
> require a lot of controls to manipulate that state.  Given the aim
> of this injection code is to just test the flows when events occur,
> inject the contents of the device health state as well.
> 
> Future work may add more sophisticate device health emulation
> including direct generation of these records when events occur
> (such as a temperature threshold being crossed).  That does not
> reduce the usefulness of this more basic generation of the events.
> 
> Reviewed-by: Ira Weiny 
> Signed-off-by: Jonathan Cameron 
> 

Reviewed-by: Fan Ni 

> ---
> v7: Expanded docs for qapi and added a lot of cross references to
> the CXL revision 3.0 specification.
> ---
>  qapi/cxl.json   | 54 
>  include/hw/cxl/cxl_events.h | 19 
>  hw/mem/cxl_type3.c  | 62 +
>  hw/mem/cxl_type3_stubs.c| 12 +++
>  4 files changed, 147 insertions(+)
> 
> diff --git a/qapi/cxl.json b/qapi/cxl.json
> index ce9adcbc55..05c560cfe5 100644
> --- a/qapi/cxl.json
> +++ b/qapi/cxl.json
> @@ -147,6 +147,60 @@
>  '*column': 'uint16', '*correction-mask': [ 'uint64' ]
> }}
>  
> +##
> +# @cxl-inject-memory-module-event:
> +#
> +# Inject an event record for a Memory Module Event (CXL r3.0
> +# 8.2.9.2.1.3). # This event includes a copy of the Device Health
> +# info at the time of the event.
> +#
> +# @path: CXL type 3 device canonical QOM path
> +#
> +# @log: Event Log to add the event to
> +#
> +# @flags: Event Record Flags. See CXL r3.0 Table 8-42 Common Event
> +# Record Format, Event Record Flags for subfield definitions.
> +#
> +# @type: Device Event Type. See CXL r3.0 Table 8-45 Memory Module
> +#Event Record for bit definitions for bit definiions.
> +#
> +# @health-status: Overall health summary bitmap. See CXL r3.0 Table
> +# 8-100 Get Health Info Output Payload, Health Status
> +# for bit definitions.
> +#
> +# @media-status: Overall media health summary. See CXL r3.0 Table
> +#8-100 Get Health Info Output Payload, Media Status
> +#for bit definitions.
> +#
> +# @additional-status: See CXL r3.0 Table 8-100 Get Health Info Output
> +# Payload, Additional Status for subfield
> +# definitions.
> +#
> +# @life-used: Percentage (0-100) of factory expected life span.
> +#
> +# @temperature: Device temperature in degrees Celsius.
> +#
> +# @dirty-shutdown-count: Number of time the device has been unable to
> +#determine whether data loss may have occurred.
> +#
> +# @corrected-volatile-error-count: Total number of correctable errors in
> +#  volatile memory.
> +#
> +# @corrected-persistent-error-count: Total number correctable errors in
> +#persistent memory
> +#
> +# Since: 8.1
> +##
> +{ 'command': 'cxl-inject-memory-module-event',
> +  'data': { 'path': 'str', 'log': 'CxlEventLog', 'flags' : 'uint8',
> +'type': 'uint8', 'health-status': 'uint8',
> +'media-status': 'uint8', 'additional-status': 'uint8',
> +'life-used': 'uint8', 'temperature' : 'int16',
> +'dirty-shutdown-count': 'uint32',
> +'corrected-volatile-error-count': 'uint32',
> +'corrected-persistent-error-count': 'uint32'
> +}}
> +
>  ##
>  # @cxl-inject-poison:
>  #
> diff --git a/include/hw/cxl/cxl_events.h b/include/hw/cxl/cxl_events.h
> index a39e30d973..089ba2091f 100644
> --- a/include/hw/cxl/cxl_events.h
> +++ b/include/hw/cxl/cxl_events.h
> @@ -146,4 +146,23 @@ typedef struct CXLEventDram {
>  uint8_t reserved[0x17];
>  } QEMU_PACKED CXLEventDram;
>  
> +/*
> + * Memory Module Event Record
> + * CXL Rev 3.0 Section 8.2.9.2.1.3: Table 8-45
> + * All fields little endian.
> + */
> +typedef struct CXLEventMemoryModule {
> +CXLEventRecordHdr hdr;
> +uint8_t type;
> +uint8_t health_status;
> +uint8_t media_status;
> +uint8_t additional_status;
> +uint8_t life_used;
> +int16_t temperature;
> +uint32_t dirty_shutdown_count;
> +uint32_t corrected_volatile_error_count;
> +uint32_t corrected_persistent_error_count;
> +uint8_t reserved[0x3d];
> +} QEMU_PACKED CXLEventMemoryModule;
> +
>  #endif /* CXL_EVENTS_H */
> diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
> index

Re: [PATCH v7 5/7] hw/cxl/events: Add injection of General Media Events

[Fan Ni]

The 05/22/2023 16:09, Jonathan Cameron wrote:
> From: Ira Weiny 
> 
> To facilitate testing provide a QMP command to inject a general media
> event.  The event can be added to the log specified.
> 
> Signed-off-by: Ira Weiny 
> Signed-off-by: Jonathan Cameron 
> 
> ---

Reviewed-by: Fan Ni 

> v7: Various docs updates and field renames including a lot more
> specification references.
> ---
>  qapi/cxl.json   |  79 +
>  include/hw/cxl/cxl_events.h |  20 +++
>  hw/mem/cxl_type3.c  | 111 
>  hw/mem/cxl_type3_stubs.c|  10 
>  4 files changed, 220 insertions(+)
> 
> diff --git a/qapi/cxl.json b/qapi/cxl.json
> index 4849fca776..7700e26a0d 100644
> --- a/qapi/cxl.json
> +++ b/qapi/cxl.json
> @@ -5,6 +5,85 @@
>  # = CXL devices
>  ##
>  
> +##
> +# @CxlEventLog:
> +#
> +# CXL has a number of separate event logs for different types of
> +# events. Each such event log is handled and signaled independently.
> +#
> +# @informational: Information Event Log
> +#
> +# @warning: Warning Event Log
> +#
> +# @failure: Failure Event Log
> +#
> +# @fatal: Fatal Event Log
> +#
> +# Since: 8.1
> +##
> +{ 'enum': 'CxlEventLog',
> +  'data': ['informational',
> +   'warning',
> +   'failure',
> +   'fatal']
> + }
> +
> +##
> +# @cxl-inject-general-media-event:
> +#
> +# Inject an event record for a General Media Event (CXL r3.0
> +# 8.2.9.2.1.1) This event type is reported via one of the event logs
> +# specified via the log parameter.
> +#
> +# @path: CXL type 3 device canonical QOM path
> +#
> +# @log: event log to add the event to
> +#
> +# @flags: Event Record Flags. See CXL r3.0 Table 8-42 Common Event
> +# Record Format, Event Record Flags for subfield definitions.
> +#
> +# @dpa: Device Physical Address (relative to @path device). Note lower
> +#   bits include some flags. See CXL r3.0 Table 8-43 General Media
> +#   Event Record, Physical Address.
> +#
> +# @descriptor: Memory Event Descriptor with additional memory
> +#  event information. See CXL r3.0 Table 8-43 General
> +#  Media Event Record, Memory Event Descriptor for bit
> +#  definitions.
> +#
> +# @type: Type of memory event that occurred. See CXL r3.0 Table 8-43
> +#General Media Event Record, Memory Event Type for possible
> +#values.
> +#
> +# @transaction-type: Type of first transaction that caused the event
> +#to occur. See CXL r3.0 Table 8-43 General Media
> +#Event Record, Transaction Type for possible
> +#values.
> +#
> +# @channel: The channel of the memory event location. A channel is
> +#   an interface that can be independently accessed for a
> +#   transaction.
> +#
> +# @rank: The rank of the memory event location. A rank is a set of
> +#memory devices on a channel that together execute a
> +#transaction.
> +#
> +# @device: Bitmask that represents all devices in the rank associated
> +#  with the memory event location.
> +#
> +# @component-id: Device specific component identifier for the event.
> +#May describe a field replaceable sub-component of
> +#the device.
> +#
> +# Since: 8.1
> +##
> +{ 'command': 'cxl-inject-general-media-event',
> +  'data': { 'path': 'str', 'log': 'CxlEventLog', 'flags': 'uint8',
> +'dpa': 'uint64', 'descriptor': 'uint8',
> +'type': 'uint8', 'transaction-type': 'uint8',
> +'*channel': 'uint8', '*rank': 'uint8',
> +'*device': 'uint32', '*component-id': 'str' } }
> +
>  ##
>  # @cxl-inject-poison:
>  #
> diff --git a/include/hw/cxl/cxl_events.h b/include/hw/cxl/cxl_events.h
> index 4bf8b7aa08..b189193f4c 100644
> --- a/include/hw/cxl/cxl_events.h
> +++ b/include/hw/cxl/cxl_events.h
> @@ -103,4 +103,24 @@ typedef struct CXLEventInterruptPolicy {
>  /* DCD is optional but other fields are not */
>  #define CXL_EVENT_INT_SETTING_MIN_LEN 4
>  
> +/*
> + * General Media Event Record
> + * CXL rev 3.0 Section 8.2.9.2.1.1; Table 8-43
> + */
> +#define CXL_EVENT_GEN_MED_COMP_ID_SIZE  0x10
> +#define CXL_EVENT_GEN_MED_RES_SIZE  0x2e
> +typedef struct CXLEventGenMedia {
> +CXLEventRecordHdr hdr;
> +uint64_t phys_addr;
> +uint8_t descriptor;
> +uint8_t type;
> +uint8_t transaction_type;
> +uint16_t validity_flags;
> +uint8_t channel;
> +uint8_t rank;
> +uint8_t device[3];
> +uint8_t component_id[CXL_EVENT_GEN_MED_COMP_ID_SIZE];
>

Re: [PATCH v7 4/7] hw/cxl/events: Add event interrupt support

[Fan Ni]

The 05/22/2023 16:09, Jonathan Cameron wrote:
> From: Ira Weiny 
> 
> Replace the stubbed out CXL Get/Set Event interrupt policy mailbox
> commands.  Enable those commands to control interrupts for each of the
> event log types.
> 
> Skip the standard input mailbox length on the Set command due to DCD
> being optional.  Perform the checks separately.
> 
> Signed-off-by: Ira Weiny 
> Signed-off-by: Jonathan Cameron 
> ---

Reviewed-by: Fan Ni 

>  include/hw/cxl/cxl_device.h |   6 +-
>  include/hw/cxl/cxl_events.h |  23 
>  hw/cxl/cxl-events.c |  33 ++-
>  hw/cxl/cxl-mailbox-utils.c  | 106 +---
>  hw/mem/cxl_type3.c  |   4 +-
>  5 files changed, 147 insertions(+), 25 deletions(-)
> 
> diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
> index d3aec1bc0e..1978730fba 100644
> --- a/include/hw/cxl/cxl_device.h
> +++ b/include/hw/cxl/cxl_device.h
> @@ -121,6 +121,8 @@ typedef struct CXLEventLog {
>  uint16_t overflow_err_count;
>  uint64_t first_overflow_timestamp;
>  uint64_t last_overflow_timestamp;
> +bool irq_enabled;
> +int irq_vec;
>  QemuMutex lock;
>  QSIMPLEQ_HEAD(, CXLEvent) events;
>  } CXLEventLog;
> @@ -369,7 +371,7 @@ MemTxResult cxl_type3_write(PCIDevice *d, hwaddr 
> host_addr, uint64_t data,
>  
>  uint64_t cxl_device_get_timestamp(CXLDeviceState *cxlds);
>  
> -void cxl_event_init(CXLDeviceState *cxlds);
> +void cxl_event_init(CXLDeviceState *cxlds, int start_msg_num);
>  bool cxl_event_insert(CXLDeviceState *cxlds, CXLEventLogType log_type,
>CXLEventRecordRaw *event);
>  CXLRetCode cxl_event_get_records(CXLDeviceState *cxlds, CXLGetEventPayload 
> *pl,
> @@ -378,6 +380,8 @@ CXLRetCode cxl_event_get_records(CXLDeviceState *cxlds, 
> CXLGetEventPayload *pl,
>  CXLRetCode cxl_event_clear_records(CXLDeviceState *cxlds,
> CXLClearEventPayload *pl);
>  
> +void cxl_event_irq_assert(CXLType3Dev *ct3d);
> +
>  void cxl_set_poison_list_overflowed(CXLType3Dev *ct3d);
>  
>  #endif
> diff --git a/include/hw/cxl/cxl_events.h b/include/hw/cxl/cxl_events.h
> index d4aaa894f1..4bf8b7aa08 100644
> --- a/include/hw/cxl/cxl_events.h
> +++ b/include/hw/cxl/cxl_events.h
> @@ -80,4 +80,27 @@ typedef struct CXLClearEventPayload {
>  uint16_t handle[];
>  } CXLClearEventPayload;
>  
> +/**
> + * Event Interrupt Policy
> + *
> + * CXL rev 3.0 section 8.2.9.2.4; Table 8-52
> + */
> +typedef enum CXLEventIntMode {
> +CXL_INT_NONE = 0x00,
> +CXL_INT_MSI_MSIX = 0x01,
> +CXL_INT_FW   = 0x02,
> +CXL_INT_RES  = 0x03,
> +} CXLEventIntMode;
> +#define CXL_EVENT_INT_MODE_MASK 0x3
> +#define CXL_EVENT_INT_SETTING(vector) uint8_t)vector & 0xf) << 4) | 
> CXL_INT_MSI_MSIX)
> +typedef struct CXLEventInterruptPolicy {
> +uint8_t info_settings;
> +uint8_t warn_settings;
> +uint8_t failure_settings;
> +uint8_t fatal_settings;
> +uint8_t dyn_cap_settings;
> +} QEMU_PACKED CXLEventInterruptPolicy;
> +/* DCD is optional but other fields are not */
> +#define CXL_EVENT_INT_SETTING_MIN_LEN 4
> +
>  #endif /* CXL_EVENTS_H */
> diff --git a/hw/cxl/cxl-events.c b/hw/cxl/cxl-events.c
> index 5da1b76b97..d161d57456 100644
> --- a/hw/cxl/cxl-events.c
> +++ b/hw/cxl/cxl-events.c
> @@ -13,6 +13,8 @@
>  #include "qemu/bswap.h"
>  #include "qemu/typedefs.h"
>  #include "qemu/error-report.h"
> +#include "hw/pci/msi.h"
> +#include "hw/pci/msix.h"
>  #include "hw/cxl/cxl.h"
>  #include "hw/cxl/cxl_events.h"
>  
> @@ -26,7 +28,7 @@ static void reset_overflow(CXLEventLog *log)
>  log->last_overflow_timestamp = 0;
>  }
>  
> -void cxl_event_init(CXLDeviceState *cxlds)
> +void cxl_event_init(CXLDeviceState *cxlds, int start_msg_num)
>  {
>  CXLEventLog *log;
>  int i;
> @@ -37,9 +39,16 @@ void cxl_event_init(CXLDeviceState *cxlds)
>  log->overflow_err_count = 0;
>  log->first_overflow_timestamp = 0;
>  log->last_overflow_timestamp = 0;
> +log->irq_enabled = false;
> +log->irq_vec = start_msg_num++;
>  qemu_mutex_init(>lock);
>  QSIMPLEQ_INIT(>events);
>  }
> +
> +/* Override -- Dynamic Capacity uses the same vector as info */
> +cxlds->event_logs[CXL_EVENT_TYPE_DYNAMIC_CAP].irq_vec =
> +  cxlds->event_logs[CXL_EVENT_TYPE_INFO].irq_vec;
> +
>  }
>  
>  static CXLEvent *cxl_event_get_head(CXLEventLog *log)
> @@ -215,3 +224,25 @@ CXLRetCode cxl_even

Re: [PATCH v7 2/7] hw/cxl: Move CXLRetCode definition to cxl_device.h

[Fan Ni]

The 05/22/2023 16:09, Jonathan Cameron wrote:
> Following patches will need access to the mailbox return code
> type so move it to the header.
> 
> Reviewed-by: Ira Weiny 
> Signed-off-by: Jonathan Cameron 
> ---

Reviewed-by: Fan Ni 

>  include/hw/cxl/cxl_device.h | 28 
>  hw/cxl/cxl-mailbox-utils.c  | 28 
>  2 files changed, 28 insertions(+), 28 deletions(-)
> 
> diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
> index 16993f7098..9f8ee85f8a 100644
> --- a/include/hw/cxl/cxl_device.h
> +++ b/include/hw/cxl/cxl_device.h
> @@ -83,6 +83,34 @@
>  (CXL_DEVICE_CAP_REG_SIZE + CXL_DEVICE_STATUS_REGISTERS_LENGTH + \
>   CXL_MAILBOX_REGISTERS_LENGTH + CXL_MEMORY_DEVICE_REGISTERS_LENGTH)
>  
> +/* 8.2.8.4.5.1 Command Return Codes */
> +typedef enum {
> +CXL_MBOX_SUCCESS = 0x0,
> +CXL_MBOX_BG_STARTED = 0x1,
> +CXL_MBOX_INVALID_INPUT = 0x2,
> +CXL_MBOX_UNSUPPORTED = 0x3,
> +CXL_MBOX_INTERNAL_ERROR = 0x4,
> +CXL_MBOX_RETRY_REQUIRED = 0x5,
> +CXL_MBOX_BUSY = 0x6,
> +CXL_MBOX_MEDIA_DISABLED = 0x7,
> +CXL_MBOX_FW_XFER_IN_PROGRESS = 0x8,
> +CXL_MBOX_FW_XFER_OUT_OF_ORDER = 0x9,
> +CXL_MBOX_FW_AUTH_FAILED = 0xa,
> +CXL_MBOX_FW_INVALID_SLOT = 0xb,
> +CXL_MBOX_FW_ROLLEDBACK = 0xc,
> +CXL_MBOX_FW_REST_REQD = 0xd,
> +CXL_MBOX_INVALID_HANDLE = 0xe,
> +CXL_MBOX_INVALID_PA = 0xf,
> +CXL_MBOX_INJECT_POISON_LIMIT = 0x10,
> +CXL_MBOX_PERMANENT_MEDIA_FAILURE = 0x11,
> +CXL_MBOX_ABORTED = 0x12,
> +CXL_MBOX_INVALID_SECURITY_STATE = 0x13,
> +CXL_MBOX_INCORRECT_PASSPHRASE = 0x14,
> +CXL_MBOX_UNSUPPORTED_MAILBOX = 0x15,
> +CXL_MBOX_INVALID_PAYLOAD_LENGTH = 0x16,
> +CXL_MBOX_MAX = 0x17
> +} CXLRetCode;
> +
>  typedef struct cxl_device_state {
>  MemoryRegion device_registers;
>  
> diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
> index e3401b6be8..d7e114aaae 100644
> --- a/hw/cxl/cxl-mailbox-utils.c
> +++ b/hw/cxl/cxl-mailbox-utils.c
> @@ -68,34 +68,6 @@ enum {
>  #define CLEAR_POISON   0x2
>  };
>  
> -/* 8.2.8.4.5.1 Command Return Codes */
> -typedef enum {
> -CXL_MBOX_SUCCESS = 0x0,
> -CXL_MBOX_BG_STARTED = 0x1,
> -CXL_MBOX_INVALID_INPUT = 0x2,
> -CXL_MBOX_UNSUPPORTED = 0x3,
> -CXL_MBOX_INTERNAL_ERROR = 0x4,
> -CXL_MBOX_RETRY_REQUIRED = 0x5,
> -CXL_MBOX_BUSY = 0x6,
> -CXL_MBOX_MEDIA_DISABLED = 0x7,
> -CXL_MBOX_FW_XFER_IN_PROGRESS = 0x8,
> -CXL_MBOX_FW_XFER_OUT_OF_ORDER = 0x9,
> -CXL_MBOX_FW_AUTH_FAILED = 0xa,
> -CXL_MBOX_FW_INVALID_SLOT = 0xb,
> -CXL_MBOX_FW_ROLLEDBACK = 0xc,
> -CXL_MBOX_FW_REST_REQD = 0xd,
> -CXL_MBOX_INVALID_HANDLE = 0xe,
> -CXL_MBOX_INVALID_PA = 0xf,
> -CXL_MBOX_INJECT_POISON_LIMIT = 0x10,
> -CXL_MBOX_PERMANENT_MEDIA_FAILURE = 0x11,
> -CXL_MBOX_ABORTED = 0x12,
> -CXL_MBOX_INVALID_SECURITY_STATE = 0x13,
> -CXL_MBOX_INCORRECT_PASSPHRASE = 0x14,
> -CXL_MBOX_UNSUPPORTED_MAILBOX = 0x15,
> -CXL_MBOX_INVALID_PAYLOAD_LENGTH = 0x16,
> -CXL_MBOX_MAX = 0x17
> -} CXLRetCode;
> -
>  struct cxl_cmd;
>  typedef CXLRetCode (*opcode_handler)(struct cxl_cmd *cmd,
> CXLDeviceState *cxl_dstate, uint16_t 
> *len);
> -- 
> 2.39.2
> 

-- 
Fan Ni 

Re: [PATCH v7 3/7] hw/cxl/events: Wire up get/clear event mailbox commands

[Fan Ni]

The 05/22/2023 16:09, Jonathan Cameron wrote:
> From: Ira Weiny 
> 
> CXL testing is benefited from an artificial event log injection
> mechanism.
> 
> Add an event log infrastructure to insert, get, and clear events from
> the various logs available on a device.
> 
> Replace the stubbed out CXL Get/Clear Event mailbox commands with
> commands that operate on the new infrastructure.
> 
> Signed-off-by: Ira Weiny 
> Signed-off-by: Jonathan Cameron 
> ---

Reviewed-by: Fan Ni 

See comments below in cxl_event_insert.

>  include/hw/cxl/cxl_device.h |  25 +
>  include/hw/cxl/cxl_events.h |  55 +
>  hw/cxl/cxl-events.c | 217 
>  hw/cxl/cxl-mailbox-utils.c  |  40 ++-
>  hw/mem/cxl_type3.c  |   1 +
>  hw/cxl/meson.build  |   1 +
>  6 files changed, 337 insertions(+), 2 deletions(-)
> 
> diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
> index 9f8ee85f8a..d3aec1bc0e 100644
> --- a/include/hw/cxl/cxl_device.h
> +++ b/include/hw/cxl/cxl_device.h
> @@ -111,6 +111,20 @@ typedef enum {
>  CXL_MBOX_MAX = 0x17
>  } CXLRetCode;
>  
> +typedef struct CXLEvent {
> +CXLEventRecordRaw data;
> +QSIMPLEQ_ENTRY(CXLEvent) node;
> +} CXLEvent;
> +
> +typedef struct CXLEventLog {
> +uint16_t next_handle;
> +uint16_t overflow_err_count;
> +uint64_t first_overflow_timestamp;
> +uint64_t last_overflow_timestamp;
> +QemuMutex lock;
> +QSIMPLEQ_HEAD(, CXLEvent) events;
> +} CXLEventLog;
> +
>  typedef struct cxl_device_state {
>  MemoryRegion device_registers;
>  
> @@ -161,6 +175,8 @@ typedef struct cxl_device_state {
>  uint64_t mem_size;
>  uint64_t pmem_size;
>  uint64_t vmem_size;
> +
> +CXLEventLog event_logs[CXL_EVENT_TYPE_MAX];
>  } CXLDeviceState;
>  
>  /* Initialize the register block for a device */
> @@ -353,6 +369,15 @@ MemTxResult cxl_type3_write(PCIDevice *d, hwaddr 
> host_addr, uint64_t data,
>  
>  uint64_t cxl_device_get_timestamp(CXLDeviceState *cxlds);
>  
> +void cxl_event_init(CXLDeviceState *cxlds);
> +bool cxl_event_insert(CXLDeviceState *cxlds, CXLEventLogType log_type,
> +  CXLEventRecordRaw *event);
> +CXLRetCode cxl_event_get_records(CXLDeviceState *cxlds, CXLGetEventPayload 
> *pl,
> + uint8_t log_type, int max_recs,
> + uint16_t *len);
> +CXLRetCode cxl_event_clear_records(CXLDeviceState *cxlds,
> +   CXLClearEventPayload *pl);
> +
>  void cxl_set_poison_list_overflowed(CXLType3Dev *ct3d);
>  
>  #endif
> diff --git a/include/hw/cxl/cxl_events.h b/include/hw/cxl/cxl_events.h
> index aeb3b0590e..d4aaa894f1 100644
> --- a/include/hw/cxl/cxl_events.h
> +++ b/include/hw/cxl/cxl_events.h
> @@ -10,6 +10,8 @@
>  #ifndef CXL_EVENTS_H
>  #define CXL_EVENTS_H
>  
> +#include "qemu/uuid.h"
> +
>  /*
>   * CXL rev 3.0 section 8.2.9.2.2; Table 8-49
>   *
> @@ -25,4 +27,57 @@ typedef enum CXLEventLogType {
>  CXL_EVENT_TYPE_MAX
>  } CXLEventLogType;
>  
> +/*
> + * Common Event Record Format
> + * CXL rev 3.0 section 8.2.9.2.1; Table 8-42
> + */
> +#define CXL_EVENT_REC_HDR_RES_LEN 0xf
> +typedef struct CXLEventRecordHdr {
> +QemuUUID id;
> +uint8_t length;
> +uint8_t flags[3];
> +uint16_t handle;
> +uint16_t related_handle;
> +uint64_t timestamp;
> +uint8_t maint_op_class;
> +uint8_t reserved[CXL_EVENT_REC_HDR_RES_LEN];
> +} QEMU_PACKED CXLEventRecordHdr;
> +
> +#define CXL_EVENT_RECORD_DATA_LENGTH 0x50
> +typedef struct CXLEventRecordRaw {
> +CXLEventRecordHdr hdr;
> +uint8_t data[CXL_EVENT_RECORD_DATA_LENGTH];
> +} QEMU_PACKED CXLEventRecordRaw;
> +#define CXL_EVENT_RECORD_SIZE (sizeof(CXLEventRecordRaw))
> +
> +/*
> + * Get Event Records output payload
> + * CXL rev 3.0 section 8.2.9.2.2; Table 8-50
> + */
> +#define CXL_GET_EVENT_FLAG_OVERFLOW BIT(0)
> +#define CXL_GET_EVENT_FLAG_MORE_RECORDS BIT(1)
> +typedef struct CXLGetEventPayload {
> +uint8_t flags;
> +uint8_t reserved1;
> +uint16_t overflow_err_count;
> +uint64_t first_overflow_timestamp;
> +uint64_t last_overflow_timestamp;
> +uint16_t record_count;
> +uint8_t reserved2[0xa];
> +CXLEventRecordRaw records[];
> +} QEMU_PACKED CXLGetEventPayload;
> +#define CXL_EVENT_PAYLOAD_HDR_SIZE (sizeof(CXLGetEventPayload))
> +
> +/*
> + * Clear Event Records input payload
> + * CXL rev 3.0 section 8.2.9.2.3; Table 8-51
> + */
> +typedef struct CXLClearEventPayload {
> +uint8_t event_log;  /* CXLE

Re: [PATCH v7 1/7] hw/cxl/events: Add event status register

[Fan Ni]

The 05/22/2023 16:09, Jonathan Cameron wrote:
> From: Ira Weiny 
> 
> The device status register block was defined.  However, there were no
> individual registers nor any data wired up.
> 
> Define the event status register [CXL 3.0; 8.2.8.3.1] as part of the
> device status register block.  Wire up the register and initialize the
> event status for each log.
> 
> To support CXL 3.0 the version of the device status register block needs
> to be 2.  Change the macro to allow for setting the version.
> 
> Signed-off-by: Ira Weiny 
> Signed-off-by: Jonathan Cameron 

Reviewed-by: Fan Ni 

> ---
>  include/hw/cxl/cxl_device.h | 23 +---
>  include/hw/cxl/cxl_events.h | 28 
>  hw/cxl/cxl-device-utils.c   | 43 -
>  3 files changed, 86 insertions(+), 8 deletions(-)
> 
> diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
> index 73328a52cf..16993f7098 100644
> --- a/include/hw/cxl/cxl_device.h
> +++ b/include/hw/cxl/cxl_device.h
> @@ -13,6 +13,7 @@
>  #include "hw/cxl/cxl_component.h"
>  #include "hw/pci/pci_device.h"
>  #include "hw/register.h"
> +#include "hw/cxl/cxl_events.h"
>  
>  /*
>   * The following is how a CXL device's Memory Device registers are laid out.
> @@ -86,7 +87,16 @@ typedef struct cxl_device_state {
>  MemoryRegion device_registers;
>  
>  /* mmio for device capabilities array - 8.2.8.2 */
> -MemoryRegion device;
> +struct {
> +MemoryRegion device;
> +union {
> +uint8_t dev_reg_state[CXL_DEVICE_STATUS_REGISTERS_LENGTH];
> +uint16_t dev_reg_state16[CXL_DEVICE_STATUS_REGISTERS_LENGTH / 2];
> +uint32_t dev_reg_state32[CXL_DEVICE_STATUS_REGISTERS_LENGTH / 4];
> +uint64_t dev_reg_state64[CXL_DEVICE_STATUS_REGISTERS_LENGTH / 8];
> +};
> +uint64_t event_status;
> +};
>  MemoryRegion memory_device;
>  struct {
>  MemoryRegion caps;
> @@ -141,6 +151,9 @@ REG64(CXL_DEV_CAP_ARRAY, 0) /* Documented as 128 bit 
> register but 64 byte access
>  FIELD(CXL_DEV_CAP_ARRAY, CAP_VERSION, 16, 8)
>  FIELD(CXL_DEV_CAP_ARRAY, CAP_COUNT, 32, 16)
>  
> +void cxl_event_set_status(CXLDeviceState *cxl_dstate, CXLEventLogType 
> log_type,
> +  bool available);
> +
>  /*
>   * Helper macro to initialize capability headers for CXL devices.
>   *
> @@ -175,7 +188,7 @@ CXL_DEVICE_CAPABILITY_HEADER_REGISTER(MEMORY_DEVICE,
>  void cxl_initialize_mailbox(CXLDeviceState *cxl_dstate);
>  void cxl_process_mailbox(CXLDeviceState *cxl_dstate);
>  
> -#define cxl_device_cap_init(dstate, reg, cap_id)   \
> +#define cxl_device_cap_init(dstate, reg, cap_id, ver)  \
>  do {   \
>  uint32_t *cap_hdrs = dstate->caps_reg_state32; \
>  int which = R_CXL_DEV_##reg##_CAP_HDR0;\
> @@ -183,7 +196,7 @@ void cxl_process_mailbox(CXLDeviceState *cxl_dstate);
>  FIELD_DP32(cap_hdrs[which], CXL_DEV_##reg##_CAP_HDR0,  \
> CAP_ID, cap_id);\
>  cap_hdrs[which] = FIELD_DP32(  \
> -cap_hdrs[which], CXL_DEV_##reg##_CAP_HDR0, CAP_VERSION, 1);\
> +cap_hdrs[which], CXL_DEV_##reg##_CAP_HDR0, CAP_VERSION, ver);  \
>  cap_hdrs[which + 1] =  \
>  FIELD_DP32(cap_hdrs[which + 1], CXL_DEV_##reg##_CAP_HDR1,  \
> CAP_OFFSET, CXL_##reg##_REGISTERS_OFFSET);  \
> @@ -192,6 +205,10 @@ void cxl_process_mailbox(CXLDeviceState *cxl_dstate);
> CAP_LENGTH, CXL_##reg##_REGISTERS_LENGTH);  \
>  } while (0)
>  
> +/* CXL 3.0 8.2.8.3.1 Event Status Register */
> +REG64(CXL_DEV_EVENT_STATUS, 0)
> +FIELD(CXL_DEV_EVENT_STATUS, EVENT_STATUS, 0, 32)
> +
>  /* CXL 2.0 8.2.8.4.3 Mailbox Capabilities Register */
>  REG32(CXL_DEV_MAILBOX_CAP, 0)
>  FIELD(CXL_DEV_MAILBOX_CAP, PAYLOAD_SIZE, 0, 5)
> diff --git a/include/hw/cxl/cxl_events.h b/include/hw/cxl/cxl_events.h
> new file mode 100644
> index 00..aeb3b0590e
> --- /dev/null
> +++ b/include/hw/cxl/cxl_events.h
> @@ -0,0 +1,28 @@
> +/*
> + * QEMU CXL Events
> + *
> + * Copyright (c) 2022 Intel
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2. See the
> + * COPYING file in the top-level directory.
> + */
> +
> +#ifndef CXL_EV

Re: [PATCH 1/2] hw/cxl: cdat: Fix open file not closed in ct3_load_cdat()

[Fan Ni]

On Fri, Apr 21, 2023 at 02:20:19PM +0100, Jonathan Cameron via wrote:
> From: Hao Zeng 
> 
> Open file descriptor not closed in error paths. Fix by replace
> open coded handling of read of whole file into a buffer with
> g_file_get_contents()
> 
> Fixes: aba578bdac ("hw/cxl: CDAT Data Object Exchange implementation")
> Signed-off-by: Zeng Hao 
> Suggested-by: Philippe Mathieu-Daudé 
> Suggested-by: Peter Maydell 
> Suggested-by: Jonathan Cameron via 
> Signed-off-by: Jonathan Cameron 
> 

Reviewed-by: Fan Ni 

> --
> Changes since v5:
> - Drop if guard on g_free() as per checkpatch warning.
> ---
>  hw/cxl/cxl-cdat.c | 29 -
>  1 file changed, 8 insertions(+), 21 deletions(-)
> 
> diff --git a/hw/cxl/cxl-cdat.c b/hw/cxl/cxl-cdat.c
> index 137abd0992..056711d63d 100644
> --- a/hw/cxl/cxl-cdat.c
> +++ b/hw/cxl/cxl-cdat.c
> @@ -110,29 +110,18 @@ static void ct3_load_cdat(CDATObject *cdat, Error 
> **errp)
>  g_autofree CDATEntry *cdat_st = NULL;
>  uint8_t sum = 0;
>  int num_ent;
> -int i = 0, ent = 1, file_size = 0;
> +int i = 0, ent = 1;
> +gsize file_size = 0;
>  CDATSubHeader *hdr;
> -FILE *fp = NULL;
> +GError *error = NULL;
>  
>  /* Read CDAT file and create its cache */
> -fp = fopen(cdat->filename, "r");
> -if (!fp) {
> -error_setg(errp, "CDAT: Unable to open file");
> +if (!g_file_get_contents(cdat->filename, (gchar **)>buf,
> + _size, )) {
> +error_setg(errp, "CDAT: File read failed: %s", error->message);
> +g_error_free(error);
>  return;
>  }
> -
> -fseek(fp, 0, SEEK_END);
> -file_size = ftell(fp);
> -fseek(fp, 0, SEEK_SET);
> -cdat->buf = g_malloc0(file_size);
> -
> -if (fread(cdat->buf, file_size, 1, fp) == 0) {
> -error_setg(errp, "CDAT: File read failed");
> -return;
> -}
> -
> -fclose(fp);
> -
>  if (file_size < sizeof(CDATTableHeader)) {
>  error_setg(errp, "CDAT: File too short");
>  return;
> @@ -218,7 +207,5 @@ void cxl_doe_cdat_release(CXLComponentState *cxl_cstate)
>  cdat->free_cdat_table(cdat->built_buf, cdat->built_buf_len,
>cdat->private);
>  }
> -if (cdat->buf) {
> -free(cdat->buf);
> -}
> +g_free(cdat->buf);
>  }
> -- 
> 2.37.2
> 
> 
> 

Re: [PATCH 2/2] hw/cxl: cdat: Fix failure to free buffer in erorr paths

[Fan Ni]

On Fri, Apr 21, 2023 at 02:20:20PM +0100, Jonathan Cameron via wrote:
> The failure paths in CDAT file loading did not clear up properly.
> Change to using g_auto_free and a local pointer for the buffer to
> ensure this function has no side effects on error.
> Also drop some unnecessary checks that can not fail.
> 
> Cleanup properly after a failure to load a CDAT file.
> 
> Suggested-by: Peter Maydell 
> Signed-off-by: Jonathan Cameron 

Reviewed-by: Fan Ni 

> ---
>  hw/cxl/cxl-cdat.c| 33 ++---
>  hw/mem/cxl_type3.c   |  4 
>  hw/pci-bridge/cxl_upstream.c |  3 +++
>  3 files changed, 25 insertions(+), 15 deletions(-)
> 
> diff --git a/hw/cxl/cxl-cdat.c b/hw/cxl/cxl-cdat.c
> index 056711d63d..d246d6885b 100644
> --- a/hw/cxl/cxl-cdat.c
> +++ b/hw/cxl/cxl-cdat.c
> @@ -108,6 +108,7 @@ static void ct3_build_cdat(CDATObject *cdat, Error **errp)
>  static void ct3_load_cdat(CDATObject *cdat, Error **errp)
>  {
>  g_autofree CDATEntry *cdat_st = NULL;
> +g_autofree char *buf = NULL;
>  uint8_t sum = 0;
>  int num_ent;
>  int i = 0, ent = 1;
> @@ -116,7 +117,7 @@ static void ct3_load_cdat(CDATObject *cdat, Error **errp)
>  GError *error = NULL;
>  
>  /* Read CDAT file and create its cache */
> -if (!g_file_get_contents(cdat->filename, (gchar **)>buf,
> +if (!g_file_get_contents(cdat->filename, (gchar **),
>   _size, )) {
>  error_setg(errp, "CDAT: File read failed: %s", error->message);
>  g_error_free(error);
> @@ -129,9 +130,17 @@ static void ct3_load_cdat(CDATObject *cdat, Error **errp)
>  i = sizeof(CDATTableHeader);
>  num_ent = 1;
>  while (i < file_size) {
> -hdr = (CDATSubHeader *)(cdat->buf + i);
> +hdr = (CDATSubHeader *)(buf + i);
> +if (i + sizeof(CDATSubHeader) > file_size) {
> +error_setg(errp, "CDAT: Truncated table");
> +return;
> +}
>  cdat_len_check(hdr, errp);
>  i += hdr->length;
> +if (i > file_size) {
> +error_setg(errp, "CDAT: Truncated table");
> +return;
> +}
>  num_ent++;
>  }
>  if (i != file_size) {
> @@ -139,33 +148,26 @@ static void ct3_load_cdat(CDATObject *cdat, Error 
> **errp)
>  return;
>  }
>  
> -cdat_st = g_malloc0(sizeof(*cdat_st) * num_ent);
> -if (!cdat_st) {
> -error_setg(errp, "CDAT: Failed to allocate entry array");
> -return;
> -}
> +cdat_st = g_new0(CDATEntry, num_ent);
>  
>  /* Set CDAT header, Entry = 0 */
> -cdat_st[0].base = cdat->buf;
> +cdat_st[0].base = buf;
>  cdat_st[0].length = sizeof(CDATTableHeader);
>  i = 0;
>  
>  while (i < cdat_st[0].length) {
> -sum += cdat->buf[i++];
> +sum += buf[i++];
>  }
>  
>  /* Read CDAT structures */
>  while (i < file_size) {
> -hdr = (CDATSubHeader *)(cdat->buf + i);
> -cdat_len_check(hdr, errp);
> -
> +hdr = (CDATSubHeader *)(buf + i);
>  cdat_st[ent].base = hdr;
>  cdat_st[ent].length = hdr->length;
>  
> -while (cdat->buf + i <
> -   (uint8_t *)cdat_st[ent].base + cdat_st[ent].length) {
> +while (buf + i < (char *)cdat_st[ent].base + cdat_st[ent].length) {
>  assert(i < file_size);
> -sum += cdat->buf[i++];
> +sum += buf[i++];
>  }
>  
>  ent++;
> @@ -176,6 +178,7 @@ static void ct3_load_cdat(CDATObject *cdat, Error **errp)
>  }
>  cdat->entry_len = num_ent;
>  cdat->entry = g_steal_pointer(_st);
> +cdat->buf = g_steal_pointer();
>  }
>  
>  void cxl_doe_cdat_init(CXLComponentState *cxl_cstate, Error **errp)
> diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
> index abe60b362c..7647122cc6 100644
> --- a/hw/mem/cxl_type3.c
> +++ b/hw/mem/cxl_type3.c
> @@ -593,6 +593,9 @@ static void ct3_realize(PCIDevice *pci_dev, Error **errp)
>  cxl_cstate->cdat.free_cdat_table = ct3_free_cdat_table;
>  cxl_cstate->cdat.private = ct3d;
>  cxl_doe_cdat_init(cxl_cstate, errp);
> +if (*errp) {
> +goto err_free_special_ops;
> +}
>  
>  pcie_cap_deverr_init(pci_dev);
>  /* Leave a bit of room for expansion */
> @@ -605,6 +608,7 @@ static void ct3_realize(PCIDevice *pci_dev, Error **errp)
>  
>  err_release_cdat:
>  cxl_doe_cdat_release(cxl_cstate);
> +err_free_special_ops:
>  g_free(regs->special_ops);
>  err

[Qemu RFC 0/7] Early enabling of DCD emulation in Qemu

[Fan Ni]

daxctl   meson.build   
rhel
tagstopology.png LICENSESccan   cscope.files
git-version  meson_options.txt  rpmbuild.shtest util


QEMU command line cxl configuration:

RP1="-object 
memory-backend-file,id=cxl-mem1,share=on,mem-path=/tmp/cxltest.raw,size=512M \
-object 
memory-backend-file,id=cxl-mem2,share=on,mem-path=/tmp/cxltest2.raw,size=512M \
-object 
memory-backend-file,id=cxl-lsa1,share=on,mem-path=/tmp/lsa.raw,size=512M \
-device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
-device cxl-rp,port=0,bus=cxl.1,id=root_port13,chassis=0,slot=2 \
-device 
cxl-type3,bus=root_port13,memdev=cxl-mem1,lsa=cxl-lsa1,dc-memdev=cxl-mem2,id=cxl-pmem0,num-dc-regions=1\
-M 
cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=8k"


Kernel DCD support used to test the changes

The code is tested with the posted kernel dcd support:
https://git.kernel.org/pub/scm/linux/kernel/git/cxl/cxl.git/log/?h=for-6.5/dcd-preview

commit: f425bc34c600e2a3721d6560202962ec41622815

To make the test work, we have made the following changes to the above kernel 
commit:

diff --git a/drivers/cxl/core/mbox.c b/drivers/cxl/core/mbox.c
index 5f04bbc18af5..5f421d3c5cef 100644
--- a/drivers/cxl/core/mbox.c
+++ b/drivers/cxl/core/mbox.c
@@ -68,6 +68,7 @@ static struct cxl_mem_command 
cxl_mem_commands[CXL_MEM_COMMAND_ID_MAX] = {
CXL_CMD(SCAN_MEDIA, 0x11, 0, 0),
CXL_CMD(GET_SCAN_MEDIA, 0, CXL_VARIABLE_PAYLOAD, 0),
CXL_CMD(GET_DC_EXTENT_LIST, 0x8, CXL_VARIABLE_PAYLOAD, 0),
+   CXL_CMD(GET_DC_CONFIG, 0x2, CXL_VARIABLE_PAYLOAD, 0),
 };
 
 /*
diff --git a/drivers/cxl/core/region.c b/drivers/cxl/core/region.c
index 291c716abd49..ae10e3cf43a1 100644
--- a/drivers/cxl/core/region.c
+++ b/drivers/cxl/core/region.c
@@ -194,7 +194,7 @@ static int cxl_region_manage_dc(struct cxl_region *cxlr)
}
cxlds->dc_list_gen_num = extent_gen_num;
dev_dbg(cxlds->dev, "No of preallocated extents :%d\n", rc);
-   enable_irq(cxlds->cxl_irq[CXL_EVENT_TYPE_DCD]);
+   /*enable_irq(cxlds->cxl_irq[CXL_EVENT_TYPE_DCD]);*/
}
return 0;
 err:
@@ -2810,7 +2810,8 @@ int cxl_add_dc_extent(struct cxl_dev_state *cxlds, struct 
resource *alloc_dpa_re
dev_dax->align, memremap_compat_align( {
rc = alloc_dev_dax_range(dev_dax, hpa,
resource_size(alloc_dpa_res));
-   return rc;
+   if (rc)
+   return rc;
}
 
rc = xa_insert(_dc->dax_dev_list, hpa, dev_dax, GFP_KERNEL);
diff --git a/drivers/cxl/pci.c b/drivers/cxl/pci.c
index 9e45b1056022..653bec203838 100644
--- a/drivers/cxl/pci.c
+++ b/drivers/cxl/pci.c
@@ -659,7 +659,7 @@ static int cxl_event_irqsetup(struct cxl_dev_state *cxlds)
 
/* Driver enables DCD interrupt after creating the dc cxl_region */
rc = cxl_event_req_irq(cxlds, policy.dyncap_settings, 
CXL_EVENT_TYPE_DCD,
-   IRQF_SHARED | IRQF_ONESHOT | 
IRQF_NO_AUTOEN);
+   IRQF_SHARED | IRQF_ONESHOT);
if (rc) {
dev_err(cxlds->dev, "Failed to get interrupt for event dc 
log\n");
return rc;
diff --git a/include/uapi/linux/cxl_mem.h b/include/uapi/linux/cxl_mem.h
index 6ca85861750c..910a48259239 100644
--- a/include/uapi/linux/cxl_mem.h
+++ b/include/uapi/linux/cxl_mem.h
@@ -47,6 +47,7 @@
___C(SCAN_MEDIA, "Scan Media"),   \
___C(GET_SCAN_MEDIA, "Get Scan Media Results"),   \
___C(GET_DC_EXTENT_LIST, "Get dynamic capacity extents"), \
+   ___C(GET_DC_CONFIG, "Get dynamic capacity configuration"), \
___C(MAX, "invalid / last command")
 
 #define ___C(a, b) CXL_MEM_COMMAND_ID_##a



Fan Ni (7):
  hw/cxl/cxl-mailbox-utils: Add dc_event_log_size field to output
payload of identify memory device command
  hw/cxl/cxl-mailbox-utils: Add dynamic capacity region representative
and mailbox command support
  hw/mem/cxl_type3: Add a parameter to pass number of DC regions the
device supports in qemu command line
  hw/mem/cxl_type3: Add DC extent representative to cxl type3 device
  hw/cxl/cxl-mailbox-utils: Add mailbox commands to support add/release
dynamic capacity response
  Add qmp interfaces to add/release dynamic capacity extents
  hw/mem/cxl_type3: add read/write support to dynamic capacity

 hw/cxl/cxl-mailbox-utils.c  | 389 +++-
 hw/mem/cxl_type3.c  | 492 +++-
 include/hw/cxl/cxl_device.h |  50 +++-
 include/hw/cxl/cxl_events.h |  16 ++
 qapi/cxl.json   |  44 
 5 files changed, 924 insertions(+), 67 deletions(-)

-- 
2.25.1

[RFC 2/7] hw/cxl/cxl-mailbox-utils: Add dynamic capacity region representative and mailbox command support

[Fan Ni]

From: Fan Ni 

Per cxl spec 3.0, add dynamic capacity region representative based on
Table 8-126 and extend the cxl type3 device definition to include dc region
information. Also, based on info in 8.2.9.8.9.1, add 'Get Dynamic Capacity
Configuration' mailbox support.

Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c  | 68 +
 include/hw/cxl/cxl_device.h | 16 +
 2 files changed, 84 insertions(+)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 7ff4fbdf22..61c77e52d8 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -81,6 +81,8 @@ enum {
 #define GET_POISON_LIST0x0
 #define INJECT_POISON  0x1
 #define CLEAR_POISON   0x2
+   DCD_CONFIG = 0x48, /*8.2.9.8.9*/
+   #define GET_DC_REGION_CONFIG   0x0
 PHYSICAL_SWITCH = 0x51
 #define IDENTIFY_SWITCH_DEVICE  0x0
 };
@@ -935,6 +937,70 @@ static CXLRetCode cmd_media_clear_poison(struct cxl_cmd 
*cmd,
 return CXL_MBOX_SUCCESS;
 }
 
+/*
+ * cxl spec 3.0: 8.2.9.8.9.2
+ * Get Dynamic Capacity Configuration
+ **/
+static CXLRetCode cmd_dcd_get_dyn_cap_config(struct cxl_cmd *cmd,
+   CXLDeviceState *cxl_dstate,
+   uint16_t *len)
+{
+   struct get_dyn_cap_config_in_pl {
+   uint8_t region_cnt;
+   uint8_t start_region_id;
+   } QEMU_PACKED;
+
+struct get_dyn_cap_config_out_pl {
+   uint8_t num_regions;
+   uint8_t rsvd1[7];
+   struct {
+   uint64_t base;
+   uint64_t decode_len;
+   uint64_t region_len;
+   uint64_t block_size;
+   uint32_t dsmadhandle;
+   uint8_t flags;
+   uint8_t rsvd2[3];
+   } QEMU_PACKED records[];
+   } QEMU_PACKED;
+
+   struct get_dyn_cap_config_in_pl *in = (void *)cmd->payload;
+   struct get_dyn_cap_config_out_pl *out = (void *)cmd->payload;
+   struct CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, 
cxl_dstate);
+   uint16_t record_count = 0, i = 0;
+   uint16_t out_pl_len;
+
+   if (in->start_region_id >= ct3d->dc.num_regions)
+   record_count = 0;
+   else if (ct3d->dc.num_regions - in->start_region_id < in->region_cnt)
+   record_count = ct3d->dc.num_regions - in->start_region_id;
+   else
+   record_count = in->region_cnt;
+
+   out_pl_len = sizeof(*out) + record_count * sizeof(out->records[0]);
+   assert(out_pl_len <= CXL_MAILBOX_MAX_PAYLOAD_SIZE);
+
+   memset(out, 0, out_pl_len);
+   out->num_regions = record_count;
+   for (; i < record_count; i++) {
+   stq_le_p(>records[i].base,
+   ct3d->dc.regions[in->start_region_id+i].base);
+   stq_le_p(>records[i].decode_len,
+   ct3d->dc.regions[in->start_region_id+i].decode_len);
+   stq_le_p(>records[i].region_len,
+   ct3d->dc.regions[in->start_region_id+i].len);
+   stq_le_p(>records[i].block_size,
+   ct3d->dc.regions[in->start_region_id+i].block_size);
+   stl_le_p(>records[i].dsmadhandle,
+   ct3d->dc.regions[in->start_region_id+i].dsmadhandle);
+   out->records[i].flags
+   = ct3d->dc.regions[in->start_region_id+i].flags;
+   }
+
+   *len = out_pl_len;
+   return CXL_MBOX_SUCCESS;
+}
+
 #define IMMEDIATE_CONFIG_CHANGE (1 << 1)
 #define IMMEDIATE_DATA_CHANGE (1 << 2)
 #define IMMEDIATE_POLICY_CHANGE (1 << 3)
@@ -973,6 +1039,8 @@ static struct cxl_cmd cxl_cmd_set[256][256] = {
 cmd_media_inject_poison, 8, 0 },
 [MEDIA_AND_POISON][CLEAR_POISON] = { "MEDIA_AND_POISON_CLEAR_POISON",
 cmd_media_clear_poison, 72, 0 },
+   [DCD_CONFIG][GET_DC_REGION_CONFIG] = { "DCD_GET_DC_REGION_CONFIG",
+   cmd_dcd_get_dyn_cap_config, 2, 0 },
 };
 
 static struct cxl_cmd cxl_cmd_set_sw[256][256] = {
diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index e285369693..8a04e53e90 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -383,6 +383,17 @@ typedef struct CXLPoison {
 typedef QLIST_HEAD(, CXLPoison) CXLPoisonList;
 #define CXL_POISON_LIST_LIMIT 256
 
+#define DCD_MAX_REGION_NUM 8
+
+typedef struct CXLDCD_Region {
+   uint64_t base;
+   uint64_t decode_len; /* in multiples of 256MB */
+   uint64_t len;
+   uint64_t block_size;
+   uint32_t dsmadhandle;
+   uint8_t flags;
+} CXLDCD_Region;
+
 struct CXLType3Dev {
 /* Private */
 PCIDevice parent_obj;
@@ -414,6 +425,11 @@ struct CXLType3Dev {
 unsigned int poison_li

[RFC 3/7] hw/mem/cxl_type3: Add a parameter to pass number of DC regions the device supports in qemu command line

[Fan Ni]

From: Fan Ni 

Add a property 'num-dc-regions' to ct3_props to allow users to create DC
regions.
With the change, users can control the number of DC regions the device
supports.
To make it easier, other parameters of the region like region base, length,
and block size are hard coded. If desired, these parameters
can be added easily.

Signed-off-by: Fan Ni 
---
 hw/mem/cxl_type3.c | 32 
 1 file changed, 32 insertions(+)

diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 2b483d3d8e..b9c375d9b4 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -684,6 +684,34 @@ static void ct3d_reg_write(void *opaque, hwaddr offset, 
uint64_t value,
 }
 }
 
+/*
+ * Create a dc region to test "Get Dynamic Capacity Configuration" command.
+ */
+static int cxl_create_toy_regions(CXLType3Dev *ct3d)
+{
+   int i;
+   uint64_t region_base = ct3d->hostvmem?ct3d->hostvmem->size
+   + ct3d->hostpmem->size:ct3d->hostpmem->size;
+   uint64_t region_len = 1024*1024*1024;
+   uint64_t decode_len = 4; /* 4*256MB */
+   uint64_t blk_size = 2*1024*1024;
+   struct CXLDCD_Region *region;
+
+   for (i = 0; i < ct3d->dc.num_regions; i++) {
+   region = >dc.regions[i];
+   region->base = region_base;
+   region->decode_len = decode_len;
+   region->len = region_len;
+   region->block_size = blk_size;
+   /* dsmad_handle is set when creating cdat table entries */
+   region->flags = 0;
+
+   region_base += region->len;
+   }
+
+   return 0;
+}
+
 static bool cxl_setup_memory(CXLType3Dev *ct3d, Error **errp)
 {
 DeviceState *ds = DEVICE(ct3d);
@@ -752,6 +780,9 @@ static bool cxl_setup_memory(CXLType3Dev *ct3d, Error 
**errp)
 g_free(p_name);
 }
 
+   if (cxl_create_toy_regions(ct3d))
+   return false;
+
 return true;
 }
 
@@ -1036,6 +1067,7 @@ static Property ct3_props[] = {
 DEFINE_PROP_UINT64("sn", CXLType3Dev, sn, UI64_NULL),
 DEFINE_PROP_STRING("cdat", CXLType3Dev, cxl_cstate.cdat.filename),
 DEFINE_PROP_UINT16("spdm", CXLType3Dev, spdm_port, 0),
+   DEFINE_PROP_UINT8("num-dc-regions", CXLType3Dev, dc.num_regions, 0),
 DEFINE_PROP_END_OF_LIST(),
 };
 
-- 
2.25.1

[RFC 4/7] hw/mem/cxl_type3: Add DC extent representative to cxl type3 device

[Fan Ni]

From: Fan Ni 

Add dynamic capacity extent information to the definition of
CXLType3Dev and add get DC extent list mailbox command based on
CXL.spec.3.0:.8.2.9.8.9.2.

With this command, we can create dc regions as below:

region=$(cat /sys/bus/cxl/devices/decoder0.0/create_dc_region)
echo $region> /sys/bus/cxl/devices/decoder0.0/create_dc_region
echo 256 > /sys/bus/cxl/devices/$region/interleave_granularity
echo 1 > /sys/bus/cxl/devices/$region/interleave_ways

echo "dc" >/sys/bus/cxl/devices/decoder2.0/mode
echo 0x3000 >/sys/bus/cxl/devices/decoder2.0/dpa_size

echo 0x3000 > /sys/bus/cxl/devices/$region/size
echo  "decoder2.0" > /sys/bus/cxl/devices/$region/target0
echo 1 > /sys/bus/cxl/devices/$region/commit
echo $region > /sys/bus/cxl/drivers/cxl_region/bind

Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c  | 73 -
 hw/mem/cxl_type3.c  |  1 +
 include/hw/cxl/cxl_device.h | 23 
 3 files changed, 96 insertions(+), 1 deletion(-)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 61c77e52d8..ed2ac154cb 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -83,6 +83,7 @@ enum {
 #define CLEAR_POISON   0x2
DCD_CONFIG = 0x48, /*8.2.9.8.9*/
#define GET_DC_REGION_CONFIG   0x0
+   #define GET_DYN_CAP_EXT_LIST   0x1
 PHYSICAL_SWITCH = 0x51
 #define IDENTIFY_SWITCH_DEVICE  0x0
 };
@@ -938,7 +939,7 @@ static CXLRetCode cmd_media_clear_poison(struct cxl_cmd 
*cmd,
 }
 
 /*
- * cxl spec 3.0: 8.2.9.8.9.2
+ * cxl spec 3.0: 8.2.9.8.9.1
  * Get Dynamic Capacity Configuration
  **/
 static CXLRetCode cmd_dcd_get_dyn_cap_config(struct cxl_cmd *cmd,
@@ -1001,6 +1002,73 @@ static CXLRetCode cmd_dcd_get_dyn_cap_config(struct 
cxl_cmd *cmd,
return CXL_MBOX_SUCCESS;
 }
 
+/*
+ * cxl spec 3.0: 8.2.9.8.9.2
+ * Get Dynamic Capacity Extent List (Opcode 4810h)
+ **/
+static CXLRetCode cmd_dcd_get_dyn_cap_ext_list(struct cxl_cmd *cmd,
+   CXLDeviceState *cxl_dstate,
+   uint16_t *len)
+{
+   struct get_dyn_cap_ext_list_in_pl {
+   uint32_t extent_cnt;
+   uint32_t start_extent_id;
+   } QEMU_PACKED;
+
+   struct get_dyn_cap_ext_list_out_pl {
+   uint32_t count;
+   uint32_t total_extents;
+   uint32_t generation_num;
+   uint8_t rsvd[4];
+   struct {
+   uint64_t start_dpa;
+   uint64_t len;
+   uint8_t tag[0x10];
+   uint16_t shared_seq;
+   uint8_t rsvd[6];
+   } QEMU_PACKED records[];
+   } QEMU_PACKED;
+
+   struct get_dyn_cap_ext_list_in_pl *in = (void *)cmd->payload;
+   struct get_dyn_cap_ext_list_out_pl *out = (void *)cmd->payload;
+   struct CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, 
cxl_dstate);
+   uint16_t record_count = 0, i = 0, record_done = 0;
+   CXLDCDExtentList *extent_list = >dc.extents;
+   CXLDCD_Extent *ent;
+   uint16_t out_pl_len;
+
+   if (in->start_extent_id > ct3d->dc.total_extent_count)
+   return CXL_MBOX_INVALID_INPUT;
+
+   if (ct3d->dc.total_extent_count - in->start_extent_id < in->extent_cnt)
+   record_count = ct3d->dc.total_extent_count - 
in->start_extent_id;
+   else
+   record_count = in->extent_cnt;
+
+   out_pl_len = sizeof(*out) + record_count * sizeof(out->records[0]);
+   assert(out_pl_len <= CXL_MAILBOX_MAX_PAYLOAD_SIZE);
+
+   memset(out, 0, out_pl_len);
+   stl_le_p(>count, record_count);
+   stl_le_p(>total_extents, ct3d->dc.total_extent_count);
+   stl_le_p(>generation_num, ct3d->dc.ext_list_gen_seq);
+
+   QTAILQ_FOREACH(ent, extent_list, node) {
+   if (i++ < in->start_extent_id)
+   continue;
+   stq_le_p(>records[i].start_dpa, ent->start_dpa);
+   stq_le_p(>records[i].len, ent->len);
+   memcpy(>records[i].tag, ent->tag, 0x10);
+   stw_le_p(>records[i].shared_seq, ent->shared_seq);
+   record_done++;
+   if (record_done == record_count)
+   break;
+   }
+
+   *len = out_pl_len;
+   return CXL_MBOX_SUCCESS;
+}
+
 #define IMMEDIATE_CONFIG_CHANGE (1 << 1)
 #define IMMEDIATE_DATA_CHANGE (1 << 2)
 #define IMMEDIATE_POLICY_CHANGE (1 << 3)
@@ -1041,6 +1109,9 @@ static struct cxl_cmd cxl_cmd_set[256][256] = {
 cmd_media_clear_poison, 72, 0 },
[DCD_CONFIG][GET_DC_REGION_CONFIG] = { "DCD_GET_DC_REGION_CONFIG",
cmd_dcd_get_dyn_cap_config, 2, 0 },
+   [DCD_CONFIG][GET_DYN_CAP_EXT_LIST] = {
+   "DCD_GET_DYN

[RFC 1/7] hw/cxl/cxl-mailbox-utils: Add dc_event_log_size field to output payload of identify memory device command

[Fan Ni]

From: Fan Ni 

Based on CXL spec 3.0 Table 8-94 (Identify Memory Device Output
Payload), dynamic capacity event log size should be part of
output of the Identify command.
Add dc_event_log_size to the output payload for the host to get the info.

Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 9f8e6722d7..7ff4fbdf22 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -21,6 +21,8 @@
 #include "sysemu/hostmem.h"
 
 #define CXL_CAPACITY_MULTIPLIER   (256 * MiB)
+/* Experimental value: dynamic capacity event log size */
+#define CXL_DC_EVENT_LOG_SIZE 8
 
 /*
  * How to add a new command, example. The command set FOO, with cmd BAR.
@@ -519,8 +521,9 @@ static CXLRetCode cmd_identify_memory_device(struct cxl_cmd 
*cmd,
 uint16_t inject_poison_limit;
 uint8_t poison_caps;
 uint8_t qos_telemetry_caps;
+   uint16_t dc_event_log_size;
 } QEMU_PACKED *id;
-QEMU_BUILD_BUG_ON(sizeof(*id) != 0x43);
+QEMU_BUILD_BUG_ON(sizeof(*id) != 0x45);
 
 CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
 CXLType3Class *cvc = CXL_TYPE3_GET_CLASS(ct3d);
@@ -543,6 +546,7 @@ static CXLRetCode cmd_identify_memory_device(struct cxl_cmd 
*cmd,
 st24_le_p(id->poison_list_max_mer, 256);
 /* No limit - so limited by main poison record limit */
 stw_le_p(>inject_poison_limit, 0);
+   stw_le_p(>dc_event_log_size, CXL_DC_EVENT_LOG_SIZE);
 
 *len = sizeof(*id);
 return CXL_MBOX_SUCCESS;
-- 
2.25.1

[RFC 7/7] hw/mem/cxl_type3: add read/write support to dynamic capacity

[Fan Ni]

From: Fan Ni 

Before the change, read from or write to dynamic capacity of the memory
device is not support as 1) no host backed file/memory is provided for
it; 2) no address space is created for the dynamic capacity.

With the change, add code to support following:
1. add a new property to type3 device "dc-memdev" to point to host
   memory backend for dynamic capacity;
2. add a bitmap for each region to track whether a block is host backed,
which will be used for address check when read/write dynamic capacity;
3. add namespace for dynamic capacity for read/write support;
4. create cdat entries for each dynamic capacity region;

Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c  |  21 ++-
 hw/mem/cxl_type3.c  | 336 +---
 include/hw/cxl/cxl_device.h |   8 +-
 3 files changed, 298 insertions(+), 67 deletions(-)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index 7212934627..efe61e67fb 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -391,9 +391,11 @@ static CXLRetCode cmd_firmware_update_get_info(struct 
cxl_cmd *cmd,
 char fw_rev4[0x10];
 } QEMU_PACKED *fw_info;
 QEMU_BUILD_BUG_ON(sizeof(*fw_info) != 0x50);
+   CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
 
 if ((cxl_dstate->vmem_size < CXL_CAPACITY_MULTIPLIER) ||
-(cxl_dstate->pmem_size < CXL_CAPACITY_MULTIPLIER)) {
+   (cxl_dstate->pmem_size < CXL_CAPACITY_MULTIPLIER) ||
+   (ct3d->dc.total_dynamic_capicity < CXL_CAPACITY_MULTIPLIER)) {
 return CXL_MBOX_INTERNAL_ERROR;
 }
 
@@ -534,7 +536,9 @@ static CXLRetCode cmd_identify_memory_device(struct cxl_cmd 
*cmd,
 CXLType3Class *cvc = CXL_TYPE3_GET_CLASS(ct3d);
 
 if ((!QEMU_IS_ALIGNED(cxl_dstate->vmem_size, CXL_CAPACITY_MULTIPLIER)) ||
-(!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER))) {
+   (!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, 
CXL_CAPACITY_MULTIPLIER)) ||
+   (!QEMU_IS_ALIGNED(ct3d->dc.total_dynamic_capicity,
+   CXL_CAPACITY_MULTIPLIER))) {
 return CXL_MBOX_INTERNAL_ERROR;
 }
 
@@ -543,7 +547,8 @@ static CXLRetCode cmd_identify_memory_device(struct cxl_cmd 
*cmd,
 
 snprintf(id->fw_revision, 0x10, "BWFW VERSION %02d", 0);
 
-stq_le_p(>total_capacity, cxl_dstate->mem_size / 
CXL_CAPACITY_MULTIPLIER);
+   stq_le_p(>total_capacity,
+   cxl_dstate->static_mem_size / CXL_CAPACITY_MULTIPLIER);
 stq_le_p(>persistent_capacity, cxl_dstate->pmem_size / 
CXL_CAPACITY_MULTIPLIER);
 stq_le_p(>volatile_capacity, cxl_dstate->vmem_size / 
CXL_CAPACITY_MULTIPLIER);
 stl_le_p(>lsa_size, cvc->get_lsa_size(ct3d));
@@ -568,9 +573,12 @@ static CXLRetCode cmd_ccls_get_partition_info(struct 
cxl_cmd *cmd,
 uint64_t next_pmem;
 } QEMU_PACKED *part_info = (void *)cmd->payload;
 QEMU_BUILD_BUG_ON(sizeof(*part_info) != 0x20);
+   CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
 
 if ((!QEMU_IS_ALIGNED(cxl_dstate->vmem_size, CXL_CAPACITY_MULTIPLIER)) ||
-(!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, CXL_CAPACITY_MULTIPLIER))) {
+   (!QEMU_IS_ALIGNED(cxl_dstate->pmem_size, 
CXL_CAPACITY_MULTIPLIER)) ||
+   (!QEMU_IS_ALIGNED(ct3d->dc.total_dynamic_capicity,
+   CXL_CAPACITY_MULTIPLIER))) {
 return CXL_MBOX_INTERNAL_ERROR;
 }
 
@@ -881,9 +889,8 @@ static CXLRetCode cmd_media_clear_poison(struct cxl_cmd 
*cmd,
 struct clear_poison_pl *in = (void *)cmd->payload;
 
 dpa = ldq_le_p(>dpa);
-if (dpa + 64 > cxl_dstate->mem_size) {
-return CXL_MBOX_INVALID_PA;
-}
+   if (dpa + 64 > cxl_dstate->static_mem_size && ct3d->dc.num_regions == 0)
+   return CXL_MBOX_INVALID_PA;
 
 QLIST_FOREACH(ent, poison_list, node) {
 /*
diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 70d47d43b9..334660bd0f 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -33,8 +33,8 @@ enum {
 };
 
 static int ct3_build_cdat_entries_for_mr(CDATSubHeader **cdat_table,
- int dsmad_handle, MemoryRegion *mr,
- bool is_pmem, uint64_t dpa_base)
+   int dsmad_handle, uint8_t flags,
+   uint64_t dpa_base, uint64_t size)
 {
 g_autofree CDATDsmas *dsmas = NULL;
 g_autofree CDATDslbis *dslbis0 = NULL;
@@ -53,9 +53,9 @@ static int ct3_build_cdat_entries_for_mr(CDATSubHeader 
**cdat_table,
 .length = sizeof(*dsmas),
 },
 .DSMADhandle = dsmad_handle,
-.flags = is_pmem ? CDAT_DSMAS_FLAG_NV : 0,
+   .flags = flags,
 .DPA_base = dpa_bas

[RFC 5/7] hw/cxl/cxl-mailbox-utils: Add mailbox commands to support add/release dynamic capacity response

[Fan Ni]

From: Fan Ni 

Per CXL spec 3.0, we implemented the two mailbox commands:
Add Dynamic Capacity Response (Opcode 4802h) 8.2.9.8.9.3, and
Release Dynamic Capacity Response (Opcode 4803h) 8.2.9.8.9.4.

Signed-off-by: Fan Ni 
---
 hw/cxl/cxl-mailbox-utils.c  | 223 
 include/hw/cxl/cxl_device.h |   3 +-
 2 files changed, 225 insertions(+), 1 deletion(-)

diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
index ed2ac154cb..7212934627 100644
--- a/hw/cxl/cxl-mailbox-utils.c
+++ b/hw/cxl/cxl-mailbox-utils.c
@@ -84,6 +84,8 @@ enum {
DCD_CONFIG = 0x48, /*8.2.9.8.9*/
#define GET_DC_REGION_CONFIG   0x0
#define GET_DYN_CAP_EXT_LIST   0x1
+   #define ADD_DYN_CAP_RSP0x2
+   #define RELEASE_DYN_CAP0x3
 PHYSICAL_SWITCH = 0x51
 #define IDENTIFY_SWITCH_DEVICE  0x0
 };
@@ -1069,6 +1071,221 @@ static CXLRetCode cmd_dcd_get_dyn_cap_ext_list(struct 
cxl_cmd *cmd,
return CXL_MBOX_SUCCESS;
 }
 
+static inline int test_bits(const unsigned long *addr, int nr, int size)
+{
+   unsigned long res = find_next_zero_bit(addr, size + nr, nr);
+
+   if (res >= nr + size)
+   return 1;
+   else
+   return 0;
+}
+
+static uint8_t find_region_id(struct CXLType3Dev *dev, uint64_t dpa
+   , uint64_t len)
+{
+   int8_t i = dev->dc.num_regions-1;
+
+   while (i > 0 && dpa < dev->dc.regions[i].base)
+   i--;
+
+   if (dpa < dev->dc.regions[i].base
+   || dpa + len > dev->dc.regions[i].base + 
dev->dc.regions[i].len)
+   return dev->dc.num_regions;
+
+   return i;
+}
+
+static CXLRetCode detect_malformed_extent_list(CXLType3Dev *dev, void *data)
+{
+   struct updated_dc_extent_list_in_pl {
+   uint32_t num_entries_updated;
+   uint8_t rsvd[4];
+   struct {
+   uint64_t start_dpa;
+   uint64_t len;
+   uint8_t rsvd[8];
+   } QEMU_PACKED updated_entries[];
+   } QEMU_PACKED;
+
+   struct updated_dc_extent_list_in_pl *in = data;
+   unsigned long *blk_bitmap;
+   uint64_t min_block_size = dev->dc.regions[0].block_size;
+   struct CXLDCD_Region *region = >dc.regions[0];
+   uint32_t i;
+   uint64_t dpa, len;
+   uint8_t rid;
+
+   for (i = 1; i < dev->dc.num_regions; i++) {
+   region = >dc.regions[i];
+   if (min_block_size > region->block_size)
+   min_block_size = region->block_size;
+   }
+   blk_bitmap = bitmap_new((region->len + region->base
+   - dev->dc.regions[0].base) / min_block_size);
+   g_assert(blk_bitmap);
+   bitmap_zero(blk_bitmap, (region->len + region->base
+   - dev->dc.regions[0].base) / min_block_size);
+
+   for (i = 0; i < in->num_entries_updated; i++) {
+   dpa = in->updated_entries[i].start_dpa;
+   len = in->updated_entries[i].len;
+
+   rid = find_region_id(dev, dpa, len);
+   if (rid == dev->dc.num_regions) {
+   g_free(blk_bitmap);
+   return CXL_MBOX_INVALID_PA;
+   }
+   region = >dc.regions[rid];
+   if (dpa % region->block_size || len % region->block_size) {
+   g_free(blk_bitmap);
+   return CXL_MBOX_INVALID_EXTENT_LIST;
+   }
+   /* the dpa range already covered by some other extents in the 
list */
+   if (test_bits(blk_bitmap, dpa/min_block_size, 
len/min_block_size)) {
+   g_free(blk_bitmap);
+   return CXL_MBOX_INVALID_EXTENT_LIST;
+   }
+   bitmap_set(blk_bitmap, dpa/min_block_size, len/min_block_size);
+   }
+
+   g_free(blk_bitmap);
+   return CXL_MBOX_SUCCESS;
+}
+
+/*
+ * cxl spec 3.0: 8.2.9.8.9.3
+ * Add Dynamic Capacity Response (opcode 4802h)
+ * Assuming extent list is updated when a extent is added, when receiving
+ * the response, verify and ensure the extent is utilized by the host, and
+ * update extent list  as needed.
+ **/
+static CXLRetCode cmd_dcd_add_dyn_cap_rsp(struct cxl_cmd *cmd,
+   CXLDeviceState *cxl_dstate,
+   uint16_t *len_unused)
+{
+   struct add_dyn_cap_ext_list_in_pl {
+   uint32_t num_entries_updated;
+   uint8_t rsvd[4];
+   struct {
+   uint64_t start_dpa;
+   uint64_t len;
+   uint8_t rsvd[8];
+   } QEMU_PACKED updated_entries[];
+   } QEMU_PACKED;
+
+   struct add_dyn_cap_ext_list_in_pl *in = (void *)cmd->payload;
+   struct

[RFC 6/7] Add qmp interfaces to add/release dynamic capacity extents

[Fan Ni]

From: Fan Ni 

Since fabric manager emulation is not supported yet, the change implements
the functions to add/release dynamic capacity extents as QMP interfaces.

1. Add dynamic capacity extents:

For example, the command to add two continuous extents (each is 128MB
long) to region 0 (starting at dpa offset 0) looks like below:

{ "execute": "qmp_capabilities" }

{ "execute": "cxl-add-dynamic-capacity-event",
"arguments": {
"path": "/machine/peripheral/cxl-pmem0",
"region-id" : 0,
"num-extent": 2,
"dpa":0,
"extent-len": 128
}
}

2. Release dynamic capacity extents:

For example, the command to release an extent of size 128MB from region
0 (starting at dpa offset 0) look like below:

{ "execute": "cxl-release-dynamic-capacity-event",
"arguments": {
 "path": "/machine/peripheral/cxl-pmem0",
    "region-id" : 0,
 "num-extent": 1 ,
"dpa":0,
"extent-len": 128
}
}

Signed-off-by: Fan Ni 
---
 hw/mem/cxl_type3.c  | 127 
 include/hw/cxl/cxl_events.h |  16 +
 qapi/cxl.json   |  44 +
 3 files changed, 187 insertions(+)

diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
index 23954711b5..70d47d43b9 100644
--- a/hw/mem/cxl_type3.c
+++ b/hw/mem/cxl_type3.c
@@ -1651,6 +1651,133 @@ void qmp_cxl_inject_memory_module_event(const char 
*path, CxlEventLog log,
 }
 }
 
+static const QemuUUID dynamic_capacity_uuid = {
+   .data = UUID(0xca95afa7, 0xf183, 0x4018, 0x8c, 0x2f,
+   0x95, 0x26, 0x8e, 0x10, 0x1a, 0x2a),
+};
+
+static void qmp_cxl_process_dynamic_capacity_event(const char *path, 
CxlEventLog log,
+   uint8_t flags, uint8_t type, uint16_t hid, uint8_t rid, 
uint32_t extent_cnt,
+   CXLDCExtent_raw *extents, Error **errp)
+{
+   Object *obj = object_resolve_path(path, NULL);
+   CXLEventDynamicCapacity dCap;
+   CXLEventRecordHdr *hdr = 
+   CXLDeviceState *cxlds;
+   CXLType3Dev *dcd;
+   int i;
+
+   if (!obj) {
+   error_setg(errp, "Unable to resolve path");
+   return;
+   }
+   if (!object_dynamic_cast(obj, TYPE_CXL_TYPE3)) {
+   error_setg(errp, "Path not point to a valid CXL type3 device");
+   return;
+   }
+
+   dcd = CXL_TYPE3(obj);
+   cxlds = >cxl_dstate;
+   memset(, 0, sizeof(dCap));
+
+   if (!dcd->dc.num_regions) {
+   error_setg(errp, "No dynamic capacity support from the device");
+   return;
+   }
+
+   /*
+* 8.2.9.1.5
+* All Dynamic Capacity event records shall set the Event Record
+* Severity field in the Common Event Record Format to Informational
+* Event. All Dynamic Capacity related events shall be logged in the
+* Dynamic Capacity Event Log.
+*/
+   assert(flags & (1<dc.regions[rid].base;
+   memcpy(_capacity_extent, [i]
+   , sizeof(CXLDCExtent_raw));
+
+   if (cxl_event_insert(cxlds, CXL_EVENT_TYPE_DYNAMIC_CAP,
+   (CXLEventRecordRaw *))) {
+   ;
+   }
+   cxl_event_irq_assert(dcd);
+   }
+}
+
+#define MEM_BLK_SIZE_MB 128
+void qmp_cxl_add_dynamic_capacity_event(const char *path, uint8_t region_id,
+   uint32_t num_exent, uint64_t dpa, uint64_t extent_len_MB, Error 
**errp)
+{
+   uint8_t flags = 1 << CXL_EVENT_TYPE_INFO;
+   CXLDCExtent_raw *extents;
+   int i;
+
+   if (extent_len_MB < MEM_BLK_SIZE_MB) {
+   error_setg(errp,
+   "extent size cannot be smaller than memory block size 
(%dMB)",
+   MEM_BLK_SIZE_MB);
+   return;
+   }
+
+   extents = g_new0(CXLDCExtent_raw, num_exent);
+   for (i = 0; i < num_exent; i++) {
+   extents[i].start_dpa = dpa;
+   extents[i].len = extent_len_MB*1024*1024;
+   memset(extents[i].tag, 0, 0x10);
+   extents[i].shared_seq = 0;
+   dpa += extents[i].len;
+   }
+
+   qmp_cxl_process_dynamic_capacity_event(path, 
CXL_EVENT_LOG_INFORMATIONAL,
+   flags, 0x0, 0, region_id, num_exent, extents, errp);
+
+   g_free(extents);
+}
+
+void qmp_cxl_release_dynamic_capacity_event(const char *path, uint8_t 
region_id,
+   uint32_t num_exent, uint64_t dpa, uint64_t extent_len_MB, Error 
**errp)
+{
+   uint8_t flags = 1 << CXL_EVENT_TYPE_INFO;
+   CXLDCExtent_raw *extents;
+   int i;
+
+   if (ext

Re: [PATCH] Subject:[PATCH] cxl-cdat:Fix open file not closed in ct3_load_cdat

[Fan Ni]

On Mon, Apr 03, 2023 at 04:42:45PM +0800, zenghao wrote:
> opened file processor not closed,May cause file processor leaks
> 
> Fixes:aba578bdace5303a441f8a37aad781b5cb06f38c
> 
> Signed-off-by: Zeng Hao 
> Suggested-by: Xie Ming 
> ---
>  hw/cxl/cxl-cdat.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/hw/cxl/cxl-cdat.c b/hw/cxl/cxl-cdat.c
> index 137abd0992..ba7ed1aafd 100644
> --- a/hw/cxl/cxl-cdat.c
> +++ b/hw/cxl/cxl-cdat.c
> @@ -128,6 +128,7 @@ static void ct3_load_cdat(CDATObject *cdat, Error **errp)
>  
>  if (fread(cdat->buf, file_size, 1, fp) == 0) {
>  error_setg(errp, "CDAT: File read failed");
> +fclose(fp);
>  return;
>  }
>  
Good catch.

Reviewed-by: Fan Ni 

> -- 
> 2.37.2
> 
> 
> No virus found
>   Checked by Hillstone Network AntiVirus

Re: property 'cxl-type3.size' not found

[Fan Ni]

On Mon, Apr 03, 2023 at 11:38:22AM +0200, Philippe Mathieu-Daudé wrote:
> Cc'ing CXL maintainers.
> 
> On 3/4/23 11:04, Maverickk 78 wrote:
> > Hello,
> > 
> > I am trying qemu-system-aarch64 & cxl configuration listed in
> > 
> > https://urldefense.com/v3/__https://www.qemu.org/docs/master/system/devices/cxl.html__;!!EwVzqGoTKBqv-0DWAJBm!RhlUw_6f1jcigdN0SxPK7vXwp0-XgEXVUUvBpQ4M4D0ourDVGBkVXgVapWI3nQuQcq-73-XqXFBIGMz1$
> >   
> >  > >
> > 
> > qemu-system-aarch64 -M virt,gic-version=3,cxl=on -m 4g,maxmem=8G,slots=8
> > -cpu max \
> > ...
> > -object 
> > memory-backend-file,id=cxl-mem0,share=on,mem-path=/tmp/cxltest.raw,size=256M
> > \
> > -object 
> > memory-backend-file,id=cxl-mem1,share=on,mem-path=/tmp/cxltest1.raw,size=256M
> > \
> > -object 
> > memory-backend-file,id=cxl-mem2,share=on,mem-path=/tmp/cxltest2.raw,size=256M
> > \
> > -object 
> > memory-backend-file,id=cxl-mem3,share=on,mem-path=/tmp/cxltest3.raw,size=256M
> > \
> > -object
> > memory-backend-file,id=cxl-lsa0,share=on,mem-path=/tmp/lsa0.raw,size=256M
> > \
> > -object
> > memory-backend-file,id=cxl-lsa1,share=on,mem-path=/tmp/lsa1.raw,size=256M
> > \
> > -object
> > memory-backend-file,id=cxl-lsa2,share=on,mem-path=/tmp/lsa2.raw,size=256M
> > \
> > -object
> > memory-backend-file,id=cxl-lsa3,share=on,mem-path=/tmp/lsa3.raw,size=256M
> > \
> > -device pxb-cxl,bus_nr=12,bus=pcie.0,id=cxl.1 \
> > -device cxl-rp,port=0,bus=cxl.1,id=root_port0,chassis=0,slot=0 \
> > -device cxl-rp,port=1,bus=cxl.1,id=root_port1,chassis=0,slot=1 \
> > -device cxl-upstream,bus=root_port0,id=us0 \
> > -device cxl-downstream,port=0,bus=us0,id=swport0,chassis=0,slot=4 \
> > -device
> > cxl-type3,bus=swport0,memdev=cxl-mem0,lsa=cxl-lsa0,id=cxl-pmem0,size=256M
> > \
> > -device cxl-downstream,port=1,bus=us0,id=swport1,chassis=0,slot=5 \
> > -device
> > cxl-type3,bus=swport1,memdev=cxl-mem1,lsa=cxl-lsa1,id=cxl-pmem1,size=256M
> > \
> > -device cxl-downstream,port=2,bus=us0,id=swport2,chassis=0,slot=6 \
> > -device
> > cxl-type3,bus=swport2,memdev=cxl-mem2,lsa=cxl-lsa2,id=cxl-pmem2,size=256M
> > \
> > -device cxl-downstream,port=3,bus=us0,id=swport3,chassis=0,slot=7 \
> > -device
> > cxl-type3,bus=swport3,memdev=cxl-mem3,lsa=cxl-lsa3,id=cxl-pmem3,size=256M
> > \
> > -M 
> > cxl-fmw.0.targets.0=cxl.1,cxl-fmw.0.size=4G,cxl-fmw.0.interleave-granularity=4k
> > 
> > 
> > 
> > I hit this following error
> > qemu-system-aarch64: -device
> > cxl-type3,bus=swport0,memdev=cxl-mem0,lsa=cxl-lsa0,id=cxl-pmem0,size=256M:
> > property 'cxl-type3.size' not found
> > 
check cxl_type3.c where you can find the properties supported by the
device. 'size' is not needed and it will take the size of cxl-pmem0.

Fan
> > 
> > Any clue if I am missing something?
> > 
> > 
> > Regards
> > 
> > 
> > 
> 

Re: [RESEND PATCH 2/2] hw/cxl: Fix incorrect reset of commit and associated clearing of committed.

[Fan Ni]

On Wed, Mar 22, 2023 at 10:33:00AM +, Jonathan Cameron wrote:
> The hardware clearing the commit bit is not spec compliant.
> Clearing of committed bit when commit is cleared is not specifically
> stated in the CXL spec, but is the expected (and simplest) permitted
> behaviour so use that for QEMU emulation.
> 
> Signed-off-by: Jonathan Cameron 
> ---

Reviewed-by: Fan Ni 
Tested-by: Fan Ni 


The patch passed the tests as shown in previous mailing list discussion:
https://lore.kernel.org/linux-cxl/640276695c8e8_5b27929...@dwillia2-xfh.jf.intel.com.notmuch/T/#m0afcfc21d68c84c07f2e9e3194f587c2ffa82d6d
The following two topologies are tested:
1. one HB with one root port and a direct attached memdev.
2. one HB with 2 root ports and a memdev is directly attached to a CXL switch
which is attached to one root port.

One minor thing below.

>  hw/cxl/cxl-component-utils.c |  6 +-
>  hw/mem/cxl_type3.c   | 21 -
>  2 files changed, 25 insertions(+), 2 deletions(-)
> 
> diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c
> index a3e6cf75cf..378f1082ce 100644
> --- a/hw/cxl/cxl-component-utils.c
> +++ b/hw/cxl/cxl-component-utils.c
> @@ -38,19 +38,23 @@ static void dumb_hdm_handler(CXLComponentState 
> *cxl_cstate, hwaddr offset,
>  ComponentRegisters *cregs = _cstate->crb;
>  uint32_t *cache_mem = cregs->cache_mem_registers;
>  bool should_commit = false;
> +bool should_uncommit = false;
>  
>  switch (offset) {
>  case A_CXL_HDM_DECODER0_CTRL:
>  should_commit = FIELD_EX32(value, CXL_HDM_DECODER0_CTRL, COMMIT);
> +should_uncommit = !should_commit;
This will cause committed always reset if COMMIT is 0, not only
1->0. No issue for now, just point out to make sure it is what we
want.
>  break;
>  default:
>  break;
>  }
>  
>  if (should_commit) {
> -value = FIELD_DP32(value, CXL_HDM_DECODER0_CTRL, COMMIT, 0);
>  value = FIELD_DP32(value, CXL_HDM_DECODER0_CTRL, ERR, 0);
>  value = FIELD_DP32(value, CXL_HDM_DECODER0_CTRL, COMMITTED, 1);
> +} else if (should_uncommit) {
> +value = FIELD_DP32(value, CXL_HDM_DECODER0_CTRL, ERR, 0);
> +value = FIELD_DP32(value, CXL_HDM_DECODER0_CTRL, COMMITTED, 0);
>  }
>  stl_le_p((uint8_t *)cache_mem + offset, value);
>  }
> diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
> index 846089ccda..9598d584ac 100644
> --- a/hw/mem/cxl_type3.c
> +++ b/hw/mem/cxl_type3.c
> @@ -320,13 +320,28 @@ static void hdm_decoder_commit(CXLType3Dev *ct3d, int 
> which)
>  
>  ctrl = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL);
>  /* TODO: Sanity checks that the decoder is possible */
> -ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, COMMIT, 0);
>  ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, ERR, 0);
>  ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED, 1);
>  
>  stl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL, ctrl);
>  }
>  
> +static void hdm_decoder_uncommit(CXLType3Dev *ct3d, int which)
> +{
> +ComponentRegisters *cregs = >cxl_cstate.crb;
> +uint32_t *cache_mem = cregs->cache_mem_registers;
> +uint32_t ctrl;
> +
> +assert(which == 0);
> +
> +ctrl = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL);
> +
> +ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, ERR, 0);
> +ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED, 0);
> +
> +stl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL, ctrl);
> +}
> +
>  static int ct3d_qmp_uncor_err_to_cxl(CxlUncorErrorType qmp_err)
>  {
>  switch (qmp_err) {
> @@ -395,6 +410,7 @@ static void ct3d_reg_write(void *opaque, hwaddr offset, 
> uint64_t value,
>  CXLType3Dev *ct3d = container_of(cxl_cstate, CXLType3Dev, cxl_cstate);
>  uint32_t *cache_mem = cregs->cache_mem_registers;
>  bool should_commit = false;
> +bool should_uncommit = false;
>  int which_hdm = -1;
>  
>  assert(size == 4);
> @@ -403,6 +419,7 @@ static void ct3d_reg_write(void *opaque, hwaddr offset, 
> uint64_t value,
>  switch (offset) {
>  case A_CXL_HDM_DECODER0_CTRL:
>  should_commit = FIELD_EX32(value, CXL_HDM_DECODER0_CTRL, COMMIT);
> +should_uncommit = !should_commit;
>  which_hdm = 0;
>  break;
>  case A_CXL_RAS_UNC_ERR_STATUS:
> @@ -489,6 +506,8 @@ static void ct3d_reg_write(void *opaque, hwaddr offset, 
> uint64_t value,
>  stl_le_p((uint8_t *)cache_mem + offset, value);
>  if (should_commit) {
>  hdm_decoder_commit(ct3d, which_hdm);
> +} else if (should_uncommit) {
> +hdm_decoder_uncommit(ct3d, which_hdm);
>  }
>  }
>  
> -- 
> 2.37.2
> 

Re: [PATCH 1/2] hw/cxl: Fix endian handling for decoder commit.

[Fan Ni]

On Tue, Mar 21, 2023 at 06:00:11PM +, Jonathan Cameron wrote:
> Not a real problem yet as all supported architectures are
> little endian, but continue to tidy these up when touching
> code for other reasons.
> 
> Signed-off-by: Jonathan Cameron 

Hi Jonathan,
Did you forget to send the other patch in this series by any chance?

Fan
> ---
>  hw/cxl/cxl-component-utils.c | 10 --
>  hw/mem/cxl_type3.c   |  9 ++---
>  2 files changed, 10 insertions(+), 9 deletions(-)
> 
> diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c
> index b665d4f565..a3e6cf75cf 100644
> --- a/hw/cxl/cxl-component-utils.c
> +++ b/hw/cxl/cxl-component-utils.c
> @@ -47,14 +47,12 @@ static void dumb_hdm_handler(CXLComponentState 
> *cxl_cstate, hwaddr offset,
>  break;
>  }
>  
> -memory_region_transaction_begin();
> -stl_le_p((uint8_t *)cache_mem + offset, value);
>  if (should_commit) {
> -ARRAY_FIELD_DP32(cache_mem, CXL_HDM_DECODER0_CTRL, COMMIT, 0);
> -ARRAY_FIELD_DP32(cache_mem, CXL_HDM_DECODER0_CTRL, ERR, 0);
> -ARRAY_FIELD_DP32(cache_mem, CXL_HDM_DECODER0_CTRL, COMMITTED, 1);
> +value = FIELD_DP32(value, CXL_HDM_DECODER0_CTRL, COMMIT, 0);
> +value = FIELD_DP32(value, CXL_HDM_DECODER0_CTRL, ERR, 0);
> +value = FIELD_DP32(value, CXL_HDM_DECODER0_CTRL, COMMITTED, 1);
>  }
> -memory_region_transaction_commit();
> +stl_le_p((uint8_t *)cache_mem + offset, value);
>  }
>  
>  static void cxl_cache_mem_write_reg(void *opaque, hwaddr offset, uint64_t 
> value,
> diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
> index abe60b362c..846089ccda 100644
> --- a/hw/mem/cxl_type3.c
> +++ b/hw/mem/cxl_type3.c
> @@ -314,14 +314,17 @@ static void hdm_decoder_commit(CXLType3Dev *ct3d, int 
> which)
>  {
>  ComponentRegisters *cregs = >cxl_cstate.crb;
>  uint32_t *cache_mem = cregs->cache_mem_registers;
> +uint32_t ctrl;
>  
>  assert(which == 0);
>  
> +ctrl = ldl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL);
>  /* TODO: Sanity checks that the decoder is possible */
> -ARRAY_FIELD_DP32(cache_mem, CXL_HDM_DECODER0_CTRL, COMMIT, 0);
> -ARRAY_FIELD_DP32(cache_mem, CXL_HDM_DECODER0_CTRL, ERR, 0);
> +ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, COMMIT, 0);
> +ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, ERR, 0);
> +ctrl = FIELD_DP32(ctrl, CXL_HDM_DECODER0_CTRL, COMMITTED, 1);
>  
> -ARRAY_FIELD_DP32(cache_mem, CXL_HDM_DECODER0_CTRL, COMMITTED, 1);
> +stl_le_p(cache_mem + R_CXL_HDM_DECODER0_CTRL, ctrl);
>  }
>  
>  static int ct3d_qmp_uncor_err_to_cxl(CxlUncorErrorType qmp_err)
> -- 
> 2.37.2
> 

[qemu PATCH] hw/cxl/cxl_device: Replace magic number in CXLError definition

[Fan Ni]

Replace the magic number 32 with CXL_RAS_ERR_HEADER_NUM for better code
readability and maintainability.

Signed-off-by: Fan Ni 
---
 include/hw/cxl/cxl_device.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
index d589f78202..34fde62eac 100644
--- a/include/hw/cxl/cxl_device.h
+++ b/include/hw/cxl/cxl_device.h
@@ -235,7 +235,7 @@ REG64(CXL_MEM_DEV_STS, 0)
 typedef struct CXLError {
 QTAILQ_ENTRY(CXLError) node;
 int type; /* Error code as per FE definition */
-uint32_t header[32];
+uint32_t header[CXL_RAS_ERR_HEADER_NUM];
 } CXLError;
 
 typedef QTAILQ_HEAD(, CXLError) CXLErrorList;
-- 
2.25.1

Re: [PATCH v4 4/6] hw/cxl: QMP based poison injection support

[Fan Ni]

The 03/03/2023 15:09, Jonathan Cameron wrote:
> Inject poison using qmp command cxl-inject-poison to add an entry to the
> poison list.
> 
> For now, the poison is not returned CXL.mem reads, but only via the
> mailbox command Get Poison List.
> 
> See CXL rev 3.0, sec 8.2.9.8.4.1 Get Poison list (Opcode 4300h)
> 
> Kernel patches to use this interface here:
> https://lore.kernel.org/linux-cxl/cover.1665606782.git.alison.schofi...@intel.com/
> 
> To inject poison using qmp (telnet to the qmp port)
> { "execute": "qmp_capabilities" }
> 
> { "execute": "cxl-inject-poison",
> "arguments": {
>  "path": "/machine/peripheral/cxl-pmem0",
>  "start": 2048,
>  "length": 256
> }
> }
> 
> Adjusted to select a device on your machine.
> 
> Note that the poison list supported is kept short enough to avoid the
> complexity of state machine that is needed to handle the MORE flag.
> 
> Signed-off-by: Jonathan Cameron 
> 

Reviewed-by: Fan Ni 

> ---
> v4:
>  - Widen the mask on Poison source (lower bits of the address)
>to allow for Vendor Defined. Change will make it easier to potentially
>add a means to inject such poison in the future. Today it has no
>impact.
> ---
>  hw/cxl/cxl-mailbox-utils.c  | 90 +
>  hw/mem/cxl_type3.c  | 56 +++
>  hw/mem/cxl_type3_stubs.c|  6 +++
>  include/hw/cxl/cxl_device.h | 20 +
>  qapi/cxl.json   | 18 
>  5 files changed, 190 insertions(+)
> 
> diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
> index 702e16ca20..25933cf62c 100644
> --- a/hw/cxl/cxl-mailbox-utils.c
> +++ b/hw/cxl/cxl-mailbox-utils.c
> @@ -62,6 +62,8 @@ enum {
>  #define GET_PARTITION_INFO 0x0
>  #define GET_LSA   0x2
>  #define SET_LSA   0x3
> +MEDIA_AND_POISON = 0x43,
> +#define GET_POISON_LIST0x0
>  };
>  
>  /* 8.2.8.4.5.1 Command Return Codes */
> @@ -295,6 +297,10 @@ static CXLRetCode cmd_identify_memory_device(struct 
> cxl_cmd *cmd,
>  stq_le_p(>persistent_capacity, cxl_dstate->pmem_size / 
> CXL_CAPACITY_MULTIPLIER);
>  stq_le_p(>volatile_capacity, cxl_dstate->vmem_size / 
> CXL_CAPACITY_MULTIPLIER);
>  stl_le_p(>lsa_size, cvc->get_lsa_size(ct3d));
> +/* 256 poison records */
> +st24_le_p(id->poison_list_max_mer, 256);
> +/* No limit - so limited by main poison record limit */
> +stw_le_p(>inject_poison_limit, 0);
>  
>  *len = sizeof(*id);
>  return CXL_MBOX_SUCCESS;
> @@ -384,6 +390,88 @@ static CXLRetCode cmd_ccls_set_lsa(struct cxl_cmd *cmd,
>  return CXL_MBOX_SUCCESS;
>  }
>  
> +/*
> + * This is very inefficient, but good enough for now!
> + * Also the payload will always fit, so no need to handle the MORE flag and
> + * make this stateful. We may want to allow longer poison lists to aid
> + * testing that kernel functionality.
> + */
> +static CXLRetCode cmd_media_get_poison_list(struct cxl_cmd *cmd,
> +CXLDeviceState *cxl_dstate,
> +uint16_t *len)
> +{
> +struct get_poison_list_pl {
> +uint64_t pa;
> +uint64_t length;
> +} QEMU_PACKED;
> +
> +struct get_poison_list_out_pl {
> +uint8_t flags;
> +uint8_t rsvd1;
> +uint64_t overflow_timestamp;
> +uint16_t count;
> +uint8_t rsvd2[0x14];
> +struct {
> +uint64_t addr;
> +uint32_t length;
> +uint32_t resv;
> +} QEMU_PACKED records[];
> +} QEMU_PACKED;
> +
> +struct get_poison_list_pl *in = (void *)cmd->payload;
> +struct get_poison_list_out_pl *out = (void *)cmd->payload;
> +CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
> +uint16_t record_count = 0, i = 0;
> +uint64_t query_start, query_length;
> +CXLPoisonList *poison_list = >poison_list;
> +CXLPoison *ent;
> +uint16_t out_pl_len;
> +
> +query_start = ldq_le_p(>pa);
> +/* 64 byte alignemnt required */
> +if (query_start & 0x3f) {
> +return CXL_MBOX_INVALID_INPUT;
> +}
> +query_length = ldq_le_p(>length) * 64;
> +
> +QLIST_FOREACH(ent, poison_list, node) {
> +/* Check for no overlap */
> +if (ent->start >= query_start + query_length ||
> +ent->start + ent->length <= query_start) {
> +continue;
> +}
&

Re: [PATCH v4 5/6] hw/cxl: Add poison injection via the mailbox.

[Fan Ni]

The 03/03/2023 15:09, Jonathan Cameron wrote:
> Very simple implementation to allow testing of corresponding
> kernel code. Note that for now we track each 64 byte section
> independently.  Whilst a valid implementation choice, it may
> make sense to fuse entries so as to prove out more complex
> corners of the kernel code.
> 
> Reviewed-by: Ira Weiny 
> Signed-off-by: Jonathan Cameron 
> ---

Reviewed-by: Fan Ni 

> v4: No change
> ---
>  hw/cxl/cxl-mailbox-utils.c | 41 ++
>  1 file changed, 41 insertions(+)
> 
> diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
> index 25933cf62c..64a3f3c1bf 100644
> --- a/hw/cxl/cxl-mailbox-utils.c
> +++ b/hw/cxl/cxl-mailbox-utils.c
> @@ -64,6 +64,7 @@ enum {
>  #define SET_LSA   0x3
>  MEDIA_AND_POISON = 0x43,
>  #define GET_POISON_LIST0x0
> +#define INJECT_POISON  0x1
>  };
>  
>  /* 8.2.8.4.5.1 Command Return Codes */
> @@ -472,6 +473,44 @@ static CXLRetCode cmd_media_get_poison_list(struct 
> cxl_cmd *cmd,
>  return CXL_MBOX_SUCCESS;
>  }
>  
> +static CXLRetCode cmd_media_inject_poison(struct cxl_cmd *cmd,
> +  CXLDeviceState *cxl_dstate,
> +  uint16_t *len)
> +{
> +CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
> +CXLPoisonList *poison_list = >poison_list;
> +CXLPoison *ent;
> +struct inject_poison_pl {
> +uint64_t dpa;
> +};
> +struct inject_poison_pl *in = (void *)cmd->payload;
> +uint64_t dpa = ldq_le_p(>dpa);
> +CXLPoison *p;
> +
> +QLIST_FOREACH(ent, poison_list, node) {
> +if (dpa >= ent->start && dpa + 64 <= ent->start + ent->length) {
> +return CXL_MBOX_SUCCESS;
> +}
> +}
> +
> +if (ct3d->poison_list_cnt == CXL_POISON_LIST_LIMIT) {
> +return CXL_MBOX_INJECT_POISON_LIMIT;
> +}
> +p = g_new0(CXLPoison, 1);
> +
> +p->length = 64;
> +p->start = dpa;
> +p->type = CXL_POISON_TYPE_INJECTED;
> +
> +/*
> + * Possible todo: Merge with existing entry if next to it and if same 
> type
> + */
> +QLIST_INSERT_HEAD(poison_list, p, node);
> +ct3d->poison_list_cnt++;
> +
> +return CXL_MBOX_SUCCESS;
> +}
> +
>  #define IMMEDIATE_CONFIG_CHANGE (1 << 1)
>  #define IMMEDIATE_DATA_CHANGE (1 << 2)
>  #define IMMEDIATE_POLICY_CHANGE (1 << 3)
> @@ -501,6 +540,8 @@ static struct cxl_cmd cxl_cmd_set[256][256] = {
>  ~0, IMMEDIATE_CONFIG_CHANGE | IMMEDIATE_DATA_CHANGE },
>  [MEDIA_AND_POISON][GET_POISON_LIST] = { 
> "MEDIA_AND_POISON_GET_POISON_LIST",
>  cmd_media_get_poison_list, 16, 0 },
> +[MEDIA_AND_POISON][INJECT_POISON] = { "MEDIA_AND_POISON_INJECT_POISON",
> +cmd_media_inject_poison, 8, 0 },
>  };
>  
>  void cxl_process_mailbox(CXLDeviceState *cxl_dstate)
> -- 
> 2.37.2
> 

-- 
John Smith
My name is not generic at all.

Re: [PATCH v4 6/6] hw/cxl: Add clear poison mailbox command support.

[Fan Ni]

The 03/03/2023 15:09, Jonathan Cameron wrote:
> Current implementation is very simple so many of the corner
> cases do not exist (e.g. fragmenting larger poison list entries)
> 
> Signed-off-by: Jonathan Cameron 
> ---

Reviewed-by: Fan Ni 

One minor thing as mentioned below.
> v4:
> - Fix off by one on check of edge of vmr (cut and paste from similar
>   but long fixed in the volatile memory series)
> - Drop unnecessary overflow check.
> - Ensure that even in case of overflow we still delete the element
>   replaced (in the hole punching case)
> ---
>  hw/cxl/cxl-mailbox-utils.c  | 77 +
>  hw/mem/cxl_type3.c  | 36 +
>  include/hw/cxl/cxl_device.h |  1 +
>  3 files changed, 114 insertions(+)
> 
> diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
> index 64a3f3c1bf..0b30307fa3 100644
> --- a/hw/cxl/cxl-mailbox-utils.c
> +++ b/hw/cxl/cxl-mailbox-utils.c
> @@ -65,6 +65,7 @@ enum {
>  MEDIA_AND_POISON = 0x43,
>  #define GET_POISON_LIST0x0
>  #define INJECT_POISON  0x1
> +#define CLEAR_POISON   0x2
>  };
>  
>  /* 8.2.8.4.5.1 Command Return Codes */
> @@ -511,6 +512,80 @@ static CXLRetCode cmd_media_inject_poison(struct cxl_cmd 
> *cmd,
>  return CXL_MBOX_SUCCESS;
>  }
>  
> +static CXLRetCode cmd_media_clear_poison(struct cxl_cmd *cmd,
> + CXLDeviceState *cxl_dstate,
> + uint16_t *len)


Since 'len' is never used in the function, I am wondering whether it
would be better to rename it to makes that more obvious like "len_unused".

> +{
> +CXLType3Dev *ct3d = container_of(cxl_dstate, CXLType3Dev, cxl_dstate);
> +CXLPoisonList *poison_list = >poison_list;
> +CXLType3Class *cvc = CXL_TYPE3_GET_CLASS(ct3d);
> +struct clear_poison_pl {
> +uint64_t dpa;
> +uint8_t data[64];
> +};
> +CXLPoison *ent;
> +uint64_t dpa;
> +
> +struct clear_poison_pl *in = (void *)cmd->payload;
> +
> +dpa = ldq_le_p(>dpa);
> +if (dpa + 64 > cxl_dstate->mem_size) {
> +return CXL_MBOX_INVALID_PA;
> +}
> +
> +/* Always exit loop on entry removal so no need for safe variant */
> +QLIST_FOREACH(ent, poison_list, node) {
> +/*
> + * Test for contained in entry. Simpler than general case
> + * as clearing 64 bytes and entries 64 byte aligned
> + */
> +if ((dpa < ent->start) || (dpa >= ent->start + ent->length)) {
> +continue;
> +}
> +/* Do accounting early as we know one will go away */
> +ct3d->poison_list_cnt--;
> +if (dpa > ent->start) {
> +CXLPoison *frag;
> +/* Cannot overflow as replacing existing entry */
> +
> +frag = g_new0(CXLPoison, 1);
> +
> +frag->start = ent->start;
> +frag->length = dpa - ent->start;
> +frag->type = ent->type;
> +
> +QLIST_INSERT_HEAD(poison_list, frag, node);
> +ct3d->poison_list_cnt++;
> +}
> +if (dpa + 64 < ent->start + ent->length) {
> +CXLPoison *frag;
> +
> +if (ct3d->poison_list_cnt == CXL_POISON_LIST_LIMIT) {
> +cxl_set_poison_list_overflowed(ct3d);
> +} else {
> +frag = g_new0(CXLPoison, 1);
> +
> +frag->start = dpa + 64;
> +frag->length = ent->start + ent->length - frag->start;
> +frag->type = ent->type;
> +QLIST_INSERT_HEAD(poison_list, frag, node);
> +ct3d->poison_list_cnt++;
> +}
> +}
> +/* Any fragments have been added, free original entry */
> +QLIST_REMOVE(ent, node);
> +g_free(ent);
> +break;
> +}
> +/* Clearing a region with no poison is not an error so always do so */
> +if (cvc->set_cacheline)
> +if (!cvc->set_cacheline(ct3d, dpa, in->data)) {
> +return CXL_MBOX_INTERNAL_ERROR;
> +}
> +
> +return CXL_MBOX_SUCCESS;
> +}
> +
>  #define IMMEDIATE_CONFIG_CHANGE (1 << 1)
>  #define IMMEDIATE_DATA_CHANGE (1 << 2)
>  #define IMMEDIATE_POLICY_CHANGE (1 << 3)
> @@ -542,6 +617,8 @@ static struct cxl_cmd cxl_cmd_set[256][256] = {
>  cmd_media_get_poison_list, 16, 0 },
>  [MEDIA_AND_POISON][INJECT_POISON] = { "MEDIA_AND_POISON_INJECT_POISON",
>  cmd_media_inject_poison, 8, 0 },
> +   

Re: [RESEND PATCH v6 8/8] hw/mem/cxl_type3: Add CXL RAS Error Injection Support.

[Fan Ni]

On Thu, Mar 02, 2023 at 01:37:09PM +, Jonathan Cameron wrote:
> CXL uses PCI AER Internal errors to signal to the host that an error has
> occurred. The host can then read more detailed status from the CXL RAS
> capability.
> 
> For uncorrectable errors: support multiple injection in one operation
> as this is needed to reliably test multiple header logging support in an
> OS. The equivalent feature doesn't exist for correctable errors, so only
> one error need be injected at a time.
> 
> Note:
>  - Header content needs to be manually specified in a fashion that
>matches the specification for what can be in the header for each
>error type.
> 
> Injection via QMP:
> { "execute": "qmp_capabilities" }
> ...
> { "execute": "cxl-inject-uncorrectable-errors",
>   "arguments": {
> "path": "/machine/peripheral/cxl-pmem0",
> "errors": [
> {
> "type": "cache-address-parity",
> "header": [ 3, 4]
> },
> {
> "type": "cache-data-parity",
> "header": 
> [0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31]
> },
> {
> "type": "internal",
> "header": [ 1, 2, 4]
> }
> ]
>   }}
> ...
> { "execute": "cxl-inject-correctable-error",
> "arguments": {
> "path": "/machine/peripheral/cxl-pmem0",
> "type": "physical"
> } }
> 
> Signed-off-by: Jonathan Cameron 
> ---

Reviewed-by: Fan Ni 

One minor thing, see below in "typedef struct CXLError".

> v6: (Thanks to Philippe Mathieu-Daudé)
> - Add Since entries in cxl.json
> - Add error prints in the stub functions so that if they are called without
>   CONFIG_CXL_MEM_DEVICE then we get a useful print rather than just silently
>   eating them.
> 
> ---
>  hw/cxl/cxl-component-utils.c   |   4 +-
>  hw/mem/cxl_type3.c | 281 +
>  hw/mem/cxl_type3_stubs.c   |  17 ++
>  hw/mem/meson.build |   2 +
>  include/hw/cxl/cxl_component.h |  26 +++
>  include/hw/cxl/cxl_device.h|  11 ++
>  qapi/cxl.json  | 128 +++
>  qapi/meson.build   |   1 +
>  qapi/qapi-schema.json  |   1 +
>  9 files changed, 470 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c
> index 737b4764b9..b665d4f565 100644
> --- a/hw/cxl/cxl-component-utils.c
> +++ b/hw/cxl/cxl-component-utils.c
> @@ -142,16 +142,18 @@ static void ras_init_common(uint32_t *reg_state, 
> uint32_t *write_msk)
>   * be handled as RO.
>   */
>  stl_le_p(reg_state + R_CXL_RAS_UNC_ERR_STATUS, 0);
> +stl_le_p(write_msk + R_CXL_RAS_UNC_ERR_STATUS, 0x1cfff);
>  /* Bits 12-13 and 17-31 reserved in CXL 2.0 */
>  stl_le_p(reg_state + R_CXL_RAS_UNC_ERR_MASK, 0x1cfff);
>  stl_le_p(write_msk + R_CXL_RAS_UNC_ERR_MASK, 0x1cfff);
>  stl_le_p(reg_state + R_CXL_RAS_UNC_ERR_SEVERITY, 0x1cfff);
>  stl_le_p(write_msk + R_CXL_RAS_UNC_ERR_SEVERITY, 0x1cfff);
>  stl_le_p(reg_state + R_CXL_RAS_COR_ERR_STATUS, 0);
> +stl_le_p(write_msk + R_CXL_RAS_COR_ERR_STATUS, 0x7f);
>  stl_le_p(reg_state + R_CXL_RAS_COR_ERR_MASK, 0x7f);
>  stl_le_p(write_msk + R_CXL_RAS_COR_ERR_MASK, 0x7f);
>  /* CXL switches and devices must set */
> -stl_le_p(reg_state + R_CXL_RAS_ERR_CAP_CTRL, 0x00);
> +stl_le_p(reg_state + R_CXL_RAS_ERR_CAP_CTRL, 0x200);
>  }
>  
>  static void hdm_init_common(uint32_t *reg_state, uint32_t *write_msk,
> diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
> index 6cdd988d1d..abe60b362c 100644
> --- a/hw/mem/cxl_type3.c
> +++ b/hw/mem/cxl_type3.c
> @@ -1,6 +1,7 @@
>  #include "qemu/osdep.h"
>  #include "qemu/units.h"
>  #include "qemu/error-report.h"
> +#include "qapi/qapi-commands-cxl.h"
>  #include "hw/mem/memory-device.h"
>  #include "hw/mem/pc-dimm.h"
>  #include "hw/pci/pci.h"
> @@ -323,6 +324,66 @@ static void hdm_decoder_commit(CXLType3Dev *ct3d, int 
> which)
>  ARRAY_FIELD_DP32(cache_mem, CXL_HDM_DECODER0_CTRL, COMMITTED, 1);
>  }
>  
> +static int ct3d_qmp_uncor_err_to_cxl(CxlUncorErrorType qmp_err)
> +{
> +switch (qmp_err) {
> +case CXL_UNCOR_ERROR_TYPE_CACHE_DATA_PARITY:
> +return CXL_RAS_UNC_ERR_CACHE_DATA_PARITY;
> +case CXL_UNCOR_ERROR_TYPE_CACHE_ADDRESS_PARITY:
&

Re: [RESEND PATCH v6 7/8] hw/pci/aer: Make PCIE AER error injection facility available for other emulation to use.

[Fan Ni]

On Thu, Mar 02, 2023 at 01:37:08PM +, Jonathan Cameron wrote:
> This infrastructure will be reused for CXL RAS error injection
> in patches that follow.
> 
> Reviewed-by: Dave Jiang 
> Reviewed-by: Philippe Mathieu-Daudé 
> Signed-off-by: Jonathan Cameron 
> ---
>  hw/pci/pci-internal.h | 1 -
>  include/hw/pci/pcie_aer.h | 1 +
>  2 files changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/hw/pci/pci-internal.h b/hw/pci/pci-internal.h
> index 2ea356bdf5..a7d6d8a732 100644
> --- a/hw/pci/pci-internal.h
> +++ b/hw/pci/pci-internal.h
> @@ -20,6 +20,5 @@ void pcibus_dev_print(Monitor *mon, DeviceState *dev, int 
> indent);
>  
>  int pcie_aer_parse_error_string(const char *error_name,
>  uint32_t *status, bool *correctable);
> -int pcie_aer_inject_error(PCIDevice *dev, const PCIEAERErr *err);
>  
>  #endif
> diff --git a/include/hw/pci/pcie_aer.h b/include/hw/pci/pcie_aer.h
> index 65e71d98fe..1234fdc4e2 100644
> --- a/include/hw/pci/pcie_aer.h
> +++ b/include/hw/pci/pcie_aer.h
> @@ -100,4 +100,5 @@ void pcie_aer_root_write_config(PCIDevice *dev,
>  uint32_t addr, uint32_t val, int len,
>  uint32_t root_cmd_prev);
>  
> +int pcie_aer_inject_error(PCIDevice *dev, const PCIEAERErr *err);
>  #endif /* QEMU_PCIE_AER_H */
> -- 
> 2.37.2
> 
> 

Reviewed-by: Fan Ni 

Re: [RESEND PATCH v6 6/8] hw/cxl: Fix endian issues in CXL RAS capability defaults / masks

[Fan Ni]

On Thu, Mar 02, 2023 at 01:37:07PM +, Jonathan Cameron wrote:
> As these are about to be modified, fix the endian handle for
> this set of registers rather than making it worse.
> 
> Note that CXL is currently only supported in QEMU on
> x86 (arm64 patches out of tree) so we aren't going to yet hit
> an problems with big endian. However it is good to avoid making
> things worse for that support in the future.
> 
> Reviewed-by: Dave Jiang 
> Reviewed-by: Philippe Mathieu-Daudé 
> Signed-off-by: Jonathan Cameron 
> ---

Reviewed-by: Fan Ni 

>  hw/cxl/cxl-component-utils.c | 18 +-
>  1 file changed, 9 insertions(+), 9 deletions(-)
> 
> diff --git a/hw/cxl/cxl-component-utils.c b/hw/cxl/cxl-component-utils.c
> index 3edd303a33..737b4764b9 100644
> --- a/hw/cxl/cxl-component-utils.c
> +++ b/hw/cxl/cxl-component-utils.c
> @@ -141,17 +141,17 @@ static void ras_init_common(uint32_t *reg_state, 
> uint32_t *write_msk)
>   * Error status is RW1C but given bits are not yet set, it can
>   * be handled as RO.
>   */
> -reg_state[R_CXL_RAS_UNC_ERR_STATUS] = 0;
> +stl_le_p(reg_state + R_CXL_RAS_UNC_ERR_STATUS, 0);
>  /* Bits 12-13 and 17-31 reserved in CXL 2.0 */
> -reg_state[R_CXL_RAS_UNC_ERR_MASK] = 0x1cfff;
> -write_msk[R_CXL_RAS_UNC_ERR_MASK] = 0x1cfff;
> -reg_state[R_CXL_RAS_UNC_ERR_SEVERITY] = 0x1cfff;
> -write_msk[R_CXL_RAS_UNC_ERR_SEVERITY] = 0x1cfff;
> -reg_state[R_CXL_RAS_COR_ERR_STATUS] = 0;
> -reg_state[R_CXL_RAS_COR_ERR_MASK] = 0x7f;
> -write_msk[R_CXL_RAS_COR_ERR_MASK] = 0x7f;
> +stl_le_p(reg_state + R_CXL_RAS_UNC_ERR_MASK, 0x1cfff);
> +stl_le_p(write_msk + R_CXL_RAS_UNC_ERR_MASK, 0x1cfff);
> +stl_le_p(reg_state + R_CXL_RAS_UNC_ERR_SEVERITY, 0x1cfff);
> +stl_le_p(write_msk + R_CXL_RAS_UNC_ERR_SEVERITY, 0x1cfff);
> +stl_le_p(reg_state + R_CXL_RAS_COR_ERR_STATUS, 0);
> +stl_le_p(reg_state + R_CXL_RAS_COR_ERR_MASK, 0x7f);
> +stl_le_p(write_msk + R_CXL_RAS_COR_ERR_MASK, 0x7f);
>  /* CXL switches and devices must set */
> -reg_state[R_CXL_RAS_ERR_CAP_CTRL] = 0x00;
> +stl_le_p(reg_state + R_CXL_RAS_ERR_CAP_CTRL, 0x00);
>  }
>  
>  static void hdm_init_common(uint32_t *reg_state, uint32_t *write_msk,
> -- 
> 2.37.2
> 
> 

Re: [RESEND PATCH v6 5/8] hw/mem/cxl-type3: Add AER extended capability

[Fan Ni]

On Thu, Mar 02, 2023 at 01:37:06PM +, Jonathan Cameron wrote:
> This enables AER error injection to function as expected.
> It is intended as a building block in enabling CXL RAS error injection
> in the following patches.
> 
> Signed-off-by: Jonathan Cameron 
> Reviewed-by: Dave Jiang 
> ---

Reviewed-by: Fan Ni 

>  hw/mem/cxl_type3.c | 13 +
>  1 file changed, 13 insertions(+)
> 
> diff --git a/hw/mem/cxl_type3.c b/hw/mem/cxl_type3.c
> index 217a5e639b..6cdd988d1d 100644
> --- a/hw/mem/cxl_type3.c
> +++ b/hw/mem/cxl_type3.c
> @@ -250,6 +250,7 @@ static void ct3d_config_write(PCIDevice *pci_dev, 
> uint32_t addr, uint32_t val,
>  
>  pcie_doe_write_config(>doe_cdat, addr, val, size);
>  pci_default_write_config(pci_dev, addr, val, size);
> +pcie_aer_write_config(pci_dev, addr, val, size);
>  }
>  
>  /*
> @@ -452,8 +453,19 @@ static void ct3_realize(PCIDevice *pci_dev, Error **errp)
>  cxl_cstate->cdat.free_cdat_table = ct3_free_cdat_table;
>  cxl_cstate->cdat.private = ct3d;
>  cxl_doe_cdat_init(cxl_cstate, errp);
> +
> +pcie_cap_deverr_init(pci_dev);
> +/* Leave a bit of room for expansion */
> +rc = pcie_aer_init(pci_dev, PCI_ERR_VER, 0x200, PCI_ERR_SIZEOF, NULL);
> +if (rc) {
> +goto err_release_cdat;
> +}
> +
>  return;
>  
> +err_release_cdat:
> +cxl_doe_cdat_release(cxl_cstate);
> +g_free(regs->special_ops);
>  err_address_space_free:
>  address_space_destroy(>hostmem_as);
>  return;
> @@ -465,6 +477,7 @@ static void ct3_exit(PCIDevice *pci_dev)
>  CXLComponentState *cxl_cstate = >cxl_cstate;
>  ComponentRegisters *regs = _cstate->crb;
>  
> +pcie_aer_exit(pci_dev);
>  cxl_doe_cdat_release(cxl_cstate);
>  g_free(regs->special_ops);
>  address_space_destroy(>hostmem_as);
> -- 
> 2.37.2
> 
> 

Re: [RESEND PATCH v6 4/8] hw/pci-bridge/cxl_root_port: Wire up MSI

[Fan Ni]

On Thu, Mar 02, 2023 at 01:37:05PM +, Jonathan Cameron wrote:
> Done to avoid fixing ACPI route description of traditional PCI interrupts on 
> q35
> and because we should probably move with the times anyway.
> 
> Signed-off-by: Jonathan Cameron 
> Reviewed-by: Dave Jiang 
> ---

Reviewed-by: Fan Ni 

>  hw/pci-bridge/cxl_root_port.c | 61 +++
>  1 file changed, 61 insertions(+)
> 
> diff --git a/hw/pci-bridge/cxl_root_port.c b/hw/pci-bridge/cxl_root_port.c
> index 00195257f7..7dfd20aa67 100644
> --- a/hw/pci-bridge/cxl_root_port.c
> +++ b/hw/pci-bridge/cxl_root_port.c
> @@ -22,6 +22,7 @@
>  #include "qemu/range.h"
>  #include "hw/pci/pci_bridge.h"
>  #include "hw/pci/pcie_port.h"
> +#include "hw/pci/msi.h"
>  #include "hw/qdev-properties.h"
>  #include "hw/sysbus.h"
>  #include "qapi/error.h"
> @@ -29,6 +30,10 @@
>  
>  #define CXL_ROOT_PORT_DID 0x7075
>  
> +#define CXL_RP_MSI_OFFSET   0x60
> +#define CXL_RP_MSI_SUPPORTED_FLAGS  PCI_MSI_FLAGS_MASKBIT
> +#define CXL_RP_MSI_NR_VECTOR2
> +
>  /* Copied from the gen root port which we derive */
>  #define GEN_PCIE_ROOT_PORT_AER_OFFSET 0x100
>  #define GEN_PCIE_ROOT_PORT_ACS_OFFSET \
> @@ -47,6 +52,49 @@ typedef struct CXLRootPort {
>  #define TYPE_CXL_ROOT_PORT "cxl-rp"
>  DECLARE_INSTANCE_CHECKER(CXLRootPort, CXL_ROOT_PORT, TYPE_CXL_ROOT_PORT)
>  
> +/*
> + * If two MSI vector are allocated, Advanced Error Interrupt Message Number
> + * is 1. otherwise 0.
> + * 17.12.5.10 RPERRSTS,  32:27 bit Advanced Error Interrupt Message Number.
> + */
> +static uint8_t cxl_rp_aer_vector(const PCIDevice *d)
> +{
> +switch (msi_nr_vectors_allocated(d)) {
> +case 1:
> +return 0;
> +case 2:
> +return 1;
> +case 4:
> +case 8:
> +case 16:
> +case 32:
> +default:
> +break;
> +}
> +abort();
> +return 0;
> +}
> +
> +static int cxl_rp_interrupts_init(PCIDevice *d, Error **errp)
> +{
> +int rc;
> +
> +rc = msi_init(d, CXL_RP_MSI_OFFSET, CXL_RP_MSI_NR_VECTOR,
> +  CXL_RP_MSI_SUPPORTED_FLAGS & PCI_MSI_FLAGS_64BIT,
> +  CXL_RP_MSI_SUPPORTED_FLAGS & PCI_MSI_FLAGS_MASKBIT,
> +  errp);
> +if (rc < 0) {
> +assert(rc == -ENOTSUP);
> +}
> +
> +return rc;
> +}
> +
> +static void cxl_rp_interrupts_uninit(PCIDevice *d)
> +{
> +msi_uninit(d);
> +}
> +
>  static void latch_registers(CXLRootPort *crp)
>  {
>  uint32_t *reg_state = crp->cxl_cstate.crb.cache_mem_registers;
> @@ -183,6 +231,15 @@ static void cxl_rp_dvsec_write_config(PCIDevice *dev, 
> uint32_t addr,
>  }
>  }
>  
> +static void cxl_rp_aer_vector_update(PCIDevice *d)
> +{
> +PCIERootPortClass *rpc = PCIE_ROOT_PORT_GET_CLASS(d);
> +
> +if (rpc->aer_vector) {
> +pcie_aer_root_set_vector(d, rpc->aer_vector(d));
> +}
> +}
> +
>  static void cxl_rp_write_config(PCIDevice *d, uint32_t address, uint32_t val,
>  int len)
>  {
> @@ -192,6 +249,7 @@ static void cxl_rp_write_config(PCIDevice *d, uint32_t 
> address, uint32_t val,
>  
>  pcie_cap_slot_get(d, _ctl, _sta);
>  pci_bridge_write_config(d, address, val, len);
> +cxl_rp_aer_vector_update(d);
>  pcie_cap_flr_write_config(d, address, val, len);
>  pcie_cap_slot_write_config(d, slt_ctl, slt_sta, address, val, len);
>  pcie_aer_write_config(d, address, val, len);
> @@ -220,6 +278,9 @@ static void cxl_root_port_class_init(ObjectClass *oc, 
> void *data)
>  
>  rpc->aer_offset = GEN_PCIE_ROOT_PORT_AER_OFFSET;
>  rpc->acs_offset = GEN_PCIE_ROOT_PORT_ACS_OFFSET;
> +rpc->aer_vector = cxl_rp_aer_vector;
> +rpc->interrupts_init = cxl_rp_interrupts_init;
> +rpc->interrupts_uninit = cxl_rp_interrupts_uninit;
>  
>  dc->hotpluggable = false;
>  }
> -- 
> 2.37.2
> 
> 

Re: [RESEND PATCH v6 3/8] hw/pci-bridge/cxl_root_port: Wire up AER

[Fan Ni]

On Thu, Mar 02, 2023 at 01:37:04PM +, Jonathan Cameron wrote:
> We are missing necessary config write handling for AER emulation in
> the CXL root port. Add it based on pcie_root_port.c
> 
> Signed-off-by: Jonathan Cameron 
> Reviewed-by: Dave Jiang 
> ---
>  hw/pci-bridge/cxl_root_port.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/hw/pci-bridge/cxl_root_port.c b/hw/pci-bridge/cxl_root_port.c
> index 6664783974..00195257f7 100644
> --- a/hw/pci-bridge/cxl_root_port.c
> +++ b/hw/pci-bridge/cxl_root_port.c
> @@ -187,12 +187,15 @@ static void cxl_rp_write_config(PCIDevice *d, uint32_t 
> address, uint32_t val,
>  int len)
>  {
>  uint16_t slt_ctl, slt_sta;
> +uint32_t root_cmd =
> +pci_get_long(d->config + d->exp.aer_cap + PCI_ERR_ROOT_COMMAND);
>  
>  pcie_cap_slot_get(d, _ctl, _sta);
>  pci_bridge_write_config(d, address, val, len);
>  pcie_cap_flr_write_config(d, address, val, len);
>  pcie_cap_slot_write_config(d, slt_ctl, slt_sta, address, val, len);
>  pcie_aer_write_config(d, address, val, len);
> +pcie_aer_root_write_config(d, address, val, len, root_cmd);
>  
>      cxl_rp_dvsec_write_config(d, address, val, len);
>  }
> -- 
> 2.37.2
> 
> 

Reviewed-by: Fan Ni 

Re: [RESEND PATCH v6 2/8] hw/pci/aer: Add missing routing for AER errors

[Fan Ni]

On Thu, Mar 02, 2023 at 01:37:03PM +, Jonathan Cameron wrote:
> PCIe r6.0 Figure 6-3 "Pseudo Logic Diagram for Selected Error Message Control
> and Status Bits" includes a right hand branch under "All PCI Express devices"
> that allows for messages to be generated or sent onwards without SERR#
> being set as long as the appropriate per error class bit in the PCIe
> Device Control Register is set.
> 
> Implement that branch thus enabling routing of ERR_COR, ERR_NONFATAL
> and ERR_FATAL under OSes that set these bits appropriately (e.g. Linux)
> 
> Signed-off-by: Jonathan Cameron 
> Reviewed-by: Dave Jiang 
> ---

Reviewed-by: Fan Ni 

>  hw/pci/pcie_aer.c | 10 +-
>  1 file changed, 9 insertions(+), 1 deletion(-)
> 
> diff --git a/hw/pci/pcie_aer.c b/hw/pci/pcie_aer.c
> index 909e027d99..103667c368 100644
> --- a/hw/pci/pcie_aer.c
> +++ b/hw/pci/pcie_aer.c
> @@ -192,8 +192,16 @@ static void pcie_aer_update_uncor_status(PCIDevice *dev)
>  static bool
>  pcie_aer_msg_alldev(PCIDevice *dev, const PCIEAERMsg *msg)
>  {
> +uint16_t devctl = pci_get_word(dev->config + dev->exp.exp_cap +
> +   PCI_EXP_DEVCTL);
>  if (!(pcie_aer_msg_is_uncor(msg) &&
> -  (pci_get_word(dev->config + PCI_COMMAND) & PCI_COMMAND_SERR))) {
> +  (pci_get_word(dev->config + PCI_COMMAND) & PCI_COMMAND_SERR)) &&
> +!((msg->severity == PCI_ERR_ROOT_CMD_NONFATAL_EN) &&
> +  (devctl & PCI_EXP_DEVCTL_NFERE)) &&
> +!((msg->severity == PCI_ERR_ROOT_CMD_COR_EN) &&
> +  (devctl & PCI_EXP_DEVCTL_CERE)) &&
> +!((msg->severity == PCI_ERR_ROOT_CMD_FATAL_EN) &&
> +  (devctl & PCI_EXP_DEVCTL_FERE))) {
>  return false;
>  }
>  
> -- 
> 2.37.2
> 
> 

Re: [RESEND PATCH v6 1/8] hw/pci/aer: Implement PCI_ERR_UNCOR_MASK register

[Fan Ni]

On Thu, Mar 02, 2023 at 01:37:02PM +, Jonathan Cameron wrote:
> This register in AER should be both writeable and should
> have a default value with a couple of the errors masked
> including the Uncorrectable Internal Error used by CXL for
> it's error reporting.
> 
> Signed-off-by: Jonathan Cameron 
> Reviewed-by: Dave Jiang 
> ---

Reviewed-by: Fan Ni 

>  hw/pci/pcie_aer.c  | 4 
>  include/hw/pci/pcie_regs.h | 3 +++
>  2 files changed, 7 insertions(+)
> 
> diff --git a/hw/pci/pcie_aer.c b/hw/pci/pcie_aer.c
> index 9a19be44ae..909e027d99 100644
> --- a/hw/pci/pcie_aer.c
> +++ b/hw/pci/pcie_aer.c
> @@ -112,6 +112,10 @@ int pcie_aer_init(PCIDevice *dev, uint8_t cap_ver, 
> uint16_t offset,
>  
>  pci_set_long(dev->w1cmask + offset + PCI_ERR_UNCOR_STATUS,
>   PCI_ERR_UNC_SUPPORTED);
> +pci_set_long(dev->config + offset + PCI_ERR_UNCOR_MASK,
> + PCI_ERR_UNC_MASK_DEFAULT);
> +pci_set_long(dev->wmask + offset + PCI_ERR_UNCOR_MASK,
> + PCI_ERR_UNC_SUPPORTED);
>  
>  pci_set_long(dev->config + offset + PCI_ERR_UNCOR_SEVER,
>   PCI_ERR_UNC_SEVERITY_DEFAULT);
> diff --git a/include/hw/pci/pcie_regs.h b/include/hw/pci/pcie_regs.h
> index 963dc2e170..6ec4785448 100644
> --- a/include/hw/pci/pcie_regs.h
> +++ b/include/hw/pci/pcie_regs.h
> @@ -155,6 +155,9 @@ typedef enum PCIExpLinkWidth {
>   PCI_ERR_UNC_ATOP_EBLOCKED |\
>   PCI_ERR_UNC_TLP_PRF_BLOCKED)
>  
> +#define PCI_ERR_UNC_MASK_DEFAULT(PCI_ERR_UNC_INTN | \
> + PCI_ERR_UNC_TLP_PRF_BLOCKED)
> +
>  #define PCI_ERR_UNC_SEVERITY_DEFAULT(PCI_ERR_UNC_DLP |  \
>   PCI_ERR_UNC_SDN |  \
>   PCI_ERR_UNC_FCP |  \
> -- 
> 2.37.2
> 
> 

Re: [PATCH v2 3/6] bswap: Add the ability to store to an unaligned 24 bit field

[Fan Ni]

On Mon, Feb 27, 2023 at 05:03:08PM +, Jonathan Cameron wrote:
> From: Ira Weiny 
> 
> CXL has 24 bit unaligned fields which need to be stored to.  CXL is
> specified as little endian.
> 
> Define st24_le_p() and the supporting functions to store such a field
> from a 32 bit host native value.
> 
> The use of b, w, l, q as the size specifier is limiting.  So "24" was
> used for the size part of the function name.
> 
> Signed-off-by: Ira Weiny 
> Signed-off-by: Jonathan Cameron 
> 

Reviewed-by: Fan Ni 

> ---
> v7:
>   - Pulled this patch out of the CXL events series as Ira pointed
> out it can be used to simplify this series.
> ---
>  include/qemu/bswap.h | 23 +++
>  1 file changed, 23 insertions(+)
> 
> diff --git a/include/qemu/bswap.h b/include/qemu/bswap.h
> index 15a78c0db5..ee71cbeaaa 100644
> --- a/include/qemu/bswap.h
> +++ b/include/qemu/bswap.h
> @@ -8,11 +8,23 @@
>  #undef  bswap64
>  #define bswap64(_x) __builtin_bswap64(_x)
>  
> +static inline uint32_t bswap24(uint32_t x)
> +{
> +return (((x & 0x00ffU) << 16) |
> +((x & 0xff00U) <<  0) |
> +((x & 0x00ffU) >> 16));
> +}
> +
>  static inline void bswap16s(uint16_t *s)
>  {
>  *s = __builtin_bswap16(*s);
>  }
>  
> +static inline void bswap24s(uint32_t *s)
> +{
> +*s = bswap24(*s);
> +}
> +
>  static inline void bswap32s(uint32_t *s)
>  {
>  *s = __builtin_bswap32(*s);
> @@ -176,6 +188,7 @@ CPU_CONVERT(le, 64, uint64_t)
>   * size is:
>   *   b: 8 bits
>   *   w: 16 bits
> + *   24: 24 bits
>   *   l: 32 bits
>   *   q: 64 bits
>   *
> @@ -248,6 +261,11 @@ static inline void stw_he_p(void *ptr, uint16_t v)
>  __builtin_memcpy(ptr, , sizeof(v));
>  }
>  
> +static inline void st24_he_p(void *ptr, uint32_t v)
> +{
> +__builtin_memcpy(ptr, , 3);
> +}
> +
>  static inline int ldl_he_p(const void *ptr)
>  {
>  int32_t r;
> @@ -297,6 +315,11 @@ static inline void stw_le_p(void *ptr, uint16_t v)
>  stw_he_p(ptr, le_bswap(v, 16));
>  }
>  
> +static inline void st24_le_p(void *ptr, uint32_t v)
> +{
> +st24_he_p(ptr, le_bswap(v, 24));
> +}
> +
>  static inline void stl_le_p(void *ptr, uint32_t v)
>  {
>  stl_he_p(ptr, le_bswap(v, 32));
> -- 
> 2.37.2
> 
> 

Re: [PATCH v2 2/6] hw/cxl: Introduce cxl_device_get_timestamp() utility function

[Fan Ni]

On Mon, Feb 27, 2023 at 05:03:07PM +, Jonathan Cameron wrote:
> From: Ira Weiny 
> 
> There are new users of this functionality coming shortly so factor
> it out from the GET_TIMESTAMP mailbox command handling.
> 
> Signed-off-by: Ira Weiny 
> Signed-off-by: Jonathan Cameron 

Reviewed-by: Fan Ni 

> ---
>  hw/cxl/cxl-device-utils.c   | 15 +++
>  hw/cxl/cxl-mailbox-utils.c  | 11 +--
>  include/hw/cxl/cxl_device.h |  2 ++
>  3 files changed, 18 insertions(+), 10 deletions(-)
> 
> diff --git a/hw/cxl/cxl-device-utils.c b/hw/cxl/cxl-device-utils.c
> index 4c5e88aaf5..86e1cea8ce 100644
> --- a/hw/cxl/cxl-device-utils.c
> +++ b/hw/cxl/cxl-device-utils.c
> @@ -269,3 +269,18 @@ void cxl_device_register_init_common(CXLDeviceState 
> *cxl_dstate)
>  
>  cxl_initialize_mailbox(cxl_dstate);
>  }
> +
> +uint64_t cxl_device_get_timestamp(CXLDeviceState *cxl_dstate)
> +{
> +uint64_t time, delta;
> +uint64_t final_time = 0;
> +
> +if (cxl_dstate->timestamp.set) {
> +/* Find the delta from the last time the host set the time. */
> +time = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
> +delta = time - cxl_dstate->timestamp.last_set;
> +final_time = cxl_dstate->timestamp.host_set + delta;
> +}
> +
> +return final_time;
> +}
> diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
> index 7b2aef0d67..702e16ca20 100644
> --- a/hw/cxl/cxl-mailbox-utils.c
> +++ b/hw/cxl/cxl-mailbox-utils.c
> @@ -163,17 +163,8 @@ static CXLRetCode cmd_timestamp_get(struct cxl_cmd *cmd,
>  CXLDeviceState *cxl_dstate,
>  uint16_t *len)
>  {
> -uint64_t time, delta;
> -uint64_t final_time = 0;
> -
> -if (cxl_dstate->timestamp.set) {
> -/* First find the delta from the last time the host set the time. */
> -time = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
> -delta = time - cxl_dstate->timestamp.last_set;
> -final_time = cxl_dstate->timestamp.host_set + delta;
> -}
> +uint64_t final_time = cxl_device_get_timestamp(cxl_dstate);
>  
> -/* Then adjust the actual time */
>  stq_le_p(cmd->payload, final_time);
>  *len = 8;
>  
> diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
> index edb9791bab..02befda0f6 100644
> --- a/include/hw/cxl/cxl_device.h
> +++ b/include/hw/cxl/cxl_device.h
> @@ -287,4 +287,6 @@ MemTxResult cxl_type3_read(PCIDevice *d, hwaddr 
> host_addr, uint64_t *data,
>  MemTxResult cxl_type3_write(PCIDevice *d, hwaddr host_addr, uint64_t data,
>  unsigned size, MemTxAttrs attrs);
>  
> +uint64_t cxl_device_get_timestamp(CXLDeviceState *cxlds);
> +
>  #endif
> -- 
> 2.37.2
> 
> 

Re: [PATCH v2 1/6] hw/cxl: rename mailbox return code type from ret_code to CXLRetCode

[Fan Ni]

On Mon, Feb 27, 2023 at 05:03:06PM +, Jonathan Cameron wrote:
> Given the increasing usage of this mailbox return code type, now
> is a good time to switch to QEMU style naming.
> 
> Reviewed-by: Ira Weiny 
> Signed-off-by: Jonathan Cameron 
> 

Reviewed-by: Fan Ni 

> --
> v7: (thanks to Ira Weiny for review)
> - Rename in place as the move to the header isn't needed for this series
>   That move patch will now be the start of the CXL events series that
>   will follow this one.
> ---
>  hw/cxl/cxl-mailbox-utils.c | 64 +++---
>  1 file changed, 32 insertions(+), 32 deletions(-)
> 
> diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
> index ed663cc04a..7b2aef0d67 100644
> --- a/hw/cxl/cxl-mailbox-utils.c
> +++ b/hw/cxl/cxl-mailbox-utils.c
> @@ -23,7 +23,7 @@
>   * FOO= 0x7f,
>   *  #define BAR 0
>   *  2. Implement the handler
> - *static ret_code cmd_foo_bar(struct cxl_cmd *cmd,
> + *static CXLRetCode cmd_foo_bar(struct cxl_cmd *cmd,
>   *  CXLDeviceState *cxl_dstate, uint16_t 
> *len)
>   *  3. Add the command to the cxl_cmd_set[][]
>   *[FOO][BAR] = { "FOO_BAR", cmd_foo_bar, x, y },
> @@ -90,10 +90,10 @@ typedef enum {
>  CXL_MBOX_UNSUPPORTED_MAILBOX = 0x15,
>  CXL_MBOX_INVALID_PAYLOAD_LENGTH = 0x16,
>  CXL_MBOX_MAX = 0x17
> -} ret_code;
> +} CXLRetCode;
>  
>  struct cxl_cmd;
> -typedef ret_code (*opcode_handler)(struct cxl_cmd *cmd,
> +typedef CXLRetCode (*opcode_handler)(struct cxl_cmd *cmd,
> CXLDeviceState *cxl_dstate, uint16_t 
> *len);
>  struct cxl_cmd {
>  const char *name;
> @@ -105,16 +105,16 @@ struct cxl_cmd {
>  
>  #define DEFINE_MAILBOX_HANDLER_ZEROED(name, size) \
>  uint16_t __zero##name = size; \
> -static ret_code cmd_##name(struct cxl_cmd *cmd,   \
> -   CXLDeviceState *cxl_dstate, uint16_t *len) \
> +static CXLRetCode cmd_##name(struct cxl_cmd *cmd,   \
> + CXLDeviceState *cxl_dstate, uint16_t *len) \
>  { \
>  *len = __zero##name;  \
>  memset(cmd->payload, 0, *len);\
>  return CXL_MBOX_SUCCESS;  \
>  }
>  #define DEFINE_MAILBOX_HANDLER_NOP(name)  \
> -static ret_code cmd_##name(struct cxl_cmd *cmd,   \
> -   CXLDeviceState *cxl_dstate, uint16_t *len) \
> +static CXLRetCode cmd_##name(struct cxl_cmd *cmd,   \
> + CXLDeviceState *cxl_dstate, uint16_t *len) \
>  { \
>  return CXL_MBOX_SUCCESS;  \
>  }
> @@ -125,9 +125,9 @@ 
> DEFINE_MAILBOX_HANDLER_ZEROED(events_get_interrupt_policy, 4);
>  DEFINE_MAILBOX_HANDLER_NOP(events_set_interrupt_policy);
>  
>  /* 8.2.9.2.1 */
> -static ret_code cmd_firmware_update_get_info(struct cxl_cmd *cmd,
> - CXLDeviceState *cxl_dstate,
> - uint16_t *len)
> +static CXLRetCode cmd_firmware_update_get_info(struct cxl_cmd *cmd,
> +   CXLDeviceState *cxl_dstate,
> +   uint16_t *len)
>  {
>  struct {
>  uint8_t slots_supported;
> @@ -159,9 +159,9 @@ static ret_code cmd_firmware_update_get_info(struct 
> cxl_cmd *cmd,
>  }
>  
>  /* 8.2.9.3.1 */
> -static ret_code cmd_timestamp_get(struct cxl_cmd *cmd,
> -  CXLDeviceState *cxl_dstate,
> -  uint16_t *len)
> +static CXLRetCode cmd_timestamp_get(struct cxl_cmd *cmd,
> +CXLDeviceState *cxl_dstate,
> +uint16_t *len)
>  {
>  uint64_t time, delta;
>  uint64_t final_time = 0;
> @@ -181,7 +181,7 @@ static ret_code cmd_timestamp_get(struct cxl_cmd *cmd,
>  }
>  
>  /* 8.2.9.3.2 */
> -static ret_code cmd_timestamp_set(struct cxl_cmd *cmd,
> +static CXLRetCode cmd_timestamp_set(struct cxl_cmd *cmd,
>CXLDeviceState *cxl_dstate,
>uint16_t *len)
>  {
> @@ -201,9 +201,9 @

Re: [PATCH] MAINTAINERS: Add Fan Ni as Compute eXpress Link QEMU reviewer

[Fan Ni]

On Tue, Feb 28, 2023 at 11:39:26AM +, Jonathan Cameron wrote:
> Fan Ni has offered to help out with QEMU CXL emulation reviewing.
> Add him as a designated reviewer.
> 
> Signed-off-by: Jonathan Cameron 
> 

Acked-by: Fan Ni 

> --
> Thanks to Fan for stepping up after I requested help following Ben
> stepping down as co-maintainer. Fan base been active in testing
> and review recently so great to have Fan on board.
> 
> Based on patch [PATCH] MAINTAINERS: Remove CXL maintainer Ben Widawsky
> 
> Based-of: Message-id: 20230220212437.1462314-1-arm...@redhat.com
> ---
>  MAINTAINERS | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 7ea56c6ccc..838e1a91a4 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -2633,6 +2633,7 @@ T: git 
> https://urldefense.com/v3/__https://gitlab.com/vsementsov/qemu.git__;!!EwVzqGoTKBqv-0DWAJBm!UcJW-DGImFKtSoRVFI1IzD3TFOpkBbeSwsmRlUdUKPdhiOMXbTFOABJvxDRBLgLAEt0RvW0cdBG19HgqiR_VceRIA91N$
>   block
>  
>  Compute Express Link
>  M: Jonathan Cameron 
> +R: Fan Ni 
>  S: Supported
>  F: hw/cxl/
>  F: hw/mem/cxl_type3.c
> -- 
> 2.37.2
> 

Re: [PATCH v4 10/10] hw/cxl/mailbox: Use new UUID network order define for cel_uuid

[Fan Ni]

On Mon, Feb 06, 2023 at 05:28:16PM +, Jonathan Cameron wrote:
> From: Ira Weiny 
> 
> The cel_uuid was programatically generated previously because there was
> no static initializer for network order UUIDs.
> 
> Use the new network order initializer for cel_uuid.  Adjust
> cxl_initialize_mailbox() because it can't fail now.
> 
> Update specification reference.
> 
> Reviewed-by: Philippe Mathieu-Daudé 
> Reviewed-by: Gregory Price 
> Tested-by: Gregory Price 
> Signed-off-by: Ira Weiny 
> Signed-off-by: Jonathan Cameron 
> 

Reviewed-by: Fan Ni 

> ---
> v2:
> Make it const (Philippe)
> ---
>  hw/cxl/cxl-device-utils.c   |  2 +-
>  hw/cxl/cxl-mailbox-utils.c  | 13 ++---
>  include/hw/cxl/cxl_device.h |  2 +-
>  3 files changed, 8 insertions(+), 9 deletions(-)
> 
> diff --git a/hw/cxl/cxl-device-utils.c b/hw/cxl/cxl-device-utils.c
> index 83ce7a8270..4c5e88aaf5 100644
> --- a/hw/cxl/cxl-device-utils.c
> +++ b/hw/cxl/cxl-device-utils.c
> @@ -267,5 +267,5 @@ void cxl_device_register_init_common(CXLDeviceState 
> *cxl_dstate)
>  cxl_device_cap_init(cxl_dstate, MEMORY_DEVICE, 0x4000);
>  memdev_reg_init_common(cxl_dstate);
>  
> -assert(cxl_initialize_mailbox(cxl_dstate) == 0);
> +cxl_initialize_mailbox(cxl_dstate);
>  }
> diff --git a/hw/cxl/cxl-mailbox-utils.c b/hw/cxl/cxl-mailbox-utils.c
> index 3f67b665f5..206e04a4b8 100644
> --- a/hw/cxl/cxl-mailbox-utils.c
> +++ b/hw/cxl/cxl-mailbox-utils.c
> @@ -193,7 +193,11 @@ static ret_code cmd_timestamp_set(struct cxl_cmd *cmd,
>  return CXL_MBOX_SUCCESS;
>  }
>  
> -static QemuUUID cel_uuid;
> +/* CXL 3.0 8.2.9.5.2.1 Command Effects Log (CEL) */
> +static const QemuUUID cel_uuid = {
> +.data = UUID(0x0da9c0b5, 0xbf41, 0x4b78, 0x8f, 0x79,
> + 0x96, 0xb1, 0x62, 0x3b, 0x3f, 0x17)
> +};
>  
>  /* 8.2.9.4.1 */
>  static ret_code cmd_logs_get_supported(struct cxl_cmd *cmd,
> @@ -458,11 +462,8 @@ void cxl_process_mailbox(CXLDeviceState *cxl_dstate)
>   DOORBELL, 0);
>  }
>  
> -int cxl_initialize_mailbox(CXLDeviceState *cxl_dstate)
> +void cxl_initialize_mailbox(CXLDeviceState *cxl_dstate)
>  {
> -/* CXL 2.0: Table 169 Get Supported Logs Log Entry */
> -const char *cel_uuidstr = "0da9c0b5-bf41-4b78-8f79-96b1623b3f17";
> -
>  for (int set = 0; set < 256; set++) {
>  for (int cmd = 0; cmd < 256; cmd++) {
>  if (cxl_cmd_set[set][cmd].handler) {
> @@ -476,6 +477,4 @@ int cxl_initialize_mailbox(CXLDeviceState *cxl_dstate)
>  }
>  }
>  }
> -
> -return qemu_uuid_parse(cel_uuidstr, _uuid);
>  }
> diff --git a/include/hw/cxl/cxl_device.h b/include/hw/cxl/cxl_device.h
> index 250adf18b2..7e5ad65c1d 100644
> --- a/include/hw/cxl/cxl_device.h
> +++ b/include/hw/cxl/cxl_device.h
> @@ -170,7 +170,7 @@ CXL_DEVICE_CAPABILITY_HEADER_REGISTER(MEMORY_DEVICE,
>CXL_DEVICE_CAP_HDR1_OFFSET +
>CXL_DEVICE_CAP_REG_SIZE * 2)
>  
> -int cxl_initialize_mailbox(CXLDeviceState *cxl_dstate);
> +void cxl_initialize_mailbox(CXLDeviceState *cxl_dstate);
>  void cxl_process_mailbox(CXLDeviceState *cxl_dstate);
>  
>  #define cxl_device_cap_init(dstate, reg, cap_id)   \
> -- 
> 2.37.2
> 
> 

Re: [PATCH v4 09/10] qemu/uuid: Add UUID static initializer

[Fan Ni]

On Mon, Feb 06, 2023 at 05:28:15PM +, Jonathan Cameron wrote:
> From: Ira Weiny 
> 
> UUID's are defined as network byte order fields.  No static initializer
> was available for UUID's in their standard big endian format.
> 
> Define a big endian initializer for UUIDs.
> 
> Reviewed-by: Gregory Price 
> Tested-by: Gregory Price 
> Signed-off-by: Ira Weiny 
> Signed-off-by: Jonathan Cameron 

Reviewed-by: Fan Ni 

> ---
>  include/qemu/uuid.h | 12 
>  1 file changed, 12 insertions(+)
> 
> diff --git a/include/qemu/uuid.h b/include/qemu/uuid.h
> index 9925febfa5..dc40ee1fc9 100644
> --- a/include/qemu/uuid.h
> +++ b/include/qemu/uuid.h
> @@ -61,6 +61,18 @@ typedef struct {
>  (clock_seq_hi_and_reserved), (clock_seq_low), (node0), (node1), (node2),\
>  (node3), (node4), (node5) }
>  
> +/* Normal (network byte order) UUID */
> +#define UUID(time_low, time_mid, time_hi_and_version,\
> +  clock_seq_hi_and_reserved, clock_seq_low, node0, node1, node2, \
> +  node3, node4, node5)   \
> +  { ((time_low) >> 24) & 0xff, ((time_low) >> 16) & 0xff,\
> +((time_low) >> 8) & 0xff, (time_low) & 0xff, \
> +((time_mid) >> 8) & 0xff, (time_mid) & 0xff, \
> +((time_hi_and_version) >> 8) & 0xff, (time_hi_and_version) & 0xff,   \
> +(clock_seq_hi_and_reserved), (clock_seq_low),\
> +(node0), (node1), (node2), (node3), (node4), (node5) \
> +  }
> +
>  #define UUID_FMT "%02hhx%02hhx%02hhx%02hhx-" \
>   "%02hhx%02hhx-%02hhx%02hhx-" \
>   "%02hhx%02hhx-" \
> -- 
> 2.37.2
> 
> 

Re: [PATCH v4 08/10] qemu/bswap: Add const_le64()

[Fan Ni]

On Mon, Feb 06, 2023 at 05:28:14PM +, Jonathan Cameron wrote:
> From: Ira Weiny 
> 
> Gcc requires constant versions of cpu_to_le* calls.
> 
> Add a 64 bit version.
> 
> Reviewed-by: Peter Maydell 
> Reviewed-by: Philippe Mathieu-Daudé 
> Reviewed-by: Gregory Price 
> Tested-by: Gregory Price 
> Signed-off-by: Ira Weiny 
> Signed-off-by: Jonathan Cameron 
> 

Reviewed-by: Fan Ni 

> ---
> v2: Update comment (Philippe)
> ---
>  include/qemu/bswap.h | 12 +++-
>  1 file changed, 11 insertions(+), 1 deletion(-)
> 
> diff --git a/include/qemu/bswap.h b/include/qemu/bswap.h
> index 3cbe52246b..eb3bcf2520 100644
> --- a/include/qemu/bswap.h
> +++ b/include/qemu/bswap.h
> @@ -129,11 +129,20 @@ CPU_CONVERT(le, 32, uint32_t)
>  CPU_CONVERT(le, 64, uint64_t)
>  
>  /*
> - * Same as cpu_to_le{16,32}, except that gcc will figure the result is
> + * Same as cpu_to_le{16,32,64}, except that gcc will figure the result is
>   * a compile-time constant if you pass in a constant.  So this can be
>   * used to initialize static variables.
>   */
>  #if HOST_BIG_ENDIAN
> +# define const_le64(_x)  \
> +_x) & 0x00ffU) << 56) |  \
> + (((_x) & 0xff00U) << 40) |  \
> + (((_x) & 0x00ffU) << 24) |  \
> + (((_x) & 0xff00U) <<  8) |  \
> + (((_x) & 0x00ffU) >>  8) |  \
> + (((_x) & 0xff00U) >> 24) |  \
> + (((_x) & 0x00ffU) >> 40) |  \
> + (((_x) & 0xff00U) >> 56))
>  # define const_le32(_x)  \
>  _x) & 0x00ffU) << 24) |  \
>   (((_x) & 0xff00U) <<  8) |  \
> @@ -143,6 +152,7 @@ CPU_CONVERT(le, 64, uint64_t)
>  _x) & 0x00ff) << 8) |\
>   (((_x) & 0xff00) >> 8))
>  #else
> +# define const_le64(_x) (_x)
>  # define const_le32(_x) (_x)
>  # define const_le16(_x) (_x)
>  #endif
> -- 
> 2.37.2
> 
> 

Re: [PATCH v4 07/10] tests: acpi: Update q35/DSDT.cxl for removed duplicate UID

[Fan Ni]

On Mon, Feb 06, 2023 at 05:28:13PM +, Jonathan Cameron wrote:
> Dropping the ID effects this table in trivial fashion.
> 
> Reviewed-by: Gregory Price 
> Tested-by: Gregory Price 
> Signed-off-by: Jonathan Cameron 
> ---

Reviewed-by: Fan Ni 

>  tests/data/acpi/q35/DSDT.cxl| Bin 9578 -> 9564 bytes
>  tests/qtest/bios-tables-test-allowed-diff.h |   1 -
>  2 files changed, 1 deletion(-)
> 
> diff --git a/tests/data/acpi/q35/DSDT.cxl b/tests/data/acpi/q35/DSDT.cxl
> index 
> 3d18b9672d124a0cf11a79e92c396a1b883d0589..4586b9a18b24acd946cd32c7e3e3a70891a246d2
>  100644
> GIT binary patch
> delta 65
> zcmaFmb;pa#CD IB*D!F0I~xVRsaA1
> 
> delta 79
> zcmccP^~#IOCDSUKwt0m6-Tor}*e5CzZ*UWUScYLp@!%?rjc`U SyId%Wytq76o(Cw;!v+8Y7Z@l2
> 
> diff --git a/tests/qtest/bios-tables-test-allowed-diff.h 
> b/tests/qtest/bios-tables-test-allowed-diff.h
> index 9ce0f596cc..dfb8523c8b 100644
> --- a/tests/qtest/bios-tables-test-allowed-diff.h
> +++ b/tests/qtest/bios-tables-test-allowed-diff.h
> @@ -1,2 +1 @@
>  /* List of comma-separated changed AML files to ignore */
> -"tests/data/acpi/q35/DSDT.cxl",
> -- 
> 2.37.2
> 
> 

Re: [PATCH v4 06/10] hw/i386/acpi: Drop duplicate _UID entry for CXL root bridge

[Fan Ni]

On Mon, Feb 06, 2023 at 05:28:12PM +, Jonathan Cameron wrote:
> Noticed as this prevents iASL disasembling the DSDT table.
> 
> Reviewed-by: Ira Weiny 
> Reviewed-by: Gregory Price 
> Tested-by: Gregory Price 
> Signed-off-by: Jonathan Cameron 
> ---

Reviewed-by: Fan Ni 

>  hw/i386/acpi-build.c | 1 -
>  1 file changed, 1 deletion(-)
> 
> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
> index 145389aa58..4840d11799 100644
> --- a/hw/i386/acpi-build.c
> +++ b/hw/i386/acpi-build.c
> @@ -1514,7 +1514,6 @@ build_dsdt(GArray *table_data, BIOSLinker *linker,
>  aml_append(pkg, aml_eisaid("PNP0A03"));
>  aml_append(dev, aml_name_decl("_CID", pkg));
>  aml_append(dev, aml_name_decl("_ADR", aml_int(0)));
> -aml_append(dev, aml_name_decl("_UID", aml_int(bus_num)));
>  build_cxl_osc_method(dev);
>  } else if (pci_bus_is_express(bus)) {
>  aml_append(dev, aml_name_decl("_HID", 
> aml_eisaid("PNP0A08")));
> -- 
> 2.37.2
> 
> 

Re: [PATCH v4 05/10] tests/acpi: Allow update of q35/DSDT.cxl

[Fan Ni]

On Mon, Feb 06, 2023 at 05:28:11PM +, Jonathan Cameron wrote:
> Next patch will drop duplicate _UID entry so allow update.
> 
> Reviewed-by: Gregory Price 
> Tested-by: Gregory Price 
> Signed-off-by: Jonathan Cameron 

Reviewed-by: Fan Ni 

> ---
>  tests/qtest/bios-tables-test-allowed-diff.h | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/tests/qtest/bios-tables-test-allowed-diff.h 
> b/tests/qtest/bios-tables-test-allowed-diff.h
> index dfb8523c8b..9ce0f596cc 100644
> --- a/tests/qtest/bios-tables-test-allowed-diff.h
> +++ b/tests/qtest/bios-tables-test-allowed-diff.h
> @@ -1 +1,2 @@
>  /* List of comma-separated changed AML files to ignore */
> +"tests/data/acpi/q35/DSDT.cxl",
> -- 
> 2.37.2
> 
>