Re: [PATCH v7 00/12] Enabling DCD emulation support in Qemu

[Gregory Price]

On Thu, May 16, 2024 at 10:05:33AM -0700, fan wrote:
> On Fri, Apr 19, 2024 at 02:24:36PM -0400, Gregory Price wrote:
> > On Thu, Apr 18, 2024 at 04:10:51PM -0700, nifan@gmail.com wrote:
> > > A git tree of this series can be found here (with one extra commit on top
> > > for printing out accepted/pending extent list): 
> > > https://github.com/moking/qemu/tree/dcd-v7
> > > 
> > > v6->v7:
> > > 
> > > 1. Fixed the dvsec range register issue mentioned in the the cover letter 
> > > in v6.
> > >Only relevant bits are set to mark the device ready (Patch 6). 
> > > (Jonathan)
> > > 2. Moved the if statement in cxl_setup_memory from Patch 6 to Patch 4. 
> > > (Jonathan)
> > > 3. Used MIN instead of if statement to get record_count in Patch 7. 
> > > (Jonathan)
> > > 4. Added "Reviewed-by" tag to Patch 7.
> > > 5. Modified cxl_dc_extent_release_dry_run so the updated extent list can 
> > > be
> > >reused in cmd_dcd_release_dyn_cap to simplify the process in Patch 8. 
> > > (Jørgen) 
> > > 6. Added comments to indicate further "TODO" items in 
> > > cmd_dcd_add_dyn_cap_rsp.
> > > (Jonathan)
> > > 7. Avoided irrelevant code reformat in Patch 8. (Jonathan)
> > > 8. Modified QMP interfaces for adding/releasing DC extents to allow 
> > > passing
> > >tags, selection policy, flags in the interface. (Jonathan, Gregory)
> > > 9. Redesigned the pending list so extents in the same requests are grouped
> > > together. A new data structure is introduced to represent "extent 
> > > group"
> > > in pending list.  (Jonathan)
> > > 10. Added support in QMP interface for "More" flag. 
> > > 11. Check "Forced removal" flag for release request and not let it pass 
> > > through.
> > > 12. Removed the dynamic capacity log type from CxlEventLog definition in 
> > > cxl.json
> > >to avoid the side effect it may introduce to inject error to DC event 
> > > log.
> > >(Jonathan)
> > > 13. Hard coded the event log type to dynamic capacity event log in QMP
> > > interfaces. (Jonathan)
> > > 14. Adding space in between "-1]". (Jonathan)
> > > 15. Some minor comment fixes.
> > > 
> > > The code is tested with similar setup and has passed similar tests as 
> > > listed
> > > in the cover letter of v5[1] and v6[2].
> > > Also, the code is tested with the latest DCD kernel patchset[3].
> > > 
> > > [1] Qemu DCD patchset v5: 
> > > https://lore.kernel.org/linux-cxl/20240304194331.1586191-1-nifan@gmail.com/T/#t
> > > [2] Qemu DCD patchset v6: 
> > > https://lore.kernel.org/linux-cxl/20240325190339.696686-1-nifan@gmail.com/T/#t
> > > [3] DCD kernel patches: 
> > > https://lore.kernel.org/linux-cxl/20240324-dcd-type2-upstream-v1-0-b7b00d623...@intel.com/T/#m11c571e21c4fe17c7d04ec5c2c7bc7cbf2cd07e3
> > >
> > 
> > added review to all patches, will hopefully be able to add a Tested-by
> > tag early next week, along with a v1 RFC for MHD bit-tracking.
> > 
> > We've been testing v5/v6 for a bit, so I expect as soon as we get the
> > MHD code ported over to v7 i'll ship a tested-by tag pretty quick.
> > 
> > The super-set release will complicate a few things but this doesn't
> > look like a blocker on our end, just a change to how we track bits in a
> > shared bit/bytemap.
> > 
> 
> Hi Gregory,
> I am planning to address all the concerns in this series and send out v8
> next week. Jonathan mentioned you have few related patches built on top
> of this series, can you point me to the latest version so I can look
> into it? Also, would you like me to carry them over to send together
> with my series in next version? It could be easier for you to avoid the
> potential rebase needed for your patches?
> 
> Let me know.
> 
> Thanks,
> Fan
>

I apologize for missing this, I was out of the country for a few weeks.
I'm still catching up on the work history.

I think i saw in passing that you picked up the CCI changes, and those
were the ones causing conflicts - so that's perfect.  I can always
rebase on that.

~ Gregory

Re: [PATCH v7 00/12] Enabling DCD emulation support in Qemu

[fan]

On Fri, May 17, 2024 at 01:18:52PM +0100, Jonathan Cameron wrote:
> On Thu, 16 May 2024 10:05:33 -0700
> fan  wrote:
> 
> > On Fri, Apr 19, 2024 at 02:24:36PM -0400, Gregory Price wrote:
> > > On Thu, Apr 18, 2024 at 04:10:51PM -0700, nifan@gmail.com wrote:  
> > > > A git tree of this series can be found here (with one extra commit on 
> > > > top
> > > > for printing out accepted/pending extent list): 
> > > > https://github.com/moking/qemu/tree/dcd-v7
> > > > 
> > > > v6->v7:
> > > > 
> > > > 1. Fixed the dvsec range register issue mentioned in the the cover 
> > > > letter in v6.
> > > >Only relevant bits are set to mark the device ready (Patch 6). 
> > > > (Jonathan)
> > > > 2. Moved the if statement in cxl_setup_memory from Patch 6 to Patch 4. 
> > > > (Jonathan)
> > > > 3. Used MIN instead of if statement to get record_count in Patch 7. 
> > > > (Jonathan)
> > > > 4. Added "Reviewed-by" tag to Patch 7.
> > > > 5. Modified cxl_dc_extent_release_dry_run so the updated extent list 
> > > > can be
> > > >reused in cmd_dcd_release_dyn_cap to simplify the process in Patch 
> > > > 8. (Jørgen) 
> > > > 6. Added comments to indicate further "TODO" items in 
> > > > cmd_dcd_add_dyn_cap_rsp.
> > > > (Jonathan)
> > > > 7. Avoided irrelevant code reformat in Patch 8. (Jonathan)
> > > > 8. Modified QMP interfaces for adding/releasing DC extents to allow 
> > > > passing
> > > >tags, selection policy, flags in the interface. (Jonathan, Gregory)
> > > > 9. Redesigned the pending list so extents in the same requests are 
> > > > grouped
> > > > together. A new data structure is introduced to represent "extent 
> > > > group"
> > > > in pending list.  (Jonathan)
> > > > 10. Added support in QMP interface for "More" flag. 
> > > > 11. Check "Forced removal" flag for release request and not let it pass 
> > > > through.
> > > > 12. Removed the dynamic capacity log type from CxlEventLog definition 
> > > > in cxl.json
> > > >to avoid the side effect it may introduce to inject error to DC 
> > > > event log.
> > > >(Jonathan)
> > > > 13. Hard coded the event log type to dynamic capacity event log in QMP
> > > > interfaces. (Jonathan)
> > > > 14. Adding space in between "-1]". (Jonathan)
> > > > 15. Some minor comment fixes.
> > > > 
> > > > The code is tested with similar setup and has passed similar tests as 
> > > > listed
> > > > in the cover letter of v5[1] and v6[2].
> > > > Also, the code is tested with the latest DCD kernel patchset[3].
> > > > 
> > > > [1] Qemu DCD patchset v5: 
> > > > https://lore.kernel.org/linux-cxl/20240304194331.1586191-1-nifan@gmail.com/T/#t
> > > > [2] Qemu DCD patchset v6: 
> > > > https://lore.kernel.org/linux-cxl/20240325190339.696686-1-nifan@gmail.com/T/#t
> > > > [3] DCD kernel patches: 
> > > > https://lore.kernel.org/linux-cxl/20240324-dcd-type2-upstream-v1-0-b7b00d623...@intel.com/T/#m11c571e21c4fe17c7d04ec5c2c7bc7cbf2cd07e3
> > > >  
> > > 
> > > added review to all patches, will hopefully be able to add a Tested-by
> > > tag early next week, along with a v1 RFC for MHD bit-tracking.
> > > 
> > > We've been testing v5/v6 for a bit, so I expect as soon as we get the
> > > MHD code ported over to v7 i'll ship a tested-by tag pretty quick.
> > > 
> > > The super-set release will complicate a few things but this doesn't
> > > look like a blocker on our end, just a change to how we track bits in a
> > > shared bit/bytemap.
> > >   
> > 
> > Hi Gregory,
> > I am planning to address all the concerns in this series and send out v8
> > next week. Jonathan mentioned you have few related patches built on top
> > of this series, can you point me to the latest version so I can look
> > into it? Also, would you like me to carry them over to send together
> > with my series in next version? It could be easier for you to avoid the
> > potential rebase needed for your patches?
> 
> I wasn't clear - I meant other way around.
> This series is built on a couple of Gregory's patches.  Gregory can suffer
> the pain of rebasing his stuff ;) (or I'll do it depending on when things
> land).
> 
> hw/cxl/mailbox: change CCI cmd set structure to be a member, not a reference 
> https://gitlab.com/jic23/qemu/-/commit/f44ebc5a455ccdd6535879b0c5824e0d76b04da5
> hw/cxl/mailbox: interface to add CCI commands to an existing CCI 
> https://gitlab.com/jic23/qemu/-/commit/00a4dd8b388add03c588298f665ee918626296a5
> 
> I was suggesting your next posting should just include those two with
> your sign-off added. That way if everyone is happy with v8 Michael Tsirkin
> can pick it up directly, saving a step.
> 
> Make sure to add Michael to the to list as well for next version.
> 
> Thanks,
> 
> Jonathan

Oh, I totally mis-understood.

Sure. I will add the two patches with my sign-off to my next post.

Fan
> 
> > 
> > Let me know.
> > 
> > Thanks,
> > Fan
> > 
> > > > 
> > > > Fan Ni (12):
> > > >   hw/cxl/cxl-mailbox-utils: Add dc_event_log_size field to output
> > > > 

Re: [PATCH v7 00/12] Enabling DCD emulation support in Qemu

[Jonathan Cameron via]

On Thu, 16 May 2024 10:05:33 -0700
fan  wrote:

> On Fri, Apr 19, 2024 at 02:24:36PM -0400, Gregory Price wrote:
> > On Thu, Apr 18, 2024 at 04:10:51PM -0700, nifan@gmail.com wrote:  
> > > A git tree of this series can be found here (with one extra commit on top
> > > for printing out accepted/pending extent list): 
> > > https://github.com/moking/qemu/tree/dcd-v7
> > > 
> > > v6->v7:
> > > 
> > > 1. Fixed the dvsec range register issue mentioned in the the cover letter 
> > > in v6.
> > >Only relevant bits are set to mark the device ready (Patch 6). 
> > > (Jonathan)
> > > 2. Moved the if statement in cxl_setup_memory from Patch 6 to Patch 4. 
> > > (Jonathan)
> > > 3. Used MIN instead of if statement to get record_count in Patch 7. 
> > > (Jonathan)
> > > 4. Added "Reviewed-by" tag to Patch 7.
> > > 5. Modified cxl_dc_extent_release_dry_run so the updated extent list can 
> > > be
> > >reused in cmd_dcd_release_dyn_cap to simplify the process in Patch 8. 
> > > (Jørgen) 
> > > 6. Added comments to indicate further "TODO" items in 
> > > cmd_dcd_add_dyn_cap_rsp.
> > > (Jonathan)
> > > 7. Avoided irrelevant code reformat in Patch 8. (Jonathan)
> > > 8. Modified QMP interfaces for adding/releasing DC extents to allow 
> > > passing
> > >tags, selection policy, flags in the interface. (Jonathan, Gregory)
> > > 9. Redesigned the pending list so extents in the same requests are grouped
> > > together. A new data structure is introduced to represent "extent 
> > > group"
> > > in pending list.  (Jonathan)
> > > 10. Added support in QMP interface for "More" flag. 
> > > 11. Check "Forced removal" flag for release request and not let it pass 
> > > through.
> > > 12. Removed the dynamic capacity log type from CxlEventLog definition in 
> > > cxl.json
> > >to avoid the side effect it may introduce to inject error to DC event 
> > > log.
> > >(Jonathan)
> > > 13. Hard coded the event log type to dynamic capacity event log in QMP
> > > interfaces. (Jonathan)
> > > 14. Adding space in between "-1]". (Jonathan)
> > > 15. Some minor comment fixes.
> > > 
> > > The code is tested with similar setup and has passed similar tests as 
> > > listed
> > > in the cover letter of v5[1] and v6[2].
> > > Also, the code is tested with the latest DCD kernel patchset[3].
> > > 
> > > [1] Qemu DCD patchset v5: 
> > > https://lore.kernel.org/linux-cxl/20240304194331.1586191-1-nifan@gmail.com/T/#t
> > > [2] Qemu DCD patchset v6: 
> > > https://lore.kernel.org/linux-cxl/20240325190339.696686-1-nifan@gmail.com/T/#t
> > > [3] DCD kernel patches: 
> > > https://lore.kernel.org/linux-cxl/20240324-dcd-type2-upstream-v1-0-b7b00d623...@intel.com/T/#m11c571e21c4fe17c7d04ec5c2c7bc7cbf2cd07e3
> > >  
> > 
> > added review to all patches, will hopefully be able to add a Tested-by
> > tag early next week, along with a v1 RFC for MHD bit-tracking.
> > 
> > We've been testing v5/v6 for a bit, so I expect as soon as we get the
> > MHD code ported over to v7 i'll ship a tested-by tag pretty quick.
> > 
> > The super-set release will complicate a few things but this doesn't
> > look like a blocker on our end, just a change to how we track bits in a
> > shared bit/bytemap.
> >   
> 
> Hi Gregory,
> I am planning to address all the concerns in this series and send out v8
> next week. Jonathan mentioned you have few related patches built on top
> of this series, can you point me to the latest version so I can look
> into it? Also, would you like me to carry them over to send together
> with my series in next version? It could be easier for you to avoid the
> potential rebase needed for your patches?

I wasn't clear - I meant other way around.
This series is built on a couple of Gregory's patches.  Gregory can suffer
the pain of rebasing his stuff ;) (or I'll do it depending on when things
land).

hw/cxl/mailbox: change CCI cmd set structure to be a member, not a reference 
https://gitlab.com/jic23/qemu/-/commit/f44ebc5a455ccdd6535879b0c5824e0d76b04da5
hw/cxl/mailbox: interface to add CCI commands to an existing CCI 
https://gitlab.com/jic23/qemu/-/commit/00a4dd8b388add03c588298f665ee918626296a5

I was suggesting your next posting should just include those two with
your sign-off added. That way if everyone is happy with v8 Michael Tsirkin
can pick it up directly, saving a step.

Make sure to add Michael to the to list as well for next version.

Thanks,

Jonathan

> 
> Let me know.
> 
> Thanks,
> Fan
> 
> > > 
> > > Fan Ni (12):
> > >   hw/cxl/cxl-mailbox-utils: Add dc_event_log_size field to output
> > > payload of identify memory device command
> > >   hw/cxl/cxl-mailbox-utils: Add dynamic capacity region representative
> > > and mailbox command support
> > >   include/hw/cxl/cxl_device: Rename mem_size as static_mem_size for
> > > type3 memory devices
> > >   hw/mem/cxl_type3: Add support to create DC regions to type3 memory
> > > devices
> > >   hw/mem/cxl-type3: Refactor 

Re: [PATCH v7 00/12] Enabling DCD emulation support in Qemu

[Zhijian Li (Fujitsu)]

Fan,

Many thanks, it helps a lot. Previous I forgot to create a new dax 
device(daxctl create-device region0)
Question: Why we need to create a the dax0.1, why the dax0.0 doesn't associate 
to the new adding DCD region.

Ira,

Let me try to report a kernel panic.

kernel: dcd-2024-04-17
qemu: dcd-2024-04-17

QEMU command line:
164 
165 
166 
167 
168 
169 
170 
171 
172 
173 


Reproducer:
  1. guest: ./create-dc.sh
  2. host: virsh qemu-monitor-command rdma-server-cxl-persistent-dcd $(cat 
cxl-add-dcd.json)
  3. guest: daxctl create-device region0 # will create dax0.1
  4. daxctl reconfigure-device  --mode=system-ram --force  dax0.1 -u  # kernel 
panic

=
# cat ./create-dc.sh
#!/bin/bash
set -ex

region=$(cat /sys/bus/cxl/devices/decoder0.0/create_dc_region)
echo $region> /sys/bus/cxl/devices/decoder0.0/create_dc_region
echo 256 > /sys/bus/cxl/devices/$region/interleave_granularity
echo 1 > /sys/bus/cxl/devices/$region/interleave_ways
echo "dc0" >/sys/bus/cxl/devices/decoder2.0/mode
echo 0x1000 >/sys/bus/cxl/devices/decoder2.0/dpa_size
echo 0x1000 > /sys/bus/cxl/devices/$region/size
echo "decoder2.0" > /sys/bus/cxl/devices/$region/target0
echo 1 > /sys/bus/cxl/devices/$region/commit
echo $region > /sys/bus/cxl/drivers/cxl_region/bind
=
# cat cxl-add-dcd.json
{ "execute": "cxl-add-dynamic-capacity",
   "arguments": {
   "path": "/machine/peripheral/pmem-dcmem",
   "hid": 0,
   "selection-policy": 2,
   "region-id": 0,
   "tag": "",
   "extents": [
   {
   "offset": 0,
   "len": 268435456
   }
   ]
   }
}



[  126.909297] Demotion targets for Node 0: preferred: 1, fallback: 1
[  126.911186] Demotion targets for Node 1: null
[  126.913808] BUG: kernel NULL pointer dereference, address: 0468
[  126.915431] #PF: supervisor read access in kernel mode
[  126.917156] #PF: error_code(0x) - not-present page
[  126.918976] PGD 86771067 P4D 86771067 PUD e777067 PMD 0
[  126.920587] Oops:  [#1] PREEMPT SMP PTI
[  126.921714] CPU: 0 PID: 1101 Comm: daxctl Kdump: loaded Not tainted 
6.9.0-rc3-lizhijian+ #489
[  126.924914] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS 
rel-1.16.3-0-ga6ed6b701f0a-prebuilt.qemu.org 04/01/2014
[  126.928620] RIP: 0010:cxl_region_perf_attrs_callback+0x25/0x110 [cxl_core]
[  126.930316] Code: 90 90 90 90 90 0f 1f 44 00 00 41 56 41 55 41 54 55 53 8b 
6a 24 83 fd ff 74 20 48 83 fe 01 75 1a 48 8b 87 58 ff ff ff 48 89 fb <48> 8b b8 
68 04 00 00 e8 cf a2 f4 e0 39 c5 74 13 45 31 e4 5b 44 89
[  126.934920] RSP: 0018:c97cbc58 EFLAGS: 00010246
[  126.936994] RAX:  RBX: 888007534d60 RCX: 0020
[  126.939378] RDX: c97cbcf8 RSI: 0001 RDI: 888007534d60
[  126.942721] RBP: 0001 R08: 0001 R09: 0001
[  126.944762] R10: 88807fc31d80 R11:  R12: 
[  126.946900] R13: 0001 R14: c97cbcf8 R15: 888007534d60
[  126.948871] FS:  7fb2ab918880() GS:88807fc0() 
knlGS:
[  126.951241] CS:  0010 DS:  ES:  CR0: 80050033
[  126.952722] CR2: 0468 CR3: 0aaf0003 CR4: 001706f0
[  126.954623] DR0:  DR1:  DR2: 
[  126.956768] DR3:  DR6: fffe0ff0 DR7: 0400
[  126.958887] Call Trace:
[  126.959814]  
[  126.960569]  ? __die+0x20/0x70
[  126.961645]  ? page_fault_oops+0x15a/0x450
[  126.962930]  ? search_module_extables+0x33/0x90
[  126.964374]  ? fixup_exception+0x22/0x310
[  126.965693]  ? exc_page_fault+0x68/0x200
[  126.967371]  ? asm_exc_page_fault+0x22/0x30
[  126.968713]  ? cxl_region_perf_attrs_callback+0x25/0x110 [cxl_core]
[  126.972508]  notifier_call_chain+0x40/0x110
[  126.974380]  blocking_notifier_call_chain+0x43/0x60
[  126.975788]  online_pages+0x24c/0x2d0
[  126.977008]  memory_subsys_online+0x233/0x290
[  126.978338]  device_online+0x64/0x90
[  126.979440]  state_store+0xae/0xc0
[  126.980510]  kernfs_fop_write_iter+0x143/0x200
[  126.981734]  vfs_write+0x3a6/0x570
[  126.982851]  ksys_write+0x65/0xf0
[  126.984006]  do_syscall_64+0x6d/0x140
[  126.985309]  entry_SYSCALL_64_after_hwframe+0x71/0x79
[  126.986927] RIP: 0033:0x7fb2abc777a7
[  126.987983] Code: 0d 00 f7 d8 64 89 02 48 c7 c0 ff ff ff ff eb b7 0f 1f 00 
f3 0f 1e fa 64 8b 04 25 18 00 00 00 85 c0 75 10 b8 01 00 00 00 0f 05 <48> 3d 00 
f0 ff ff 77 51 c3 48 83 ec 28 48 89 54 24 18 48 89 74 24
[  126.992770] RSP: 002b:7ffebec70b98 EFLAGS: 0246 ORIG_RAX: 
0001
[  126.994874] RAX: ffda RBX: 0040e1f0 RCX: 7fb2abc777a7
[  126.996906] RDX: 000f RSI: 7fb2abdb6434 RDI: 0004
[  126.998911] RBP: 7ffebec70bd0 R08:  R09: 7ffebec70640
[  127.000879] R10: 

Re: [PATCH v7 00/12] Enabling DCD emulation support in Qemu

[fan]

On Tue, May 14, 2024 at 02:16:51AM +, Zhijian Li (Fujitsu) wrote:
> Hi Fan
> 
> 
> Do you have a newer instruction to play with the DCD. It seems that
> the instruction in RFC[0] doesn't work for current code.
> 
> [0] https://lore.kernel.org/all/20230511175609.2091136-1-fan...@samsung.com/
> 

For the testing, the only thing that has been changed for this series is
the QMP interface for add/release DC extents.

https://lore.kernel.org/linux-cxl/d708f7c8-2598-4a17-9cbb-935c6ae2a...@fujitsu.com/T/#m05066f0098e976fb1c4b05db5e7ff7ca1bf27b1e

1. Add dynamic capacity extents:

For example, the command to add two continuous extents (each 128MiB long)
to region 0 (starting at DPA offset 0) looks like below:

{ "execute": "qmp_capabilities" }

{ "execute": "cxl-add-dynamic-capacity",
  "arguments": {
  "path": "/machine/peripheral/cxl-dcd0",
  "hid": 0,
  "selection-policy": 2,
  "region-id": 0,
  "tag": "",
  "extents": [
  {
  "offset": 0,
  "len": 134217728
  },
  {
  "offset": 134217728,
  "len": 134217728
  }
  ]
  }
}

2. Release dynamic capacity extents:

For example, the command to release an extent of size 128MiB from region 0
(DPA offset 128MiB) looks like below:

{ "execute": "cxl-release-dynamic-capacity",
  "arguments": {
  "path": "/machine/peripheral/cxl-dcd0",
  "hid": 0,
  "flags": 1,
  "region-id": 0,
  "tag": "",
  "extents": [
  {
  "offset": 134217728,
  "len": 134217728
  }
  ]
  }
}

btw, I have a wiki page to explain how to test CXL DCD with a tool I
wrote.
https://github.com/moking/moking.github.io/wiki/cxl%E2%80%90test%E2%80%90tool:-A-tool-to-ease-CXL-test-with-QEMU-setup%E2%80%90%E2%80%90Using-DCD-test-as-an-example

Let me know if you need more info for testing.


Fan

> 
> 
> On 19/04/2024 07:10, nifan@gmail.com wrote:
> > A git tree of this series can be found here (with one extra commit on top
> > for printing out accepted/pending extent list):
> > https://github.com/moking/qemu/tree/dcd-v7
> > 
> > v6->v7:
> > 
> > 1. Fixed the dvsec range register issue mentioned in the the cover letter 
> > in v6.
> > Only relevant bits are set to mark the device ready (Patch 6). 
> > (Jonathan)
> > 2. Moved the if statement in cxl_setup_memory from Patch 6 to Patch 4. 
> > (Jonathan)
> > 3. Used MIN instead of if statement to get record_count in Patch 7. 
> > (Jonathan)
> > 4. Added "Reviewed-by" tag to Patch 7.
> > 5. Modified cxl_dc_extent_release_dry_run so the updated extent list can be
> > reused in cmd_dcd_release_dyn_cap to simplify the process in Patch 8. 
> > (Jørgen)
> > 6. Added comments to indicate further "TODO" items in 
> > cmd_dcd_add_dyn_cap_rsp.
> >  (Jonathan)
> > 7. Avoided irrelevant code reformat in Patch 8. (Jonathan)
> > 8. Modified QMP interfaces for adding/releasing DC extents to allow passing
> > tags, selection policy, flags in the interface. (Jonathan, Gregory)
> > 9. Redesigned the pending list so extents in the same requests are grouped
> >  together. A new data structure is introduced to represent "extent 
> > group"
> >  in pending list.  (Jonathan)
> > 10. Added support in QMP interface for "More" flag.
> > 11. Check "Forced removal" flag for release request and not let it pass 
> > through.
> > 12. Removed the dynamic capacity log type from CxlEventLog definition in 
> > cxl.json
> > to avoid the side effect it may introduce to inject error to DC event 
> > log.
> > (Jonathan)
> > 13. Hard coded the event log type to dynamic capacity event log in QMP
> >  interfaces. (Jonathan)
> > 14. Adding space in between "-1]". (Jonathan)
> > 15. Some minor comment fixes.
> > 
> > The code is tested with similar setup and has passed similar tests as listed
> > in the cover letter of v5[1] and v6[2].
> > Also, the code is tested with the latest DCD kernel patchset[3].
> > 
> > [1] Qemu DCD patchset v5: 
> > https://lore.kernel.org/linux-cxl/20240304194331.1586191-1-nifan@gmail.com/T/#t
> > [2] Qemu DCD patchset v6: 
> > https://lore.kernel.org/linux-cxl/20240325190339.696686-1-nifan@gmail.com/T/#t
> > [3] DCD kernel patches: 
> > https://lore.kernel.org/linux-cxl/20240324-dcd-type2-upstream-v1-0-b7b00d623...@intel.com/T/#m11c571e21c4fe17c7d04ec5c2c7bc7cbf2cd07e3
> > 
> > 
> > Fan Ni (12):
> >hw/cxl/cxl-mailbox-utils: Add dc_event_log_size field to output
> >  payload of identify memory device command
> >hw/cxl/cxl-mailbox-utils: Add dynamic capacity region representative
> >  and mailbox command support
> >include/hw/cxl/cxl_device: Rename mem_size as static_mem_size for
> >  type3 memory devices
> >hw/mem/cxl_type3: Add support to create DC regions to type3 memory
> >  devices
> >hw/mem/cxl-type3: Refactor ct3_build_cdat_entries_for_mr to take mr
> >  size instead of mr as argument
> >hw/mem/cxl_type3: Add host backend and address space handling for DC

Re: [PATCH v7 00/12] Enabling DCD emulation support in Qemu

[fan]

On Fri, Apr 19, 2024 at 02:24:36PM -0400, Gregory Price wrote:
> On Thu, Apr 18, 2024 at 04:10:51PM -0700, nifan@gmail.com wrote:
> > A git tree of this series can be found here (with one extra commit on top
> > for printing out accepted/pending extent list): 
> > https://github.com/moking/qemu/tree/dcd-v7
> > 
> > v6->v7:
> > 
> > 1. Fixed the dvsec range register issue mentioned in the the cover letter 
> > in v6.
> >Only relevant bits are set to mark the device ready (Patch 6). (Jonathan)
> > 2. Moved the if statement in cxl_setup_memory from Patch 6 to Patch 4. 
> > (Jonathan)
> > 3. Used MIN instead of if statement to get record_count in Patch 7. 
> > (Jonathan)
> > 4. Added "Reviewed-by" tag to Patch 7.
> > 5. Modified cxl_dc_extent_release_dry_run so the updated extent list can be
> >reused in cmd_dcd_release_dyn_cap to simplify the process in Patch 8. 
> > (Jørgen) 
> > 6. Added comments to indicate further "TODO" items in 
> > cmd_dcd_add_dyn_cap_rsp.
> > (Jonathan)
> > 7. Avoided irrelevant code reformat in Patch 8. (Jonathan)
> > 8. Modified QMP interfaces for adding/releasing DC extents to allow passing
> >tags, selection policy, flags in the interface. (Jonathan, Gregory)
> > 9. Redesigned the pending list so extents in the same requests are grouped
> > together. A new data structure is introduced to represent "extent group"
> > in pending list.  (Jonathan)
> > 10. Added support in QMP interface for "More" flag. 
> > 11. Check "Forced removal" flag for release request and not let it pass 
> > through.
> > 12. Removed the dynamic capacity log type from CxlEventLog definition in 
> > cxl.json
> >to avoid the side effect it may introduce to inject error to DC event 
> > log.
> >(Jonathan)
> > 13. Hard coded the event log type to dynamic capacity event log in QMP
> > interfaces. (Jonathan)
> > 14. Adding space in between "-1]". (Jonathan)
> > 15. Some minor comment fixes.
> > 
> > The code is tested with similar setup and has passed similar tests as listed
> > in the cover letter of v5[1] and v6[2].
> > Also, the code is tested with the latest DCD kernel patchset[3].
> > 
> > [1] Qemu DCD patchset v5: 
> > https://lore.kernel.org/linux-cxl/20240304194331.1586191-1-nifan@gmail.com/T/#t
> > [2] Qemu DCD patchset v6: 
> > https://lore.kernel.org/linux-cxl/20240325190339.696686-1-nifan@gmail.com/T/#t
> > [3] DCD kernel patches: 
> > https://lore.kernel.org/linux-cxl/20240324-dcd-type2-upstream-v1-0-b7b00d623...@intel.com/T/#m11c571e21c4fe17c7d04ec5c2c7bc7cbf2cd07e3
> >
> 
> added review to all patches, will hopefully be able to add a Tested-by
> tag early next week, along with a v1 RFC for MHD bit-tracking.
> 
> We've been testing v5/v6 for a bit, so I expect as soon as we get the
> MHD code ported over to v7 i'll ship a tested-by tag pretty quick.
> 
> The super-set release will complicate a few things but this doesn't
> look like a blocker on our end, just a change to how we track bits in a
> shared bit/bytemap.
> 

Hi Gregory,
I am planning to address all the concerns in this series and send out v8
next week. Jonathan mentioned you have few related patches built on top
of this series, can you point me to the latest version so I can look
into it? Also, would you like me to carry them over to send together
with my series in next version? It could be easier for you to avoid the
potential rebase needed for your patches?

Let me know.

Thanks,
Fan

> > 
> > Fan Ni (12):
> >   hw/cxl/cxl-mailbox-utils: Add dc_event_log_size field to output
> > payload of identify memory device command
> >   hw/cxl/cxl-mailbox-utils: Add dynamic capacity region representative
> > and mailbox command support
> >   include/hw/cxl/cxl_device: Rename mem_size as static_mem_size for
> > type3 memory devices
> >   hw/mem/cxl_type3: Add support to create DC regions to type3 memory
> > devices
> >   hw/mem/cxl-type3: Refactor ct3_build_cdat_entries_for_mr to take mr
> > size instead of mr as argument
> >   hw/mem/cxl_type3: Add host backend and address space handling for DC
> > regions
> >   hw/mem/cxl_type3: Add DC extent list representative and get DC extent
> > list mailbox support
> >   hw/cxl/cxl-mailbox-utils: Add mailbox commands to support add/release
> > dynamic capacity response
> >   hw/cxl/events: Add qmp interfaces to add/release dynamic capacity
> > extents
> >   hw/mem/cxl_type3: Add DPA range validation for accesses to DC regions
> >   hw/cxl/cxl-mailbox-utils: Add superset extent release mailbox support
> >   hw/mem/cxl_type3: Allow to release extent superset in QMP interface
> > 
> >  hw/cxl/cxl-mailbox-utils.c  | 620 ++-
> >  hw/mem/cxl_type3.c  | 633 +---
> >  hw/mem/cxl_type3_stubs.c|  20 ++
> >  include/hw/cxl/cxl_device.h |  81 -
> >  include/hw/cxl/cxl_events.h |  18 +
> >  qapi/cxl.json   |  69 
> >  6 files changed, 1396 

Re: [PATCH v7 00/12] Enabling DCD emulation support in Qemu

[Zhijian Li (Fujitsu)]

Hi Fan


Do you have a newer instruction to play with the DCD. It seems that
the instruction in RFC[0] doesn't work for current code.

[0] https://lore.kernel.org/all/20230511175609.2091136-1-fan...@samsung.com/



On 19/04/2024 07:10, nifan@gmail.com wrote:
> A git tree of this series can be found here (with one extra commit on top
> for printing out accepted/pending extent list):
> https://github.com/moking/qemu/tree/dcd-v7
> 
> v6->v7:
> 
> 1. Fixed the dvsec range register issue mentioned in the the cover letter in 
> v6.
> Only relevant bits are set to mark the device ready (Patch 6). (Jonathan)
> 2. Moved the if statement in cxl_setup_memory from Patch 6 to Patch 4. 
> (Jonathan)
> 3. Used MIN instead of if statement to get record_count in Patch 7. (Jonathan)
> 4. Added "Reviewed-by" tag to Patch 7.
> 5. Modified cxl_dc_extent_release_dry_run so the updated extent list can be
> reused in cmd_dcd_release_dyn_cap to simplify the process in Patch 8. 
> (Jørgen)
> 6. Added comments to indicate further "TODO" items in cmd_dcd_add_dyn_cap_rsp.
>  (Jonathan)
> 7. Avoided irrelevant code reformat in Patch 8. (Jonathan)
> 8. Modified QMP interfaces for adding/releasing DC extents to allow passing
> tags, selection policy, flags in the interface. (Jonathan, Gregory)
> 9. Redesigned the pending list so extents in the same requests are grouped
>  together. A new data structure is introduced to represent "extent group"
>  in pending list.  (Jonathan)
> 10. Added support in QMP interface for "More" flag.
> 11. Check "Forced removal" flag for release request and not let it pass 
> through.
> 12. Removed the dynamic capacity log type from CxlEventLog definition in 
> cxl.json
> to avoid the side effect it may introduce to inject error to DC event log.
> (Jonathan)
> 13. Hard coded the event log type to dynamic capacity event log in QMP
>  interfaces. (Jonathan)
> 14. Adding space in between "-1]". (Jonathan)
> 15. Some minor comment fixes.
> 
> The code is tested with similar setup and has passed similar tests as listed
> in the cover letter of v5[1] and v6[2].
> Also, the code is tested with the latest DCD kernel patchset[3].
> 
> [1] Qemu DCD patchset v5: 
> https://lore.kernel.org/linux-cxl/20240304194331.1586191-1-nifan@gmail.com/T/#t
> [2] Qemu DCD patchset v6: 
> https://lore.kernel.org/linux-cxl/20240325190339.696686-1-nifan@gmail.com/T/#t
> [3] DCD kernel patches: 
> https://lore.kernel.org/linux-cxl/20240324-dcd-type2-upstream-v1-0-b7b00d623...@intel.com/T/#m11c571e21c4fe17c7d04ec5c2c7bc7cbf2cd07e3
> 
> 
> Fan Ni (12):
>hw/cxl/cxl-mailbox-utils: Add dc_event_log_size field to output
>  payload of identify memory device command
>hw/cxl/cxl-mailbox-utils: Add dynamic capacity region representative
>  and mailbox command support
>include/hw/cxl/cxl_device: Rename mem_size as static_mem_size for
>  type3 memory devices
>hw/mem/cxl_type3: Add support to create DC regions to type3 memory
>  devices
>hw/mem/cxl-type3: Refactor ct3_build_cdat_entries_for_mr to take mr
>  size instead of mr as argument
>hw/mem/cxl_type3: Add host backend and address space handling for DC
>  regions
>hw/mem/cxl_type3: Add DC extent list representative and get DC extent
>  list mailbox support
>hw/cxl/cxl-mailbox-utils: Add mailbox commands to support add/release
>  dynamic capacity response
>hw/cxl/events: Add qmp interfaces to add/release dynamic capacity
>  extents
>hw/mem/cxl_type3: Add DPA range validation for accesses to DC regions
>hw/cxl/cxl-mailbox-utils: Add superset extent release mailbox support
>hw/mem/cxl_type3: Allow to release extent superset in QMP interface
> 
>   hw/cxl/cxl-mailbox-utils.c  | 620 ++-
>   hw/mem/cxl_type3.c  | 633 +---
>   hw/mem/cxl_type3_stubs.c|  20 ++
>   include/hw/cxl/cxl_device.h |  81 -
>   include/hw/cxl/cxl_events.h |  18 +
>   qapi/cxl.json   |  69 
>   6 files changed, 1396 insertions(+), 45 deletions(-)
> 

Re: [PATCH v7 00/12] Enabling DCD emulation support in Qemu

[Svetly Todorov]

> On Apr 24, 2024, at 1:16 PM, Svetly Todorov  
> wrote:
> 
> I tested the v7 DC add/release as part of the MHSLD RFC work I did
> with Gregory. Those commands worked as expected. Feel free to tack on:
> 
> Tested-by: Svetly Todorov  >

Er,

Tested-by: Svetly Todorov 

Didn’t realize the Apple Mail client had snuck a link in there.

Re: [PATCH v7 00/12] Enabling DCD emulation support in Qemu

[Svetly Todorov]

I tested the v7 DC add/release as part of the MHSLD RFC work I did
with Gregory. Those commands worked as expected. Feel free to tack on:

Tested-by: Svetly Todorov mailto:svetly.todo...@memverge.com>>

Re: [PATCH v7 00/12] Enabling DCD emulation support in Qemu

[Gregory Price]

On Mon, Apr 22, 2024 at 01:04:48PM +0100, Jonathan Cameron wrote:
> On Sat, 20 Apr 2024 16:35:46 -0400
> Gregory Price  wrote:
> 
> > On Fri, Apr 19, 2024 at 11:43:14AM -0700, fan wrote:
> > > On Fri, Apr 19, 2024 at 02:24:36PM -0400, Gregory Price wrote:  
> > > > 
> > > > added review to all patches, will hopefully be able to add a Tested-by
> > > > tag early next week, along with a v1 RFC for MHD bit-tracking.
> > > > 
> > > > We've been testing v5/v6 for a bit, so I expect as soon as we get the
> > > > MHD code ported over to v7 i'll ship a tested-by tag pretty quick.
> > > > 
> > > > The super-set release will complicate a few things but this doesn't
> > > > look like a blocker on our end, just a change to how we track bits in a
> > > > shared bit/bytemap.
> > > >   
> > > 
> > > Hi Gregory,
> > > Thanks for reviewing the patches so quickly. 
> > > 
> > > No pressure, but look forward to your MHD work. :)
> > > 
> > > Fan  
> > 
> > Starting to get into versioniong hell a bit, since the Niagara work was
> > based off of jonathan's branch and the mhd-dcd work needs some of the
> > extentions from that branch - while this branch is based on master.
> > 
> > Probably we'll need to wait for a new cxl dated branch to try and sus
> > out the pain points before we push an RFC.  I would not want to have
> > conflicting commits for something like this for example:
> > 
> > https://lore.kernel.org/qemu-devel/20230901012914.226527-2-gregory.pr...@memverge.com/
> > 
> > We get merge conflicts here because this is behind that patch. So
> > pushing up an RFC in this state would be mostly useless to everyone
> 
> Subtle hint noted ;) 
>

Gentle nudge/poke/prod :P

Got your updates, thank you!  We should have something cleaned up today 
hopefully.

> I'll build a fresh tree - any remaining rebases until QEMU 9.0 should be
> straight forward anyway.   My ideal is that the NUMA GP series lands early
> in 9.1 cycle and this can go in parallel.  I'd really like to
> get this in early if possible so we can start clearing some of the other
> stuff that ended up built on top of it!
> 
> Jonathan
> 
> > 
> > ~Gregory
> 

Re: [PATCH v7 00/12] Enabling DCD emulation support in Qemu

[Jonathan Cameron via]

On Mon, 22 Apr 2024 15:23:16 +0100
Jonathan Cameron  wrote:

> On Mon, 22 Apr 2024 13:04:48 +0100
> Jonathan Cameron  wrote:
> 
> > On Sat, 20 Apr 2024 16:35:46 -0400
> > Gregory Price  wrote:
> >   
> > > On Fri, Apr 19, 2024 at 11:43:14AM -0700, fan wrote:
> > > > On Fri, Apr 19, 2024 at 02:24:36PM -0400, Gregory Price wrote:  
> > > > > 
> > > > > added review to all patches, will hopefully be able to add a Tested-by
> > > > > tag early next week, along with a v1 RFC for MHD bit-tracking.
> > > > > 
> > > > > We've been testing v5/v6 for a bit, so I expect as soon as we get the
> > > > > MHD code ported over to v7 i'll ship a tested-by tag pretty quick.
> > > > > 
> > > > > The super-set release will complicate a few things but this doesn't
> > > > > look like a blocker on our end, just a change to how we track bits in 
> > > > > a
> > > > > shared bit/bytemap.
> > > > >   
> > > > 
> > > > Hi Gregory,
> > > > Thanks for reviewing the patches so quickly. 
> > > > 
> > > > No pressure, but look forward to your MHD work. :)
> > > > 
> > > > Fan  
> > > 
> > > Starting to get into versioniong hell a bit, since the Niagara work was
> > > based off of jonathan's branch and the mhd-dcd work needs some of the
> > > extentions from that branch - while this branch is based on master.
> > > 
> > > Probably we'll need to wait for a new cxl dated branch to try and sus
> > > out the pain points before we push an RFC.  I would not want to have
> > > conflicting commits for something like this for example:
> > > 
> > > https://lore.kernel.org/qemu-devel/20230901012914.226527-2-gregory.pr...@memverge.com/
> > > 
> > > We get merge conflicts here because this is behind that patch. So
> > > pushing up an RFC in this state would be mostly useless to everyone
> > 
> > Subtle hint noted ;) 
> > 
> > I'll build a fresh tree - any remaining rebases until QEMU 9.0 should be
> > straight forward anyway.   My ideal is that the NUMA GP series lands early
> > in 9.1 cycle and this can go in parallel.  I'd really like to
> > get this in early if possible so we can start clearing some of the other
> > stuff that ended up built on top of it!  
> 
> I've pushed to gitlab.com/jic23/qemu cxl-2024-04-22-draft
> Its extremely lightly tested so far.
> 
> To save time, I've temporarily dropped the fm-api DCD initiate
> dynamic capacity add patch as that needs non trivial updates.
> 
> I've not yet caught up with some other outstanding series, but
> I will almost certainly put them on top of DCD.

If anyone pulled in meantime... I failed to push down a fix from
my working tree on top of this.
Goes to show I shouldn't ignore patches simply named "Push down" :(

Updated on same branch.

Jonathan
> 
> Jonathan
> 
> > 
> > Jonathan
> >   
> > > 
> > > ~Gregory
> > 
> >   
> 
> 

Re: [PATCH v7 00/12] Enabling DCD emulation support in Qemu

[Jonathan Cameron via]

On Mon, 22 Apr 2024 13:04:48 +0100
Jonathan Cameron  wrote:

> On Sat, 20 Apr 2024 16:35:46 -0400
> Gregory Price  wrote:
> 
> > On Fri, Apr 19, 2024 at 11:43:14AM -0700, fan wrote:  
> > > On Fri, Apr 19, 2024 at 02:24:36PM -0400, Gregory Price wrote:
> > > > 
> > > > added review to all patches, will hopefully be able to add a Tested-by
> > > > tag early next week, along with a v1 RFC for MHD bit-tracking.
> > > > 
> > > > We've been testing v5/v6 for a bit, so I expect as soon as we get the
> > > > MHD code ported over to v7 i'll ship a tested-by tag pretty quick.
> > > > 
> > > > The super-set release will complicate a few things but this doesn't
> > > > look like a blocker on our end, just a change to how we track bits in a
> > > > shared bit/bytemap.
> > > > 
> > > 
> > > Hi Gregory,
> > > Thanks for reviewing the patches so quickly. 
> > > 
> > > No pressure, but look forward to your MHD work. :)
> > > 
> > > Fan
> > 
> > Starting to get into versioniong hell a bit, since the Niagara work was
> > based off of jonathan's branch and the mhd-dcd work needs some of the
> > extentions from that branch - while this branch is based on master.
> > 
> > Probably we'll need to wait for a new cxl dated branch to try and sus
> > out the pain points before we push an RFC.  I would not want to have
> > conflicting commits for something like this for example:
> > 
> > https://lore.kernel.org/qemu-devel/20230901012914.226527-2-gregory.pr...@memverge.com/
> > 
> > We get merge conflicts here because this is behind that patch. So
> > pushing up an RFC in this state would be mostly useless to everyone  
> 
> Subtle hint noted ;) 
> 
> I'll build a fresh tree - any remaining rebases until QEMU 9.0 should be
> straight forward anyway.   My ideal is that the NUMA GP series lands early
> in 9.1 cycle and this can go in parallel.  I'd really like to
> get this in early if possible so we can start clearing some of the other
> stuff that ended up built on top of it!

I've pushed to gitlab.com/jic23/qemu cxl-2024-04-22-draft
Its extremely lightly tested so far.

To save time, I've temporarily dropped the fm-api DCD initiate
dynamic capacity add patch as that needs non trivial updates.

I've not yet caught up with some other outstanding series, but
I will almost certainly put them on top of DCD.

Jonathan

> 
> Jonathan
> 
> > 
> > ~Gregory  
> 
> 

Re: [PATCH v7 00/12] Enabling DCD emulation support in Qemu

[Jonathan Cameron via]

On Sat, 20 Apr 2024 16:35:46 -0400
Gregory Price  wrote:

> On Fri, Apr 19, 2024 at 11:43:14AM -0700, fan wrote:
> > On Fri, Apr 19, 2024 at 02:24:36PM -0400, Gregory Price wrote:  
> > > 
> > > added review to all patches, will hopefully be able to add a Tested-by
> > > tag early next week, along with a v1 RFC for MHD bit-tracking.
> > > 
> > > We've been testing v5/v6 for a bit, so I expect as soon as we get the
> > > MHD code ported over to v7 i'll ship a tested-by tag pretty quick.
> > > 
> > > The super-set release will complicate a few things but this doesn't
> > > look like a blocker on our end, just a change to how we track bits in a
> > > shared bit/bytemap.
> > >   
> > 
> > Hi Gregory,
> > Thanks for reviewing the patches so quickly. 
> > 
> > No pressure, but look forward to your MHD work. :)
> > 
> > Fan  
> 
> Starting to get into versioniong hell a bit, since the Niagara work was
> based off of jonathan's branch and the mhd-dcd work needs some of the
> extentions from that branch - while this branch is based on master.
> 
> Probably we'll need to wait for a new cxl dated branch to try and sus
> out the pain points before we push an RFC.  I would not want to have
> conflicting commits for something like this for example:
> 
> https://lore.kernel.org/qemu-devel/20230901012914.226527-2-gregory.pr...@memverge.com/
> 
> We get merge conflicts here because this is behind that patch. So
> pushing up an RFC in this state would be mostly useless to everyone

Subtle hint noted ;) 

I'll build a fresh tree - any remaining rebases until QEMU 9.0 should be
straight forward anyway.   My ideal is that the NUMA GP series lands early
in 9.1 cycle and this can go in parallel.  I'd really like to
get this in early if possible so we can start clearing some of the other
stuff that ended up built on top of it!

Jonathan

> 
> ~Gregory

Re: [PATCH v7 00/12] Enabling DCD emulation support in Qemu

[Gregory Price]

On Fri, Apr 19, 2024 at 11:43:14AM -0700, fan wrote:
> On Fri, Apr 19, 2024 at 02:24:36PM -0400, Gregory Price wrote:
> > 
> > added review to all patches, will hopefully be able to add a Tested-by
> > tag early next week, along with a v1 RFC for MHD bit-tracking.
> > 
> > We've been testing v5/v6 for a bit, so I expect as soon as we get the
> > MHD code ported over to v7 i'll ship a tested-by tag pretty quick.
> > 
> > The super-set release will complicate a few things but this doesn't
> > look like a blocker on our end, just a change to how we track bits in a
> > shared bit/bytemap.
> > 
> 
> Hi Gregory,
> Thanks for reviewing the patches so quickly. 
> 
> No pressure, but look forward to your MHD work. :)
> 
> Fan

Starting to get into versioniong hell a bit, since the Niagara work was
based off of jonathan's branch and the mhd-dcd work needs some of the
extentions from that branch - while this branch is based on master.

Probably we'll need to wait for a new cxl dated branch to try and sus
out the pain points before we push an RFC.  I would not want to have
conflicting commits for something like this for example:

https://lore.kernel.org/qemu-devel/20230901012914.226527-2-gregory.pr...@memverge.com/

We get merge conflicts here because this is behind that patch. So
pushing up an RFC in this state would be mostly useless to everyone.

~Gregory

Re: [PATCH v7 00/12] Enabling DCD emulation support in Qemu

[fan]

On Fri, Apr 19, 2024 at 02:24:36PM -0400, Gregory Price wrote:
> On Thu, Apr 18, 2024 at 04:10:51PM -0700, nifan@gmail.com wrote:
> > A git tree of this series can be found here (with one extra commit on top
> > for printing out accepted/pending extent list): 
> > https://github.com/moking/qemu/tree/dcd-v7
> > 
> > v6->v7:
> > 
> > 1. Fixed the dvsec range register issue mentioned in the the cover letter 
> > in v6.
> >Only relevant bits are set to mark the device ready (Patch 6). (Jonathan)
> > 2. Moved the if statement in cxl_setup_memory from Patch 6 to Patch 4. 
> > (Jonathan)
> > 3. Used MIN instead of if statement to get record_count in Patch 7. 
> > (Jonathan)
> > 4. Added "Reviewed-by" tag to Patch 7.
> > 5. Modified cxl_dc_extent_release_dry_run so the updated extent list can be
> >reused in cmd_dcd_release_dyn_cap to simplify the process in Patch 8. 
> > (Jørgen) 
> > 6. Added comments to indicate further "TODO" items in 
> > cmd_dcd_add_dyn_cap_rsp.
> > (Jonathan)
> > 7. Avoided irrelevant code reformat in Patch 8. (Jonathan)
> > 8. Modified QMP interfaces for adding/releasing DC extents to allow passing
> >tags, selection policy, flags in the interface. (Jonathan, Gregory)
> > 9. Redesigned the pending list so extents in the same requests are grouped
> > together. A new data structure is introduced to represent "extent group"
> > in pending list.  (Jonathan)
> > 10. Added support in QMP interface for "More" flag. 
> > 11. Check "Forced removal" flag for release request and not let it pass 
> > through.
> > 12. Removed the dynamic capacity log type from CxlEventLog definition in 
> > cxl.json
> >to avoid the side effect it may introduce to inject error to DC event 
> > log.
> >(Jonathan)
> > 13. Hard coded the event log type to dynamic capacity event log in QMP
> > interfaces. (Jonathan)
> > 14. Adding space in between "-1]". (Jonathan)
> > 15. Some minor comment fixes.
> > 
> > The code is tested with similar setup and has passed similar tests as listed
> > in the cover letter of v5[1] and v6[2].
> > Also, the code is tested with the latest DCD kernel patchset[3].
> > 
> > [1] Qemu DCD patchset v5: 
> > https://lore.kernel.org/linux-cxl/20240304194331.1586191-1-nifan@gmail.com/T/#t
> > [2] Qemu DCD patchset v6: 
> > https://lore.kernel.org/linux-cxl/20240325190339.696686-1-nifan@gmail.com/T/#t
> > [3] DCD kernel patches: 
> > https://lore.kernel.org/linux-cxl/20240324-dcd-type2-upstream-v1-0-b7b00d623...@intel.com/T/#m11c571e21c4fe17c7d04ec5c2c7bc7cbf2cd07e3
> >
> 
> added review to all patches, will hopefully be able to add a Tested-by
> tag early next week, along with a v1 RFC for MHD bit-tracking.
> 
> We've been testing v5/v6 for a bit, so I expect as soon as we get the
> MHD code ported over to v7 i'll ship a tested-by tag pretty quick.
> 
> The super-set release will complicate a few things but this doesn't
> look like a blocker on our end, just a change to how we track bits in a
> shared bit/bytemap.
> 

Hi Gregory,
Thanks for reviewing the patches so quickly. 

No pressure, but look forward to your MHD work. :)

Fan

> > 
> > Fan Ni (12):
> >   hw/cxl/cxl-mailbox-utils: Add dc_event_log_size field to output
> > payload of identify memory device command
> >   hw/cxl/cxl-mailbox-utils: Add dynamic capacity region representative
> > and mailbox command support
> >   include/hw/cxl/cxl_device: Rename mem_size as static_mem_size for
> > type3 memory devices
> >   hw/mem/cxl_type3: Add support to create DC regions to type3 memory
> > devices
> >   hw/mem/cxl-type3: Refactor ct3_build_cdat_entries_for_mr to take mr
> > size instead of mr as argument
> >   hw/mem/cxl_type3: Add host backend and address space handling for DC
> > regions
> >   hw/mem/cxl_type3: Add DC extent list representative and get DC extent
> > list mailbox support
> >   hw/cxl/cxl-mailbox-utils: Add mailbox commands to support add/release
> > dynamic capacity response
> >   hw/cxl/events: Add qmp interfaces to add/release dynamic capacity
> > extents
> >   hw/mem/cxl_type3: Add DPA range validation for accesses to DC regions
> >   hw/cxl/cxl-mailbox-utils: Add superset extent release mailbox support
> >   hw/mem/cxl_type3: Allow to release extent superset in QMP interface
> > 
> >  hw/cxl/cxl-mailbox-utils.c  | 620 ++-
> >  hw/mem/cxl_type3.c  | 633 +---
> >  hw/mem/cxl_type3_stubs.c|  20 ++
> >  include/hw/cxl/cxl_device.h |  81 -
> >  include/hw/cxl/cxl_events.h |  18 +
> >  qapi/cxl.json   |  69 
> >  6 files changed, 1396 insertions(+), 45 deletions(-)
> > 
> > -- 
> > 2.43.0
> > 

Re: [PATCH v7 00/12] Enabling DCD emulation support in Qemu

[Gregory Price]

On Thu, Apr 18, 2024 at 04:10:51PM -0700, nifan@gmail.com wrote:
> A git tree of this series can be found here (with one extra commit on top
> for printing out accepted/pending extent list): 
> https://github.com/moking/qemu/tree/dcd-v7
> 
> v6->v7:
> 
> 1. Fixed the dvsec range register issue mentioned in the the cover letter in 
> v6.
>Only relevant bits are set to mark the device ready (Patch 6). (Jonathan)
> 2. Moved the if statement in cxl_setup_memory from Patch 6 to Patch 4. 
> (Jonathan)
> 3. Used MIN instead of if statement to get record_count in Patch 7. (Jonathan)
> 4. Added "Reviewed-by" tag to Patch 7.
> 5. Modified cxl_dc_extent_release_dry_run so the updated extent list can be
>reused in cmd_dcd_release_dyn_cap to simplify the process in Patch 8. 
> (Jørgen) 
> 6. Added comments to indicate further "TODO" items in cmd_dcd_add_dyn_cap_rsp.
> (Jonathan)
> 7. Avoided irrelevant code reformat in Patch 8. (Jonathan)
> 8. Modified QMP interfaces for adding/releasing DC extents to allow passing
>tags, selection policy, flags in the interface. (Jonathan, Gregory)
> 9. Redesigned the pending list so extents in the same requests are grouped
> together. A new data structure is introduced to represent "extent group"
> in pending list.  (Jonathan)
> 10. Added support in QMP interface for "More" flag. 
> 11. Check "Forced removal" flag for release request and not let it pass 
> through.
> 12. Removed the dynamic capacity log type from CxlEventLog definition in 
> cxl.json
>to avoid the side effect it may introduce to inject error to DC event log.
>(Jonathan)
> 13. Hard coded the event log type to dynamic capacity event log in QMP
> interfaces. (Jonathan)
> 14. Adding space in between "-1]". (Jonathan)
> 15. Some minor comment fixes.
> 
> The code is tested with similar setup and has passed similar tests as listed
> in the cover letter of v5[1] and v6[2].
> Also, the code is tested with the latest DCD kernel patchset[3].
> 
> [1] Qemu DCD patchset v5: 
> https://lore.kernel.org/linux-cxl/20240304194331.1586191-1-nifan@gmail.com/T/#t
> [2] Qemu DCD patchset v6: 
> https://lore.kernel.org/linux-cxl/20240325190339.696686-1-nifan@gmail.com/T/#t
> [3] DCD kernel patches: 
> https://lore.kernel.org/linux-cxl/20240324-dcd-type2-upstream-v1-0-b7b00d623...@intel.com/T/#m11c571e21c4fe17c7d04ec5c2c7bc7cbf2cd07e3
>

added review to all patches, will hopefully be able to add a Tested-by
tag early next week, along with a v1 RFC for MHD bit-tracking.

We've been testing v5/v6 for a bit, so I expect as soon as we get the
MHD code ported over to v7 i'll ship a tested-by tag pretty quick.

The super-set release will complicate a few things but this doesn't
look like a blocker on our end, just a change to how we track bits in a
shared bit/bytemap.

> 
> Fan Ni (12):
>   hw/cxl/cxl-mailbox-utils: Add dc_event_log_size field to output
> payload of identify memory device command
>   hw/cxl/cxl-mailbox-utils: Add dynamic capacity region representative
> and mailbox command support
>   include/hw/cxl/cxl_device: Rename mem_size as static_mem_size for
> type3 memory devices
>   hw/mem/cxl_type3: Add support to create DC regions to type3 memory
> devices
>   hw/mem/cxl-type3: Refactor ct3_build_cdat_entries_for_mr to take mr
> size instead of mr as argument
>   hw/mem/cxl_type3: Add host backend and address space handling for DC
> regions
>   hw/mem/cxl_type3: Add DC extent list representative and get DC extent
> list mailbox support
>   hw/cxl/cxl-mailbox-utils: Add mailbox commands to support add/release
> dynamic capacity response
>   hw/cxl/events: Add qmp interfaces to add/release dynamic capacity
> extents
>   hw/mem/cxl_type3: Add DPA range validation for accesses to DC regions
>   hw/cxl/cxl-mailbox-utils: Add superset extent release mailbox support
>   hw/mem/cxl_type3: Allow to release extent superset in QMP interface
> 
>  hw/cxl/cxl-mailbox-utils.c  | 620 ++-
>  hw/mem/cxl_type3.c  | 633 +---
>  hw/mem/cxl_type3_stubs.c|  20 ++
>  include/hw/cxl/cxl_device.h |  81 -
>  include/hw/cxl/cxl_events.h |  18 +
>  qapi/cxl.json   |  69 
>  6 files changed, 1396 insertions(+), 45 deletions(-)
> 
> -- 
> 2.43.0
> 

[PATCH v7 00/12] Enabling DCD emulation support in Qemu

[nifan . cxl]

A git tree of this series can be found here (with one extra commit on top
for printing out accepted/pending extent list): 
https://github.com/moking/qemu/tree/dcd-v7

v6->v7:

1. Fixed the dvsec range register issue mentioned in the the cover letter in v6.
   Only relevant bits are set to mark the device ready (Patch 6). (Jonathan)
2. Moved the if statement in cxl_setup_memory from Patch 6 to Patch 4. 
(Jonathan)
3. Used MIN instead of if statement to get record_count in Patch 7. (Jonathan)
4. Added "Reviewed-by" tag to Patch 7.
5. Modified cxl_dc_extent_release_dry_run so the updated extent list can be
   reused in cmd_dcd_release_dyn_cap to simplify the process in Patch 8. 
(Jørgen) 
6. Added comments to indicate further "TODO" items in cmd_dcd_add_dyn_cap_rsp.
(Jonathan)
7. Avoided irrelevant code reformat in Patch 8. (Jonathan)
8. Modified QMP interfaces for adding/releasing DC extents to allow passing
   tags, selection policy, flags in the interface. (Jonathan, Gregory)
9. Redesigned the pending list so extents in the same requests are grouped
together. A new data structure is introduced to represent "extent group"
in pending list.  (Jonathan)
10. Added support in QMP interface for "More" flag. 
11. Check "Forced removal" flag for release request and not let it pass through.
12. Removed the dynamic capacity log type from CxlEventLog definition in 
cxl.json
   to avoid the side effect it may introduce to inject error to DC event log.
   (Jonathan)
13. Hard coded the event log type to dynamic capacity event log in QMP
interfaces. (Jonathan)
14. Adding space in between "-1]". (Jonathan)
15. Some minor comment fixes.

The code is tested with similar setup and has passed similar tests as listed
in the cover letter of v5[1] and v6[2].
Also, the code is tested with the latest DCD kernel patchset[3].

[1] Qemu DCD patchset v5: 
https://lore.kernel.org/linux-cxl/20240304194331.1586191-1-nifan@gmail.com/T/#t
[2] Qemu DCD patchset v6: 
https://lore.kernel.org/linux-cxl/20240325190339.696686-1-nifan@gmail.com/T/#t
[3] DCD kernel patches: 
https://lore.kernel.org/linux-cxl/20240324-dcd-type2-upstream-v1-0-b7b00d623...@intel.com/T/#m11c571e21c4fe17c7d04ec5c2c7bc7cbf2cd07e3


Fan Ni (12):
  hw/cxl/cxl-mailbox-utils: Add dc_event_log_size field to output
payload of identify memory device command
  hw/cxl/cxl-mailbox-utils: Add dynamic capacity region representative
and mailbox command support
  include/hw/cxl/cxl_device: Rename mem_size as static_mem_size for
type3 memory devices
  hw/mem/cxl_type3: Add support to create DC regions to type3 memory
devices
  hw/mem/cxl-type3: Refactor ct3_build_cdat_entries_for_mr to take mr
size instead of mr as argument
  hw/mem/cxl_type3: Add host backend and address space handling for DC
regions
  hw/mem/cxl_type3: Add DC extent list representative and get DC extent
list mailbox support
  hw/cxl/cxl-mailbox-utils: Add mailbox commands to support add/release
dynamic capacity response
  hw/cxl/events: Add qmp interfaces to add/release dynamic capacity
extents
  hw/mem/cxl_type3: Add DPA range validation for accesses to DC regions
  hw/cxl/cxl-mailbox-utils: Add superset extent release mailbox support
  hw/mem/cxl_type3: Allow to release extent superset in QMP interface

 hw/cxl/cxl-mailbox-utils.c  | 620 ++-
 hw/mem/cxl_type3.c  | 633 +---
 hw/mem/cxl_type3_stubs.c|  20 ++
 include/hw/cxl/cxl_device.h |  81 -
 include/hw/cxl/cxl_events.h |  18 +
 qapi/cxl.json   |  69 
 6 files changed, 1396 insertions(+), 45 deletions(-)

-- 
2.43.0