On Wed, Aug 19, 2026 at 2:48 PM Peter Xu <[email protected]> wrote: > > On Wed, Aug 19, 2026 at 02:01:49PM +0200, Jinpu Wang wrote: > > On Wed, Aug 19, 2026 at 12:52 PM Jinpu Wang <[email protected]> > > wrote: > > > > > > On Mon, Aug 17, 2026 at 10:24 PM Peter Xu <[email protected]> wrote: > > > > > > > > The unregister code was there since the first commit RDMA migration was > > > > merged, but it was never functioning. Remove the dead code. > > > > > > > > Since the two control messages are the last ones, we don't even need to > > > > worry about compatibility of legacy RDMA control commands, we can > > > > directly > > > > remove the messages too. > > > > > > > > As a side effect, this patch closes a report by removing the code > > > > completely. > > > > > > > > Reported-by: Tristan (@TristanInSec) > > > > Closes: https://gitlab.com/qemu-project/qemu/-/work_items/4003 > > > > Signed-off-by: Peter Xu <[email protected]> > > > Hi Peter, hi Yanfei, > > Hi, Jinpu, > > > > > > > The change looks fine, and I have also tested to ensure it doesn't > > > break compatibility with older versions. As already discussed, > > > removing this code could result in more MRs being used during > > > migration; however, once the migration is complete, they will be > > > unregistered, so there is no leak. > > Could you help explain why removing this code could result in more MRs > being used? I didn't see these two control messages used, so I expect the > behavior should be exactly the same before/after this patch, with either a > new or old QEMU binary. You are right, write of rdma->unregistrations[...] = <wr_id> (which is what the historical qemu_rdma_signal_unregister() used to do, which was removed 4 years ago in 246683c22f21 ("migration: remove unreachble RDMA code in save_hook impl") )
I was under impression, ibv_dereg_mr is call less with the change, I missed the fact above. > > > > > > > Reviewed-by: Jinpu Wang <[email protected]> > > > > --- > > > > migration/rdma.c | 133 ----------------------------------------- > > > > migration/trace-events | 7 --- > > > > 2 files changed, 140 deletions(-) > > > > > > > > diff --git a/migration/rdma.c b/migration/rdma.c > > > > index 62a509b236..eae4afd6df 100644 > > > > --- a/migration/rdma.c > > > > +++ b/migration/rdma.c > > > > @@ -142,8 +142,6 @@ enum { > > > > RDMA_CONTROL_REGISTER_REQUEST, /* dynamic page registration */ > > > > RDMA_CONTROL_REGISTER_RESULT, /* key to use after registration > > > > */ > > > > RDMA_CONTROL_REGISTER_FINISHED, /* current iteration finished */ > > > > - RDMA_CONTROL_UNREGISTER_REQUEST, /* dynamic UN-registration */ > > > > - RDMA_CONTROL_UNREGISTER_FINISHED, /* unpinning finished */ > > > > RDMA_CONTROL_NUM, > > > > }; > > > > > > > > @@ -232,8 +230,6 @@ static const char *control_desc(unsigned int > > > > rdma_control) > > > > [RDMA_CONTROL_REGISTER_REQUEST] = "REGISTER REQUEST", > > > > [RDMA_CONTROL_REGISTER_RESULT] = "REGISTER RESULT", > > > > [RDMA_CONTROL_REGISTER_FINISHED] = "REGISTER FINISHED", > > > > - [RDMA_CONTROL_UNREGISTER_REQUEST] = "UNREGISTER REQUEST", > > > > - [RDMA_CONTROL_UNREGISTER_FINISHED] = "UNREGISTER FINISHED", > > > > }; > > > > > > > > if (rdma_control >= RDMA_CONTROL_NUM) { > > > > @@ -370,9 +366,6 @@ typedef struct RDMAContext { > > > > int total_registrations; > > > > int total_writes; > > > > > > > > - int unregister_current, unregister_next; > > > > - uint64_t unregistrations[RDMA_SIGNALED_SEND_MAX]; > > > > - > > > > GHashTable *blockmap; > > > > > > > > /* the RDMAContext for return path */ > > > > @@ -1186,91 +1179,6 @@ static int qemu_rdma_reg_control(RDMAContext > > > > *rdma, int idx) > > > > return -1; > > > > } > > > > > > > > -/* > > > > - * Perform a non-optimized memory unregistration after every transfer > > > > - * for demonstration purposes, only if pin-all is not requested. > > > > - * > > > > - * Potential optimizations: > > > > - * 1. Start a new thread to run this function continuously > > > > - - for bit clearing > > > > - - and for receipt of unregister messages > > > > - * 2. Use an LRU. > > > > - * 3. Use workload hints. > > > > - */ > > > > -static int qemu_rdma_unregister_waiting(RDMAContext *rdma) > > > > -{ > > > > - Error *err = NULL; > > > > - > > > > - while (rdma->unregistrations[rdma->unregister_current]) { > > > > - int ret; > > > > - uint64_t wr_id = > > > > rdma->unregistrations[rdma->unregister_current]; > > > > - uint64_t chunk = > > > > - (wr_id & RDMA_WRID_CHUNK_MASK) >> RDMA_WRID_CHUNK_SHIFT; > > > > - uint64_t index = > > > > - (wr_id & RDMA_WRID_BLOCK_MASK) >> RDMA_WRID_BLOCK_SHIFT; > > > > - RDMALocalBlock *block = > > > > - &(rdma->local_ram_blocks.block[index]); > > > > - RDMARegister reg = { .current_index = index }; > > > > - RDMAControlHeader resp = { .type = > > > > RDMA_CONTROL_UNREGISTER_FINISHED, > > > > - }; > > > > - RDMAControlHeader head = { .len = sizeof(RDMARegister), > > > > - .type = > > > > RDMA_CONTROL_UNREGISTER_REQUEST, > > > > - .repeat = 1, > > > > - }; > > > > - > > > > - trace_qemu_rdma_unregister_waiting_proc(chunk, > > > > - > > > > rdma->unregister_current); > > > > - > > > > - rdma->unregistrations[rdma->unregister_current] = 0; > > > > - rdma->unregister_current++; > > > > - > > > > - if (rdma->unregister_current == RDMA_SIGNALED_SEND_MAX) { > > > > - rdma->unregister_current = 0; > > > > - } > > > > - > > > > - > > > > - /* > > > > - * Unregistration is speculative (because migration is > > > > single-threaded > > > > - * and we cannot break the protocol's inifinband message > > > > ordering). > > > > - * Thus, if the memory is currently being used for > > > > transmission, > > > > - * then abort the attempt to unregister and try again > > > > - * later the next time a completion is received for this > > > > memory. > > > > - */ > > > > - clear_bit(chunk, block->unregister_bitmap); > > > > - > > > > - if (test_bit(chunk, block->transit_bitmap)) { > > > > - trace_qemu_rdma_unregister_waiting_inflight(chunk); > > > > - continue; > > > > - } > > > > - > > > > - trace_qemu_rdma_unregister_waiting_send(chunk); > > > > - > > > > - ret = ibv_dereg_mr(block->pmr[chunk]); > > > > - block->pmr[chunk] = NULL; > > > > - block->remote_keys[chunk] = 0; > > > > - > > > > - if (ret != 0) { > > > > - error_report("unregistration chunk failed: %s", > > > > - strerror(ret)); > > > > - return -1; > > > > - } > > > > - rdma->total_registrations--; > > > > - > > > > - reg.key.chunk = chunk; > > > > - register_to_network(rdma, ®); > > > > - ret = qemu_rdma_exchange_send(rdma, &head, (uint8_t *) ®, > > > > - &resp, NULL, NULL, &err); > > > > - if (ret < 0) { > > > > - error_report_err(err); > > > > - return -1; > > > > - } > > > > - > > > > - trace_qemu_rdma_unregister_waiting_complete(chunk); > > > > - } > > > > - > > > > - return 0; > > > > -} > > > > - > > > > static uint64_t qemu_rdma_make_wrid(uint64_t wr_id, uint64_t index, > > > > uint64_t chunk) > > > > { > > > > @@ -2757,8 +2665,6 @@ static int qemu_rdma_drain_cq(RDMAContext *rdma) > > > > } > > > > } > > > > > > > > - qemu_rdma_unregister_waiting(rdma); > > > > - > > > > return 0; > > > > } > > > > > > > > @@ -3336,10 +3242,6 @@ int rdma_registration_handle(QEMUFile *f) > > > > .type = RDMA_CONTROL_REGISTER_RESULT, > > > > .repeat = 0, > > > > }; > > > > - RDMAControlHeader unreg_resp = { .len = 0, > > > > - .type = > > > > RDMA_CONTROL_UNREGISTER_FINISHED, > > > > - .repeat = 0, > > > > - }; > > > > RDMAControlHeader blocks = { .type = > > > > RDMA_CONTROL_RAM_BLOCKS_RESULT, > > > > .repeat = 1 }; > > > > QIOChannelRDMA *rioc; > > > > @@ -3552,41 +3454,6 @@ int rdma_registration_handle(QEMUFile *f) > > > > ret = qemu_rdma_post_send_control(rdma, > > > > (uint8_t *) results, ®_resp, &err); > > > > > > > > - if (ret < 0) { > > > > - error_report_err(err); > > > > - goto err; > > > > - } > > > > - break; > > > > - case RDMA_CONTROL_UNREGISTER_REQUEST: > > > > - trace_rdma_registration_handle_unregister(head.repeat); > > > > - unreg_resp.repeat = head.repeat; > > > > - registers = (RDMARegister *) > > > > rdma->wr_data[idx].control_curr; > > > > - > > > > - for (int count = 0; count < head.repeat; count++) { > > > > - reg = ®isters[count]; > > > > - network_to_register(reg); > > > > - > > > > - trace_rdma_registration_handle_unregister_loop(count, > > > > - reg->current_index, reg->key.chunk); > > > > - > > > > - block = > > > > &(rdma->local_ram_blocks.block[reg->current_index]); > > > > - > > > > - ret = ibv_dereg_mr(block->pmr[reg->key.chunk]); > > > > - block->pmr[reg->key.chunk] = NULL; > > > > - > > > > - if (ret != 0) { > > > > - error_report("rdma unregistration chunk failed: > > > > %s", > > > > - strerror(errno)); > > > > - goto err; > > > > - } > > > > - > > > > - rdma->total_registrations--; > > > > - > > > > - > > > > trace_rdma_registration_handle_unregister_success(reg->key.chunk); > > > > - } > > > > - > > > > - ret = qemu_rdma_post_send_control(rdma, NULL, &unreg_resp, > > > > &err); > > > > - > > > > if (ret < 0) { > > > > error_report_err(err); > > > > goto err; > > > > diff --git a/migration/trace-events b/migration/trace-events > > > > index af0e784535..d1fe8f1382 100644 > > > > --- a/migration/trace-events > > > > +++ b/migration/trace-events > > > > @@ -244,10 +244,6 @@ qemu_rdma_advise_mr(const char *name, uint32_t > > > > len, uint64_t addr, const char *r > > > > qemu_rdma_resolve_host_trying(const char *host, const char *ip) > > > > "Trying %s => %s" > > > > qemu_rdma_signal_unregister_append(uint64_t chunk, int pos) "Appending > > > > unregister chunk %" PRIu64 " at position %d" > > > > qemu_rdma_signal_unregister_already(uint64_t chunk) "Unregister chunk > > > > %" PRIu64 " already in queue" > > Not related to this change. Above 2 functions are not used, we might > > want to remove them. > > True, I'll drop them altogether. > > Thanks, > > > > > > > -qemu_rdma_unregister_waiting_inflight(uint64_t chunk) "Cannot > > > > unregister inflight chunk: %" PRIu64 > > > > -qemu_rdma_unregister_waiting_proc(uint64_t chunk, int pos) "Processing > > > > unregister for chunk: %" PRIu64 " at position %d" > > > > -qemu_rdma_unregister_waiting_send(uint64_t chunk) "Sending unregister > > > > for chunk: %" PRIu64 > > > > -qemu_rdma_unregister_waiting_complete(uint64_t chunk) "Unregister for > > > > chunk: %" PRIu64 " complete." > > > > qemu_rdma_write_flush(int sent) "sent total: %d" > > > > qemu_rdma_write_one_block(int count, int block, uint64_t chunk, > > > > uint64_t current, uint64_t len, int nb_sent, int nb_chunks) "(%d) Not > > > > clobbering: block: %d chunk %" PRIu64 " current %" PRIu64 " len %" > > > > PRIu64 " %d %d" > > > > qemu_rdma_write_one_post(uint64_t chunk, long addr, long remote, > > > > uint32_t len) "Posting chunk: %" PRIu64 ", addr: 0x%lx remote: 0x%lx, > > > > bytes %" PRIu32 > > > > @@ -266,9 +262,6 @@ rdma_registration_handle_ram_blocks_loop(const char > > > > *name, uint64_t offset, uint > > > > rdma_registration_handle_register(int requests) "%d requests" > > > > rdma_registration_handle_register_loop(int req, int index, uint64_t > > > > addr, uint64_t chunks) "Registration request (%d): index %d, > > > > current_addr %" PRIu64 " chunks: %" PRIu64 > > > > rdma_registration_handle_register_rkey(int rkey) "0x%x" > > > > -rdma_registration_handle_unregister(int requests) "%d requests" > > > > -rdma_registration_handle_unregister_loop(int count, int index, > > > > uint64_t chunk) "Unregistration request (%d): index %d, chunk %" PRIu64 > > > > -rdma_registration_handle_unregister_success(uint64_t chunk) "%" PRIu64 > > > > rdma_registration_handle_wait(void) "" > > > > rdma_registration_start(uint64_t flags) "%" PRIu64 > > > > rdma_registration_stop(uint64_t flags) "%" PRIu64 > > > > -- > > > > 2.54.0 > > > > > > > > -- > Peter Xu >
