On 6/4/05, Christoph Hellwig <[EMAIL PROTECTED]> wrote: > > > That is > > why the Quadrics extensions for zero placement are not > > applicable to an RDMA device. That extension, for example, > > only remaps memory when the IO device is not actively > > doing a transfer. Something that the kernel does not > > know with an RDMA device. > > How is memory remapping related to device removal? > >
Different problem, but the required solutions overlap. To remap a pinned memory region the OS must tell the RDMA device to suspend it's access to that memory. In doing so it must wait for in-progress transfers to complete. The RDMA Device is then in a state where it defers all activities associated with the suspended memory. Work Reqeusts that reference it are effectively fenced. Inbound packets that refernce it must be deferred (by dropping it, RNRs or interim buffering). After the OS has migrated the pages it supplies the new mapping to the RDMA Device, which lets it resume the Memory Region (after it has updated the PBLEs). The Quadrics patch informed zero copy devices of new mappings, but does so when the device is not actively doing work. That latter piece of information is not available for an RDMA device. For normal devices the OS has a reasonable expectation on when the device is performing DMA, because it is doing so in response to a request that the OS knows about. For an RDMA device a DMA transfer can be in progress as as a result of an RDMA Read -- something the host processor knows nothing about. _______________________________________________ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general