Re: Problem Pinning Physical Memory
On Thu, Dec 2, 2010 at 2:19 AM, Or Gerlitz ogerl...@voltaire.com wrote: Alan, kernel patches posted to this mailing list are against the mainline (upstream) Linux kernel Thanks for pointing me to the correct kernel. I found Tom's patch from July, but was unsure if that was the patch in question or not. It seems that he will have an updated version with ibv_reg_mr() and not ibv_reg_io_mr() doing the registration, so I will just wait for the update. -- Alan -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem Pinning Physical Memory
Tom Tucker t...@... writes: What I implemented was support for mmap'd memory. In practical terms for your application you would write a driver that supported the mmap file op. The driver's mmap routine would ioremap the pci memory of interest and stuff it in the provided vma. The user-mode app then ibv_reg_mr the address and length returned by mmap. Make sense? I believe so. Something along these lines has already been implemented for the device with which I am working, but was failing due to lacking your patch. I am not able to find the patch that you submitted (bad search terms?). I also am having a difficult time tracking down (with any certainty) the sources that your patch would be made against. If you could provide lines to both I would greatly appreciate it. Thanks. -- Alan -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem Pinning Physical Memory
On 11/30/10 9:24 AM, Alan Cook wrote: Tom Tuckert...@... writes: Yes. I removed the new verb and followed Jason's recommendation of adding this support to the core reg_mr support. I used the type bits in the vma struct to determine the type of memory being registered and just did the right thing. I'll repost in the the next day or two. Tom Tom, Couple of questions: I noticed that OFED 1.5.3 was released last week. Are the changes you speak of part of that release? No. If not, is there an alternate branch/project that I should be looking at or into to for the mentioned changes? The patch will be against the top-of-tree Linux kernel. Also, I am inferring that the changes allowing the registering of physical memory will only happen if my application is running in kernel space. Actually, no. Is this correct? or will I be able to register the physical memory from user space now as well? What I implemented was support for mmap'd memory. In practical terms for your application you would write a driver that supported the mmap file op. The driver's mmap routine would ioremap the pci memory of interest and stuff it in the provided vma. The user-mode app then ibv_reg_mr the address and length returned by mmap. Make sense? Tom -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem Pinning Physical Memory
On 11/24/2010 11:42 AM, Jason Gunthorpe wrote: The last time this came up I said that the kernel side of ibv_reg_mr should do the right thing for all types of memory that are mmap'd into a process and I still think that is true. RDMA to device memory could be very useful and with things like GEM managing the allocation of device (video) memory to userspace, so it can be done safely. Jason Tom posted changes to support this a while back. Tom, do you have an updated patch series for this support? Steve. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem Pinning Physical Memory
On 11/29/10 11:10 AM, Steve Wise wrote: On 11/24/2010 11:42 AM, Jason Gunthorpe wrote: The last time this came up I said that the kernel side of ibv_reg_mr should do the right thing for all types of memory that are mmap'd into a process and I still think that is true. RDMA to device memory could be very useful and with things like GEM managing the allocation of device (video) memory to userspace, so it can be done safely. Jason Tom posted changes to support this a while back. Tom, do you have an updated patch series for this support? Yes. I removed the new verb and followed Jason's recommendation of adding this support to the core reg_mr support. I used the type bits in the vma struct to determine the type of memory being registered and just did the right thing. I'll repost in the the next day or two. Tom Steve. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem Pinning Physical Memory
Eli Cohen e...@... writes: You can't regiter IO memory using ibv_reg_mr() - you can only do that by registering that memory using physical memory registration which is available only for kernel consumers. You can't allow userspace to register physical pages as this is a security breach. So, would ibv_reg_mr work with physical memory if run from a kernel module? (or would something else have to be used instead). I have no requirement that this run in userspace. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem Pinning Physical Memory
On Mon, Nov 22, 2010 at 06:28:47PM +, Brian wrote: ibv_reg_mr works fine as long as the shared region is in RAM. However, if I change the memory region to the physical memory on a PCI device the call to ibv_reg_mr fails. What should I be doing to register a physical memory region? In searching through this forum I've seen some patches relating to reg_io_mr/dereg_io_mr. However, I don't see those changes in the 1.5.2 ofed sources. I had hoped to try applying the patches myself but the structure of the unchanged code in the patches appears to be different than the 1.5.2 ofed sources. Is there another, more recent, code base that I should be using? You can't regiter IO memory using ibv_reg_mr() - you can only do that by registering that memory using physical memory registration which is available only for kernel consumers. You can't allow userspace to register physical pages as this is a security breach. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Problem Pinning Physical Memory
On Wed, Nov 24, 2010 at 03:55:07PM +0200, Eli Cohen wrote: On Mon, Nov 22, 2010 at 06:28:47PM +, Brian wrote: ibv_reg_mr works fine as long as the shared region is in RAM. However, if I change the memory region to the physical memory on a PCI device the call to ibv_reg_mr fails. What should I be doing to register a physical memory region? In searching through this forum I've seen some patches relating to reg_io_mr/dereg_io_mr. However, I don't see those changes in the 1.5.2 ofed sources. I had hoped to try applying the patches myself but the structure of the unchanged code in the patches appears to be different than the 1.5.2 ofed sources. Is there another, more recent, code base that I should be using? You can't regiter IO memory using ibv_reg_mr() - you can only do that by registering that memory using physical memory registration which is available only for kernel consumers. You can't allow userspace to register physical pages as this is a security breach. Eh? If it is mmap'd into my processes address space then letting it be used for RDMA makes no difference to the security of the system. The last time this came up I said that the kernel side of ibv_reg_mr should do the right thing for all types of memory that are mmap'd into a process and I still think that is true. RDMA to device memory could be very useful and with things like GEM managing the allocation of device (video) memory to userspace, so it can be done safely. Jason -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Problem Pinning Physical Memory
ibv_reg_mr works fine as long as the shared region is in RAM. However, if I change the memory region to the physical memory on a PCI device the call to ibv_reg_mr fails. What should I be doing to register a physical memory region? In searching through this forum I've seen some patches relating to reg_io_mr/dereg_io_mr. However, I don't see those changes in the 1.5.2 ofed sources. I had hoped to try applying the patches myself but the structure of the unchanged code in the patches appears to be different than the 1.5.2 ofed sources. Is there another, more recent, code base that I should be using? Thank you in advance for your time and help! -Brian PS. My apologies if this appears posted twice. I waited a couple of hours for the original to post and it didn't - despite seeing later posts appear. -- To unsubscribe from this list: send the line unsubscribe linux-rdma in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
