Re: forcibly unmap pages in driver?
Just an update to my situation... I've implemented my idea of clearing the associated PTE's when I need to free the DMA buffer, then re-filling them in nopage(). This seems to work fine; if the user process tries anything fishy, it gets a SIGBUS instead of accessing the old mapping. I encountered two difficulties with the implementation: 1) zap_page_range(), flush_cache_range(), and flush_tlb_range() are not exported to drivers. I basically copied the guts of zap_page_range() into my driver, which seems to work OK on x86, but I know it will have trouble with architectures that require special treatment of PTE manipulation... 2) the state of mm->mmap_sem is unknown when my file_operations->release() function is called. If release() is called when the last FD closes, then mm->mmap_sem is not taken. But if release() is called from do_munmap, then mmap_sem has already been taken. So, it is risky to mess with vma's inside of release()... Regards, Dan > >> Later, the program calls the ioctl() again to set a smaller > >> buffer size, or closes the file descriptor. At this point > >> I'd like to shrink the buffer or free it completely. But I > >> can't assume that the program will be nice and munmap() the > >> region for me > > > Look at drivers/char/drm, for example. At mmap time they allocate a > > vm_ops to the address space. With that you catch changes to the vma > > structure initiated by a user mmap, munmap, etc. You could also > > dynamically map the pages in using the nopage method (optional). - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: forcibly unmap pages in driver?
On Tue, 5 Jun 2001, Pete Wyckoff wrote: > [EMAIL PROTECTED] said: > > user-space program is still running. I need to remove the user-space > > mapping -- otherwise the user process would still have access to the > > now-freed pages. I need an inverse of remap_page_range(). > > That seems a bit perverse. How will the poor userspace program know > not to access the pages you have yanked away from it? If you plan > to kill it, better to do that directly. If you plan to signal it > that the mapping is gone, it can just call munmap() itself. I can very well imagine situations where you want to unmap a buffer beneath the userspace program to make it dying when not following the rules - see below. > However, do_munmap() will call zap_page_range() for you and take care of > cache and TLB flushing if you're going to do this in the kernel. > > Your driver mmap function is called by do_mmap_pgoff() which takes > care of those issues, and there is no (*munmap) in file_operations--- > perhaps you are the first driver writer to want to unmap in the kernel. Well, I don't know whether he is the first one, but if so, I'm probably the other one thinking about (temporarily) unmapping mmap(2)ed memory directly on behalf of the kernel. The idea is as follows: Let's assume the mmap'ed buffer should be either accessible by userland or (mutually exclusive) dedicated to I/O or DMA. So you need some synchronisation like userland calling ioctl(2) to release the buffer and blocks on select(2) or read(2) to get buffer access re-granted after DMA has finished. Let's further assume you want to make absolutely sure the userland app would really _never_, under no circumstance, access the buffer at the wrong moment where it would read corrupted data, i.e. after the ioctl() but before the file gets ready for nonblocking read again. Well, you could validate all the paths in the app that might access the mmap'ed buffer at some point and make sure this is either impossible or will be detected somehow. Furthermore, for any future change you have to do some re-validation... --- or, you simply make the mapping inaccessible inside the critical window so the application would immediately die if it would violate the rule. Note that it is not sufficient to require the app to do the unmap itself after calling ioctl() because this would not solve the problem - so you pay two syscalls for nothing. I believe this is a good reason to do the unmap in the drivers' ioctl() and remap again after DMA has finished. Basically, one could also temporarily switch to PROT_NONE to get the same behaviour, but I personally prefer unmapping due to more flexibility. Although not tested so far, I think it should be possible by simply calling do_munmap(), which is exported by mm/mmap.c. AFAICS the only thing to take care of is the correct handling of mm->mmap_sem. Question remains however, whether there is any reason not to do so, wrt. future changes for example. Regards Martin - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: forcibly unmap pages in driver?
On Tue, 5 Jun 2001, Pete Wyckoff wrote: [EMAIL PROTECTED] said: user-space program is still running. I need to remove the user-space mapping -- otherwise the user process would still have access to the now-freed pages. I need an inverse of remap_page_range(). That seems a bit perverse. How will the poor userspace program know not to access the pages you have yanked away from it? If you plan to kill it, better to do that directly. If you plan to signal it that the mapping is gone, it can just call munmap() itself. I can very well imagine situations where you want to unmap a buffer beneath the userspace program to make it dying when not following the rules - see below. However, do_munmap() will call zap_page_range() for you and take care of cache and TLB flushing if you're going to do this in the kernel. Your driver mmap function is called by do_mmap_pgoff() which takes care of those issues, and there is no (*munmap) in file_operations--- perhaps you are the first driver writer to want to unmap in the kernel. Well, I don't know whether he is the first one, but if so, I'm probably the other one thinking about (temporarily) unmapping mmap(2)ed memory directly on behalf of the kernel. The idea is as follows: Let's assume the mmap'ed buffer should be either accessible by userland or (mutually exclusive) dedicated to I/O or DMA. So you need some synchronisation like userland calling ioctl(2) to release the buffer and blocks on select(2) or read(2) to get buffer access re-granted after DMA has finished. Let's further assume you want to make absolutely sure the userland app would really _never_, under no circumstance, access the buffer at the wrong moment where it would read corrupted data, i.e. after the ioctl() but before the file gets ready for nonblocking read again. Well, you could validate all the paths in the app that might access the mmap'ed buffer at some point and make sure this is either impossible or will be detected somehow. Furthermore, for any future change you have to do some re-validation... --- or, you simply make the mapping inaccessible inside the critical window so the application would immediately die if it would violate the rule. Note that it is not sufficient to require the app to do the unmap itself after calling ioctl() because this would not solve the problem - so you pay two syscalls for nothing. I believe this is a good reason to do the unmap in the drivers' ioctl() and remap again after DMA has finished. Basically, one could also temporarily switch to PROT_NONE to get the same behaviour, but I personally prefer unmapping due to more flexibility. Although not tested so far, I think it should be possible by simply calling do_munmap(), which is exported by mm/mmap.c. AFAICS the only thing to take care of is the correct handling of mm-mmap_sem. Question remains however, whether there is any reason not to do so, wrt. future changes for example. Regards Martin - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: forcibly unmap pages in driver?
Just an update to my situation... I've implemented my idea of clearing the associated PTE's when I need to free the DMA buffer, then re-filling them in nopage(). This seems to work fine; if the user process tries anything fishy, it gets a SIGBUS instead of accessing the old mapping. I encountered two difficulties with the implementation: 1) zap_page_range(), flush_cache_range(), and flush_tlb_range() are not exported to drivers. I basically copied the guts of zap_page_range() into my driver, which seems to work OK on x86, but I know it will have trouble with architectures that require special treatment of PTE manipulation... 2) the state of mm-mmap_sem is unknown when my file_operations-release() function is called. If release() is called when the last FD closes, then mm-mmap_sem is not taken. But if release() is called from do_munmap, then mmap_sem has already been taken. So, it is risky to mess with vma's inside of release()... Regards, Dan Later, the program calls the ioctl() again to set a smaller buffer size, or closes the file descriptor. At this point I'd like to shrink the buffer or free it completely. But I can't assume that the program will be nice and munmap() the region for me Look at drivers/char/drm, for example. At mmap time they allocate a vm_ops to the address space. With that you catch changes to the vma structure initiated by a user mmap, munmap, etc. You could also dynamically map the pages in using the nopage method (optional). - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: forcibly unmap pages in driver?
>> Later, the program calls the ioctl() again to set a smaller >> buffer size, or closes the file descriptor. At this point >> I'd like to shrink the buffer or free it completely. But I >> can't assume that the program will be nice and munmap() the >> region for me > Look at drivers/char/drm, for example. At mmap time they allocate a > vm_ops to the address space. With that you catch changes to the vma > structure initiated by a user mmap, munmap, etc. You could also > dynamically map the pages in using the nopage method (optional). OK I think I have a solution... Whenever I need to re-allocate or free the DMA buffer, I could set all of the user's corresponding page table entries to deny all access. Then I'd get a page fault on the next access to the buffer, and inside nopage() I could update the user's mapping or send a SIGBUS as appropriate (hmm, just like restoring a file mapping that was thrown away)... So I just have to figure out how to find the user's page table entries that are pointing to the DMA buffer. Regards, Dan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: forcibly unmap pages in driver?
[EMAIL PROTECTED] said: > My driver uses a variable-size DMA buffer that it shares with user-space; I > provide an ioctl() to choose the buffer size and allocate the buffer. Say > the user program chooses a large buffer size, and mmap()s the entire buffer. > Later, the program calls the ioctl() again to set a smaller buffer size, or > closes the file descriptor. At this point I'd like to shrink the buffer or > free it completely. But I can't assume that the program will be nice and > munmap() the region for me - it might still have the large buffer mapped. > What should I do here? > > An easy solution would to allocate the largest possible buffer as my driver > is loaded, even if not all of it will be exposed to user-space. I don't > really like this choice because the buffer needs to be pinned in memory, and > the largest useful buffer size is very big (several tens of MB). Maybe I > should disallow more than one buffer allocation per open() of the device... > But the memory mapping will stay around even after close(), correct? I'd > hate to have to keep the buffer around until my driver module is unloaded. I see. Your explanation makes sense, and close won't affect the mmap (unless you explicitly trigger it in the driver, which you should). Other drivers deal with this; you may want to go grepping for do_munmap and see how they handle it. [ > [EMAIL PROTECTED] said: ] > > However, do_munmap() will call zap_page_range() for you and take care of > > cache and TLB flushing if you're going to do this in the kernel. > > I'm not sure if I could use do_munmap() -- how will I know if the user > program has called munmap() already, and then mmap()ed something else in the > same place? Then I'd be killing the wrong mapping... Look at drivers/char/drm, for example. At mmap time they allocate a vm_ops to the address space. With that you catch changes to the vma structure initiated by a user mmap, munmap, etc. You could also dynamically map the pages in using the nopage method (optional). The less elegant way of doing this is to put in your userspace API some conditions which say: if you do the following: open(fd); ioctl(fd, give_me_the_buf); munmap(some_or_all_of_the_buf); buf2 = mmap(...); close(fd); /* or ioctl(fd, shrink_the_buf); */ buf2[27] = 3; you will be killed with a sigbus. I.e. don't manually munmap the region. -- Pete - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: forcibly unmap pages in driver?
> That seems a bit perverse. How will the poor userspace program know > not to access the pages you have yanked away from it? If you plan > to kill it, better to do that directly. If you plan to signal it > that the mapping is gone, it can just call munmap() itself. Thanks Pete. I will explain situation I am envisioning; perhaps there is a better way to handle this -- My driver uses a variable-size DMA buffer that it shares with user-space; I provide an ioctl() to choose the buffer size and allocate the buffer. Say the user program chooses a large buffer size, and mmap()s the entire buffer. Later, the program calls the ioctl() again to set a smaller buffer size, or closes the file descriptor. At this point I'd like to shrink the buffer or free it completely. But I can't assume that the program will be nice and munmap() the region for me - it might still have the large buffer mapped. What should I do here? An easy solution would to allocate the largest possible buffer as my driver is loaded, even if not all of it will be exposed to user-space. I don't really like this choice because the buffer needs to be pinned in memory, and the largest useful buffer size is very big (several tens of MB). Maybe I should disallow more than one buffer allocation per open() of the device... But the memory mapping will stay around even after close(), correct? I'd hate to have to keep the buffer around until my driver module is unloaded. > However, do_munmap() will call zap_page_range() for you and take care of > cache and TLB flushing if you're going to do this in the kernel. I'm not sure if I could use do_munmap() -- how will I know if the user program has called munmap() already, and then mmap()ed something else in the same place? Then I'd be killing the wrong mapping... Regards, Dan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: forcibly unmap pages in driver?
[EMAIL PROTECTED] said: > I am writing a device driver that, like many others, exposes a shared memory > region to user-space via mmap(). The region is allocated with vmalloc(), the > pages are marked reserved, and the user-space mapping is implemented with > remap_page_range(). > > In my driver, I may have to free the underlying vmalloc() region while the > user-space program is still running. I need to remove the user-space > mapping -- otherwise the user process would still have access to the > now-freed pages. I need an inverse of remap_page_range(). > > Is zap_page_range() the function I am looking for? Unfortunately it's not > exported to modules =(. As a quick fix, I was thinking I could just remap > all of the user pages to point to a zeroed page or something... > > Another question- in the mm.c sources, I see that many of the memory-mapping > functions are surrounded by calls to flush_cache_range() and > flush_tlb_range(). But I don't see these calls in many drivers. Is it > necessary to make them when my driver maps or unmaps the shared memory > region? That seems a bit perverse. How will the poor userspace program know not to access the pages you have yanked away from it? If you plan to kill it, better to do that directly. If you plan to signal it that the mapping is gone, it can just call munmap() itself. However, do_munmap() will call zap_page_range() for you and take care of cache and TLB flushing if you're going to do this in the kernel. Your driver mmap function is called by do_mmap_pgoff() which takes care of those issues, and there is no (*munmap) in file_operations--- perhaps you are the first driver writer to want to unmap in the kernel. -- Pete - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: forcibly unmap pages in driver?
[EMAIL PROTECTED] said: I am writing a device driver that, like many others, exposes a shared memory region to user-space via mmap(). The region is allocated with vmalloc(), the pages are marked reserved, and the user-space mapping is implemented with remap_page_range(). In my driver, I may have to free the underlying vmalloc() region while the user-space program is still running. I need to remove the user-space mapping -- otherwise the user process would still have access to the now-freed pages. I need an inverse of remap_page_range(). Is zap_page_range() the function I am looking for? Unfortunately it's not exported to modules =(. As a quick fix, I was thinking I could just remap all of the user pages to point to a zeroed page or something... Another question- in the mm.c sources, I see that many of the memory-mapping functions are surrounded by calls to flush_cache_range() and flush_tlb_range(). But I don't see these calls in many drivers. Is it necessary to make them when my driver maps or unmaps the shared memory region? That seems a bit perverse. How will the poor userspace program know not to access the pages you have yanked away from it? If you plan to kill it, better to do that directly. If you plan to signal it that the mapping is gone, it can just call munmap() itself. However, do_munmap() will call zap_page_range() for you and take care of cache and TLB flushing if you're going to do this in the kernel. Your driver mmap function is called by do_mmap_pgoff() which takes care of those issues, and there is no (*munmap) in file_operations--- perhaps you are the first driver writer to want to unmap in the kernel. -- Pete - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: forcibly unmap pages in driver?
That seems a bit perverse. How will the poor userspace program know not to access the pages you have yanked away from it? If you plan to kill it, better to do that directly. If you plan to signal it that the mapping is gone, it can just call munmap() itself. Thanks Pete. I will explain situation I am envisioning; perhaps there is a better way to handle this -- My driver uses a variable-size DMA buffer that it shares with user-space; I provide an ioctl() to choose the buffer size and allocate the buffer. Say the user program chooses a large buffer size, and mmap()s the entire buffer. Later, the program calls the ioctl() again to set a smaller buffer size, or closes the file descriptor. At this point I'd like to shrink the buffer or free it completely. But I can't assume that the program will be nice and munmap() the region for me - it might still have the large buffer mapped. What should I do here? An easy solution would to allocate the largest possible buffer as my driver is loaded, even if not all of it will be exposed to user-space. I don't really like this choice because the buffer needs to be pinned in memory, and the largest useful buffer size is very big (several tens of MB). Maybe I should disallow more than one buffer allocation per open() of the device... But the memory mapping will stay around even after close(), correct? I'd hate to have to keep the buffer around until my driver module is unloaded. However, do_munmap() will call zap_page_range() for you and take care of cache and TLB flushing if you're going to do this in the kernel. I'm not sure if I could use do_munmap() -- how will I know if the user program has called munmap() already, and then mmap()ed something else in the same place? Then I'd be killing the wrong mapping... Regards, Dan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: forcibly unmap pages in driver?
[EMAIL PROTECTED] said: My driver uses a variable-size DMA buffer that it shares with user-space; I provide an ioctl() to choose the buffer size and allocate the buffer. Say the user program chooses a large buffer size, and mmap()s the entire buffer. Later, the program calls the ioctl() again to set a smaller buffer size, or closes the file descriptor. At this point I'd like to shrink the buffer or free it completely. But I can't assume that the program will be nice and munmap() the region for me - it might still have the large buffer mapped. What should I do here? An easy solution would to allocate the largest possible buffer as my driver is loaded, even if not all of it will be exposed to user-space. I don't really like this choice because the buffer needs to be pinned in memory, and the largest useful buffer size is very big (several tens of MB). Maybe I should disallow more than one buffer allocation per open() of the device... But the memory mapping will stay around even after close(), correct? I'd hate to have to keep the buffer around until my driver module is unloaded. I see. Your explanation makes sense, and close won't affect the mmap (unless you explicitly trigger it in the driver, which you should). Other drivers deal with this; you may want to go grepping for do_munmap and see how they handle it. [ [EMAIL PROTECTED] said: ] However, do_munmap() will call zap_page_range() for you and take care of cache and TLB flushing if you're going to do this in the kernel. I'm not sure if I could use do_munmap() -- how will I know if the user program has called munmap() already, and then mmap()ed something else in the same place? Then I'd be killing the wrong mapping... Look at drivers/char/drm, for example. At mmap time they allocate a vm_ops to the address space. With that you catch changes to the vma structure initiated by a user mmap, munmap, etc. You could also dynamically map the pages in using the nopage method (optional). The less elegant way of doing this is to put in your userspace API some conditions which say: if you do the following: open(fd); ioctl(fd, give_me_the_buf); munmap(some_or_all_of_the_buf); buf2 = mmap(...); close(fd); /* or ioctl(fd, shrink_the_buf); */ buf2[27] = 3; you will be killed with a sigbus. I.e. don't manually munmap the region. -- Pete - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: forcibly unmap pages in driver?
Later, the program calls the ioctl() again to set a smaller buffer size, or closes the file descriptor. At this point I'd like to shrink the buffer or free it completely. But I can't assume that the program will be nice and munmap() the region for me Look at drivers/char/drm, for example. At mmap time they allocate a vm_ops to the address space. With that you catch changes to the vma structure initiated by a user mmap, munmap, etc. You could also dynamically map the pages in using the nopage method (optional). OK I think I have a solution... Whenever I need to re-allocate or free the DMA buffer, I could set all of the user's corresponding page table entries to deny all access. Then I'd get a page fault on the next access to the buffer, and inside nopage() I could update the user's mapping or send a SIGBUS as appropriate (hmm, just like restoring a file mapping that was thrown away)... So I just have to figure out how to find the user's page table entries that are pointing to the DMA buffer. Regards, Dan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
forcibly unmap pages in driver?
I am writing a device driver that, like many others, exposes a shared memory region to user-space via mmap(). The region is allocated with vmalloc(), the pages are marked reserved, and the user-space mapping is implemented with remap_page_range(). In my driver, I may have to free the underlying vmalloc() region while the user-space program is still running. I need to remove the user-space mapping -- otherwise the user process would still have access to the now-freed pages. I need an inverse of remap_page_range(). Is zap_page_range() the function I am looking for? Unfortunately it's not exported to modules =(. As a quick fix, I was thinking I could just remap all of the user pages to point to a zeroed page or something... Another question- in the mm.c sources, I see that many of the memory-mapping functions are surrounded by calls to flush_cache_range() and flush_tlb_range(). But I don't see these calls in many drivers. Is it necessary to make them when my driver maps or unmaps the shared memory region? Regards, Dan - To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
forcibly unmap pages in driver?
I am writing a device driver that, like many others, exposes a shared memory region to user-space via mmap(). The region is allocated with vmalloc(), the pages are marked reserved, and the user-space mapping is implemented with remap_page_range(). In my driver, I may have to free the underlying vmalloc() region while the user-space program is still running. I need to remove the user-space mapping -- otherwise the user process would still have access to the now-freed pages. I need an inverse of remap_page_range(). Is zap_page_range() the function I am looking for? Unfortunately it's not exported to modules =(. As a quick fix, I was thinking I could just remap all of the user pages to point to a zeroed page or something... Another question- in the mm.c sources, I see that many of the memory-mapping functions are surrounded by calls to flush_cache_range() and flush_tlb_range(). But I don't see these calls in many drivers. Is it necessary to make them when my driver maps or unmaps the shared memory region? Regards, Dan - To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
