Re: [uClinux-dev] [PATCH 1/3] MPU support
On 08/26/2010 06:07 PM, Mike Frysinger wrote: On Thursday, August 26, 2010 18:45:08 Steve Longerbeam wrote: On 08/26/2010 12:04 PM, Mike Frysinger wrote: On Thursday, August 26, 2010 14:19:41 Steve Longerbeam wrote: The ARM MPU can do something similar. MPU regions can overlap, and a simple priority scheme is used to decide which region's permissions apply to a memory access that overlaps (higher numbered regions have higher priority). So on ARM we can "lock" a PTE/region, by defining region 0 to cover the entire address space, and give kernel read/write access, user no access. And region 0 is never overwritten or disabled. So if an access is made to an address not described by any other region, region 0 permissions are applied to the access (and a protection fault is generated if the access was made in user mode). Note that, with region 0 locked, that only leaves 7 PTEs/regions that can be swapped in and out for user processes. So with the ARM MPU, we can't create a region for every mmap(), we would run out of available entries. So we have to use a trade-off, only create an MPU region for XIP file mappings (text). All other mappings (non-XIP file mappings and anonymous mappings) allocate from a common user memory pool (which is another patch I plan to submit). i dont understand why running out of entries is a problem. we run out of entries too as you cant cover 512MiB of SDRAM with 16 entries. we simply take an exception when this occurs and in the exception handler, we use a basic round-robin replacement scheme to install a valid PTE (assuming of course the user has a valid mapping for the excepting address). then we return to the user process and it continues on. why wont this scheme work for you too ? no, you're right, that would work. Of course, it would have a bigger memory usage for the page tables, and a performance hit (with my implementation when a process is running there are no faults). But it is more inline with how MMU kernels work, and it adds process-to-process protection too. we used a bitmap to save on memory and execution. each bit representing a 4k chunk. this is the "page_rwx_mask" and similar stuff that appears in the Blackfin asm/mmu*.h headers. ok, I'll take a closer look. have you done performance measurements to see the overhead with the MPU turned on in your scheme compared to off ? doing something like a ffmpeg decode to another file. if the performance trade offs of your current scheme (per- mapping) is significant compared to the classic per-page, then it is worth while to extend the MPU Kconfig option so people can select per-page or per- mapping schema. yes, if performance degrades a lot for per-page compared to my current scheme, that would be worthwhile to offer both options. OTOH, other people may have different requirements (better protection being more important than memory footprint or performance, or vice-versa). So it might make sense to offer both options anyway. btw, i dont think it was mentioned earlier, but these ranges you're working with ... do they have alignment requirements ? yes, but it varies by ARM cores. For instance, on the SC100, the MPU regions must be 64-byte aligned, but the ARM940T has the same alignment requirement as blackfin (alignment = size). the thing about Blackfin PTEs is that they must be aligned according to the size they represent. so if it is a 4KiB mapping, it must be aligned to 4KiB. if it's 1MiB, it must be aligned to 1MiB. it'd be nice if that alignment restriction wasnt there as we could then do a flexible range mapping similar to what you have. -mike ___ uClinux-dev mailing list uClinux-dev@uclinux.org http://mailman.uclinux.org/mailman/listinfo/uclinux-dev This message was resent by uclinux-dev@uclinux.org To unsubscribe see: http://mailman.uclinux.org/mailman/options/uclinux-dev
Re: [uClinux-dev] [PATCH 1/3] MPU support
On 08/26/2010 12:04 PM, Mike Frysinger wrote: On Thursday, August 26, 2010 14:19:41 Steve Longerbeam wrote: The ARM MPU can do something similar. MPU regions can overlap, and a simple priority scheme is used to decide which region's permissions apply to a memory access that overlaps (higher numbered regions have higher priority). So on ARM we can "lock" a PTE/region, by defining region 0 to cover the entire address space, and give kernel read/write access, user no access. And region 0 is never overwritten or disabled. So if an access is made to an address not described by any other region, region 0 permissions are applied to the access (and a protection fault is generated if the access was made in user mode). Note that, with region 0 locked, that only leaves 7 PTEs/regions that can be swapped in and out for user processes. So with the ARM MPU, we can't create a region for every mmap(), we would run out of available entries. So we have to use a trade-off, only create an MPU region for XIP file mappings (text). All other mappings (non-XIP file mappings and anonymous mappings) allocate from a common user memory pool (which is another patch I plan to submit). i dont understand why running out of entries is a problem. we run out of entries too as you cant cover 512MiB of SDRAM with 16 entries. we simply take an exception when this occurs and in the exception handler, we use a basic round-robin replacement scheme to install a valid PTE (assuming of course the user has a valid mapping for the excepting address). then we return to the user process and it continues on. why wont this scheme work for you too ? no, you're right, that would work. Of course, it would have a bigger memory usage for the page tables, and a performance hit (with my implementation when a process is running there are no faults). But it is more inline with how MMU kernels work, and it adds process-to-process protection too. Steve ___ uClinux-dev mailing list uClinux-dev@uclinux.org http://mailman.uclinux.org/mailman/listinfo/uclinux-dev This message was resent by uclinux-dev@uclinux.org To unsubscribe see: http://mailman.uclinux.org/mailman/options/uclinux-dev
Re: [uClinux-dev] [PATCH 1/3] MPU support
On 08/24/2010 03:43 PM, Mike Frysinger wrote: Apparently the ARM MPU's are not nearly as capable as the blackfin MPU. The ARM MPU deals with whole regions, and typically only up to 8 memory regions can be controlled by the MPU at any one time, each region having one protection setting (r/w/x for kernel mode, r/w/x for user mode). Not nearly as fine grained as per-page. i dont quite understand what you mean by "whole region". if you define a "region" as 4KiB, dont you get the granularity expected ? could you describe the flexibility/restrictions of this a little more (i'm not an ARM core guy) ? the Blackfin MPU has separate insn/data TLBs, and each TLB has 16 entries (PTEs i believe is the common naming). each PTE has supervisor rwx and usermode rwx permissions. further, each PTE has a size field which may be 1KiB, 4KiB, 1MiB, or 4MiB. ok, sounds like the blackfin MPU has all the features of a true MMU but without the v-->p address translation. The ARM MPU, using MMU language, has an 8-entry TLB (some ARM MPUs have separate insn/data TLBs, others don't). But here's the kicker, the entire address space can only be described by 8 PTE's (aka MPU regions), total! So actually there is no need for a page table in main memory at all, since the TLB already has enough entries to cover the entire address space. i guess we cheat a little and we lock a PTE for the kernel itself so that it'll always be covered so we can process PTE misses without triggering a miss (nested exceptions). i'm not entirely familiar with the exact gory details of other arches, so i cant say how unique we are in this regard. The ARM MPU can do something similar. MPU regions can overlap, and a simple priority scheme is used to decide which region's permissions apply to a memory access that overlaps (higher numbered regions have higher priority). So on ARM we can "lock" a PTE/region, by defining region 0 to cover the entire address space, and give kernel read/write access, user no access. And region 0 is never overwritten or disabled. So if an access is made to an address not described by any other region, region 0 permissions are applied to the access (and a protection fault is generated if the access was made in user mode). Note that, with region 0 locked, that only leaves 7 PTEs/regions that can be swapped in and out for user processes. So with the ARM MPU, we can't create a region for every mmap(), we would run out of available entries. So we have to use a trade-off, only create an MPU region for XIP file mappings (text). All other mappings (non-XIP file mappings and anonymous mappings) allocate from a common user memory pool (which is another patch I plan to submit). So another locked region is used (region 1) that covers this user memory pool. User mode has read/write access of course, as well as kernel. And so we actually now only have 6 regions that user processes can play with. What this trade-off means is that we have process-to-kernel protection, but not process-to-process protection. So ARM could use something higher-level than protect_page(), something like protect_region(start, end, flags), or just all of protect_vma() could be moved to include/asm/mmu_context.h. That way ARM can operate on the whole region, while blackfin would add protection for every page in the VMA as it is doing now. i think you could use the existing framework, and perhaps optionally extend it. maybe if i knew a little more about your "regions", i could suggest something else. I'll work on another patch that better merges my original ARM MPU work into the blackfin work, and resubmit. great, thanks Btw, I probably should be working in whatever git tree people are submitting patches against, rather than the 20100628 release. Which git tree should I submit against? that's hard to say. if current mainline (2.6.36-rc2) has everything you need to boot a working system, then that is probably the place to base your work. i understand though that the arm/nommu work is taking a while to get into mainline, so that might not be feasible. in which case, you should find the very latest uclinux tree and use that. i know people like to base their work off a release, but in order to get merged, the focus has to be on the latest development tree. ok. Greg says that the core non-MMU stuff is in mainline now, so I'll work from mainline. Steve ___ uClinux-dev mailing list uClinux-dev@uclinux.org http://mailman.uclinux.org/mailman/listinfo/uclinux-dev This message was resent by uclinux-dev@uclinux.org To unsubscribe see: http://mailman.uclinux.org/mailman/options/uclinux-dev
Re: [uClinux-dev] [PATCH 1/3] MPU support
On 08/23/2010 01:18 PM, Mike Frysinger wrote: On Monday, August 23, 2010 14:16:30 Steve Longerbeam wrote: On 08/23/2010 10:47 AM, Steve Longerbeam wrote: On 08/22/2010 05:20 PM, Mike Frysinger wrote: >> the common code changes will need justification as to why they exist at all. we're doing MPU on Blackfin/nommu today without any of these. we support pretty much all the same features of a MMU system short of virtual memory -- 4k pages, RWX granularity, process to process protection, process to kernel protection (include kernel modules), kernel XIP, and userspace XIP. further, why did you go with CONFIG_CPU_CP15_MPU ? there is already a CONFIG_MPU option that is used in common nommu code. which tree has CONFIG_MPU, and the MPU support for blackfin? There is no such thing in the 888 uclinux release tree. sorry, I see CONFIG_MPU under blackfin in the 888 release. I'm not familiar with the blackfin arch, but my patches of course are specific to ARM MPU's. i dont see how the processor matters. you're running Linux without virtual memory support (CONFIG_MMU=n) and you want to do memory protection (CONFIG_MPU=y). there is no need to stick a specific cpu name in there. after all, the option is CONFIG_MPU and not CONFIG_BFIN_MPU because all the changes we made (which were few) to common code were processor independent (exactly like all changes to common code should be). we specifically left the door open for other processors to support MMU=n MPU=y without an ifdef mess. -mike Hi Mike, I don't disagree with what you're saying, but I was a bit confused because in the 888 kernel I was looking at, the common MPU code didn't exist yet. But looking at the 20100628 release, I can see mm/nommu:protect_vma(), that calls the common MPU API's protect_page() and update_protections(). Apparently the ARM MPU's are not nearly as capable as the blackfin MPU. The ARM MPU deals with whole regions, and typically only up to 8 memory regions can be controlled by the MPU at any one time, each region having one protection setting (r/w/x for kernel mode, r/w/x for user mode). Not nearly as fine grained as per-page. So ARM could use something higher-level than protect_page(), something like protect_region(start, end, flags), or just all of protect_vma() could be moved to include/asm/mmu_context.h. That way ARM can operate on the whole region, while blackfin would add protection for every page in the VMA as it is doing now. I'll work on another patch that better merges my original ARM MPU work into the blackfin work, and resubmit. Btw, I probably should be working in whatever git tree people are submitting patches against, rather than the 20100628 release. Which git tree should I submit against? Steve ___ uClinux-dev mailing list uClinux-dev@uclinux.org http://mailman.uclinux.org/mailman/listinfo/uclinux-dev This message was resent by uclinux-dev@uclinux.org To unsubscribe see: http://mailman.uclinux.org/mailman/options/uclinux-dev
Re: [uClinux-dev] [PATCH 1/3] MPU support
On 08/23/2010 10:47 AM, Steve Longerbeam wrote: On 08/22/2010 05:20 PM, Mike Frysinger wrote: as it stands, this breaks all non-arm NOMMU ports. the patch will need to be broken up into arm-specific and arm-independent parts. ok, I can do that. the common code changes will need justification as to why they exist at all. we're doing MPU on Blackfin/nommu today without any of these. we support pretty much all the same features of a MMU system short of virtual memory -- 4k pages, RWX granularity, process to process protection, process to kernel protection (include kernel modules), kernel XIP, and userspace XIP. further, why did you go with CONFIG_CPU_CP15_MPU ? there is already a CONFIG_MPU option that is used in common nommu code. which tree has CONFIG_MPU, and the MPU support for blackfin? There is no such thing in the 888 uclinux release tree. sorry, I see CONFIG_MPU under blackfin in the 888 release. I'm not familiar with the blackfin arch, but my patches of course are specific to ARM MPU's. Steve ___ uClinux-dev mailing list uClinux-dev@uclinux.org http://mailman.uclinux.org/mailman/listinfo/uclinux-dev This message was resent by uclinux-dev@uclinux.org To unsubscribe see: http://mailman.uclinux.org/mailman/options/uclinux-dev
Re: [uClinux-dev] [PATCH 1/3] MPU support
On 08/22/2010 05:20 PM, Mike Frysinger wrote: as it stands, this breaks all non-arm NOMMU ports. the patch will need to be broken up into arm-specific and arm-independent parts. ok, I can do that. the common code changes will need justification as to why they exist at all. we're doing MPU on Blackfin/nommu today without any of these. we support pretty much all the same features of a MMU system short of virtual memory -- 4k pages, RWX granularity, process to process protection, process to kernel protection (include kernel modules), kernel XIP, and userspace XIP. further, why did you go with CONFIG_CPU_CP15_MPU ? there is already a CONFIG_MPU option that is used in common nommu code. which tree has CONFIG_MPU, and the MPU support for blackfin? There is no such thing in the 888 uclinux release tree. I don't know which repo I should be developing under. I suppose it should be the git tree that people are posting patches against in the uclinux-dev list. Which tree is that? Thanks, Steve ___ uClinux-dev mailing list uClinux-dev@uclinux.org http://mailman.uclinux.org/mailman/listinfo/uclinux-dev This message was resent by uclinux-dev@uclinux.org To unsubscribe see: http://mailman.uclinux.org/mailman/options/uclinux-dev
[uClinux-dev] [PATCH 1/3] MPU support
This is a patch that adds Memory Protection Unit support for no-MMU ARM. The default MPU configuration on most ARM7-based cores only allows MPU control of the first 1MB of address space. However this can be easily changed, and on our core the MPU controls the first 1GB of address space, and the space above that is accessible only in privileged kernel mode. With this the MPU actually becomes a useful means of restricting userland access to kernel space in uClinux. This is the first in a set of three patches that provide a way to completely isolate userland from read/write access to kernel memory space. This first patch provides the base MPU framework. The platform-level code must implement a 'struct mpu' object that describes the MPU configuration of the core, and how to enable and disable MPU regions in the core. An example working implementation of 'struct mpu' is provided in arch/arm/mm/sc100.c. A process keeps track of its MPU regions with a 'struct mpu_region' object attached to its VMA's. During a context switch, the old processes mpu_regions are disabled, and the new processes mpu_regions are enabled, similar to how process page tables are switched. See switch_mm() for the no-MMU case. The next patch will implement a private user memory pool that processes allocate from, instead of allocating from kernel memory with kmalloc. It uses the general purpose allocator library (genalloc.c). In combination with the MPU support, userland by default has access only to this private pool of memory. The platform-level code will normally initialize two "background" MPU regions that are always active. The first gives kernel read/write access to all of memory space. The second gives user-level read/write access to the user memory pool. MPU regions that are attached to process VMA's are due to mmap() requests for XIP private file mappings. Note that non-XIP file mappings allocate memory for the mapping from the user memory pool, so with the user memory background MPU region there is no need to create a new mpu-region for the mapping. This patch is against the uclinux-888 release kernel. Steve diff -Nuar -X /home/stevel/dontdiff linux-2.6.x.orig/arch/arm/mm/Makefile linux-2.6.x/arch/arm/mm/Makefile --- linux-2.6.x.orig/arch/arm/mm/Makefile 2010-08-22 15:22:54.391149924 -0700 +++ linux-2.6.x/arch/arm/mm/Makefile 2010-08-22 15:43:33.761212277 -0700 @@ -9,6 +9,7 @@ ifneq ($(CONFIG_MMU),y) obj-y+= nommu.o consistent-nommu.o +obj-$(CONFIG_CPU_CP15_MPU) += mpu.o endif obj-$(CONFIG_MODULES) += proc-syms.o diff -Nuar -X /home/stevel/dontdiff linux-2.6.x.orig/arch/arm/mm/mpu.c linux-2.6.x/arch/arm/mm/mpu.c --- linux-2.6.x.orig/arch/arm/mm/mpu.c 1969-12-31 16:00:00.0 -0800 +++ linux-2.6.x/arch/arm/mm/mpu.c 2010-08-22 15:29:28.916058187 -0700 @@ -0,0 +1,197 @@ +/* + * linux/arch/arm/mm/mpu.c + * + * Support for no-MMU processors that contain a Memory Protection Unit (MPU). + * + * Copyright (C) 2009 Netspectrum Communication Systems, Inc. + * Copyright (C) 2009 Silicon Storage Technology, Inc. + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + */ +#include +#include +#include +#include +#include +#include + +static struct mpu * mpu = NULL; + +static const char * access_str[] = { + [MPU_ACCESS_NONE] = "no access", + [MPU_ACCESS_RO] = "r/o", + [MPU_ACCESS_RW] = "r/w", +}; + +/* + * Static background regions are allocated from lowest priority. The + * first user region requested will fix highest_bg_prio, and no more + * background region requests will be allowed after that. + */ +static int highest_bg_prio = 0; +static int bg_prio_fixed = 0; + +static inline int find_free_prio(struct mm_struct * mm) +{ + struct vm_list_struct *vml; + int prio, inuse; + int ret = -ENOSPC; + + for (prio = highest_bg_prio; prio < mpu->num_prio; prio++) { + inuse = 0; + for (vml = mm->context.vmlist; vml; vml = vml->next) { + if (!vml->vma || !vml->vma->vm_mpu) +continue; + if (prio == vml->vma->vm_mpu->prio) { +inuse = 1; +break; + } + } + if (!inuse) { + ret = prio; + break; + } + } + + return ret; +} + +static inline int alloc_prio(struct mm_struct * mm) +{ + int prio; + + if (!mm) { + if (bg_prio_fixed) + return -EINVAL; + if (highest_bg_prio >= mpu->num_prio - 1) + return -ENOSPC; + prio = highest_bg_prio++; + } else { + bg_prio_fixed = 1; + prio = find_free_prio(mm); + } + + return prio; +} + +/* + * The mm->mmap_sem should be held atleast for read when mm != NULL. + */ +int mpu_add_region(struct mm_struct * mm, struct mpu_region * region) +{ + int ret; + + if (!mpu || !mpu->add_region) + return -ENODEV; + if (!region || + region->base > mpu->max_addr || + region->limit > mpu->max_addr || + region->base > region->limit) + return -EINVAL; + + ret = alloc_prio(mm); + if (ret