Re: [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Wed, Aug 03, 2011 at 12:43:50PM -0500, James Bottomley wrote: > I assume from the above that ARM has a hardware page walker? Correct, and speculative prefetch (which isn't prevented by not having TLB entries), so you can't keep entries out of the TLB. If it's in the page tables it can end up in the TLB. The problem is that we could end up with conflicting attributes available to the hardware for the same physical page, and it is _completely_ undefined how hardware behaves with that (except that it does not halt - and there's no exception path for the condition because there's no detection of the problem case.) So, if you had one mapping which was fully cacheable and another mapping which wasn't, you can flush the TLB all you like - it could be possible that you still up with an access through the non-cacheable mapping being cached (either hitting speculatively prefetched cache lines via the cacheable mapping, or the cacheable attributes being applied to the non-cacheable mapping - or conversely uncacheable attributes applied to the cacheable mapping.) Essentially, the condition is labelled 'unpredictable' in the TRMs, which basically means that not even observed behaviour can be relied upon, because there may be cases where the observed behaviour fails. -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 6/8] drivers: add Contiguous Memory Allocator
Hello, I'm sorry for the late reply. I must have missed this mail... On Wednesday, August 03, 2011 7:44 PM James Bottomley wrote: > [cc to ks-discuss added, since this may be a relevant topic] > > On Tue, 2011-07-05 at 14:27 +0200, Arnd Bergmann wrote: > > On Tuesday 05 July 2011, Russell King - ARM Linux wrote: > > > On Tue, Jul 05, 2011 at 09:41:48AM +0200, Marek Szyprowski wrote: > > > > The Contiguous Memory Allocator is a set of helper functions for DMA > > > > mapping framework that improves allocations of contiguous memory chunks. > > > > > > > > CMA grabs memory on system boot, marks it with CMA_MIGRATE_TYPE and > > > > gives back to the system. Kernel is allowed to allocate movable pages > > > > within CMA's managed memory so that it can be used for example for page > > > > cache when DMA mapping do not use it. On dma_alloc_from_contiguous() > > > > request such pages are migrated out of CMA area to free required > > > > contiguous block and fulfill the request. This allows to allocate large > > > > contiguous chunks of memory at any time assuming that there is enough > > > > free memory available in the system. > > > > > > > > This code is heavily based on earlier works by Michal Nazarewicz. > > > > > > And how are you addressing the technical concerns about aliasing of > > > cache attributes which I keep bringing up with this and you keep > > > ignoring and telling me that I'm standing in your way. > > Just to chime in here, parisc has an identical issue. If the CPU ever > sees an alias with different attributes for the same page, it will HPMC > the box (that's basically the bios will kill the system as being > architecturally inconsistent), so an architecture neutral solution on > this point is essential to us as well. > > > This is of course an important issue, and it's the one item listed as > > TODO in the introductory mail that sent. > > > > It's also a preexisting problem as far as I can tell, and it needs > > to be solved in __dma_alloc for both cases, dma_alloc_from_contiguous > > and __alloc_system_pages as introduced in patch 7. > > > > We've discussed this back and forth, and it always comes down to > > one of two ugly solutions: > > > > 1. Put all of the MIGRATE_CMA and pages into highmem and change > > __alloc_system_pages so it also allocates only from highmem pages. > > The consequences of this are that we always need to build kernels > > with highmem enabled and that we have less lowmem on systems that > > are already small, both of which can be fairly expensive unless > > you have lots of highmem already. > > So this would require that systems using the API have a highmem? (parisc > doesn't today). Yes, such solution will require highmem. It will introduce the highmem issues to systems that typically don't use highmem, that's why I searched for other solutions. > > 2. Add logic to unmap pages from the linear mapping, which is > > very expensive because it forces the use of small pages in the > > linear mapping (or in parts of it), and possibly means walking > > all page tables to remove the PTEs on alloc and put them back > > in on free. > > > > I believe that Chunsang Jeong from Linaro is planning to > > implement both variants and post them for review, so we can > > decide which one to merge, or even to merge both and make > > it a configuration option. See also > > https://blueprints.launchpad.net/linaro-mm-sig/+spec/engr-mm-dma-mapping-2011.07 > > > > I don't think we need to make merging the CMA patches depending on > > the other patches, it's clear that both need to be solved, and > > they are independent enough. > > I assume from the above that ARM has a hardware page walker? Right. > The way I'd fix this on parisc, because we have a software based TLB, is > to rely on the fact that a page may only be used either for DMA or for > Page Cache, so the aliases should never be interleaved. Since you know > the point at which the page flips from DMA to Cache (and vice versa), > I'd purge the TLB entry and flush the page at that point and rely on the > usage guarantees to ensure that the alias TLB entry doesn't reappear. > This isn't inexpensive but the majority of the cost is the cache flush > which is a requirement to clean the aliases anyway (a TLB entry purge is > pretty cheap). > > Would this work for the ARM hardware walker as well? It would require > you to have a TLB entry purge instruction as well as some architectural > guarantees about not speculating the TLB. The main problem with ARM linear mapping is the fact that it is created using 2MiB sections, so entries for kernel linear mapping fits entirely in first lever of process page table. This implies that direct changing this linear mapping is not easy task and must be performed for all tasks in the system. In my CMA v12+ patches I decided to use simpler way of solving this issue. I rely on the fact that DMA memory is allocated only from CMA regions, so during early boot I change the kernel linear
[PATCH 6/8] drivers: add Contiguous Memory Allocator
The Contiguous Memory Allocator is a set of helper functions for DMA mapping framework that improves allocations of contiguous memory chunks. CMA grabs memory on system boot, marks it with CMA_MIGRATE_TYPE and gives back to the system. Kernel is allowed to allocate movable pages within CMA's managed memory so that it can be used for example for page cache when DMA mapping do not use it. On dma_alloc_from_contiguous() request such pages are migrated out of CMA area to free required contiguous block and fulfill the request. This allows to allocate large contiguous chunks of memory at any time assuming that there is enough free memory available in the system. This code is heavily based on earlier works by Michal Nazarewicz. Signed-off-by: Marek Szyprowski Signed-off-by: Kyungmin Park CC: Michal Nazarewicz --- arch/Kconfig |3 + drivers/base/Kconfig | 79 drivers/base/Makefile |1 + drivers/base/dma-contiguous.c | 386 include/linux/dma-contiguous.h | 102 +++ 5 files changed, 571 insertions(+), 0 deletions(-) create mode 100644 drivers/base/dma-contiguous.c create mode 100644 include/linux/dma-contiguous.h diff --git a/arch/Kconfig b/arch/Kconfig index 4b0669c..a3b39a2 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -124,6 +124,9 @@ config HAVE_ARCH_TRACEHOOK config HAVE_DMA_ATTRS bool +config HAVE_DMA_CONTIGUOUS + bool + config USE_GENERIC_SMP_HELPERS bool diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig index 21cf46f..bffe7fb 100644 --- a/drivers/base/Kconfig +++ b/drivers/base/Kconfig @@ -174,4 +174,83 @@ config SYS_HYPERVISOR source "drivers/base/regmap/Kconfig" +config CMA + bool "Contiguous Memory Allocator" + depends on HAVE_DMA_CONTIGUOUS && HAVE_MEMBLOCK + select MIGRATION + select CMA_MIGRATE_TYPE + help + This enables the Contiguous Memory Allocator which allows drivers + to allocate big physically-contiguous blocks of memory for use with + hardware components that do not support I/O map nor scatter-gather. + + For more information see . + If unsure, say "n". + +if CMA + +config CMA_DEBUG + bool "CMA debug messages (DEVELOPEMENT)" + help + Turns on debug messages in CMA. This produces KERN_DEBUG + messages for every CMA call as well as various messages while + processing calls such as dma_alloc_from_contiguous(). + This option does not affect warning and error messages. + +comment "Default contiguous memory area size:" + +config CMA_SIZE_ABSOLUTE + int "Absolute size (in MiB)" + depends on !CMA_SIZE_SEL_PERCENTAGE + default 16 + help + Defines the size (in MiB) of the default memory area for Contiguous + Memory Allocator. + +config CMA_SIZE_PERCENTAGE + int "Percentage of total memory" + depends on !CMA_SIZE_SEL_ABSOLUTE + default 10 + help + Defines the size of the default memory area for Contiguous Memory + Allocator as a percentage of the total memory in the system. + +choice + prompt "Selected region size" + default CMA_SIZE_SEL_ABSOLUTE + +config CMA_SIZE_SEL_ABSOLUTE + bool "Use absolute value only" + +config CMA_SIZE_SEL_PERCENTAGE + bool "Use percentage value only" + +config CMA_SIZE_SEL_MIN + bool "Use lower value (minimum)" + +config CMA_SIZE_SEL_MAX + bool "Use higher value (maximum)" + +endchoice + +config CMA_ALIGNMENT + int "Maximum PAGE_SIZE order of alignment for contiguous buffers" + range 4 9 + default 8 + help + DMA mapping framework by default aligns all buffers to the smallest + PAGE_SIZE order which is greater than or equal to the requested buffer + size. This works well for buffers up to a few hundreds kilobytes, but + for larger buffers it just a memory waste. With this parameter you can + specify the maximum PAGE_SIZE order for contiguous buffers. Larger + buffers will be aligned only to this specified order. The order is + expressed as a power of two multiplied by the PAGE_SIZE. + + For example, if your system defaults to 4KiB pages, the order value + of 8 means that the buffers will be aligned up to 1MiB only. + + If unsure, leave the default value "8". + +endif + endmenu diff --git a/drivers/base/Makefile b/drivers/base/Makefile index 99a375a..794546f 100644 --- a/drivers/base/Makefile +++ b/drivers/base/Makefile @@ -5,6 +5,7 @@ obj-y := core.o sys.o bus.o dd.o syscore.o \ cpu.o firmware.o init.o map.o devres.o \ attribute_container.o transport_class.o obj-$(CONFIG_DEVTMPFS) += devtmpfs.o +obj-$(CONFIG_CMA) += dma-contiguous.o obj-y += power/ obj-$(CONFIG_HAS_DMA) += dma-mapping.o obj-$(CONFIG
Re: [PATCH 6/8] drivers: add Contiguous Memory Allocator
[cc to ks-discuss added, since this may be a relevant topic] On Tue, 2011-07-05 at 14:27 +0200, Arnd Bergmann wrote: > On Tuesday 05 July 2011, Russell King - ARM Linux wrote: > > On Tue, Jul 05, 2011 at 09:41:48AM +0200, Marek Szyprowski wrote: > > > The Contiguous Memory Allocator is a set of helper functions for DMA > > > mapping framework that improves allocations of contiguous memory chunks. > > > > > > CMA grabs memory on system boot, marks it with CMA_MIGRATE_TYPE and > > > gives back to the system. Kernel is allowed to allocate movable pages > > > within CMA's managed memory so that it can be used for example for page > > > cache when DMA mapping do not use it. On dma_alloc_from_contiguous() > > > request such pages are migrated out of CMA area to free required > > > contiguous block and fulfill the request. This allows to allocate large > > > contiguous chunks of memory at any time assuming that there is enough > > > free memory available in the system. > > > > > > This code is heavily based on earlier works by Michal Nazarewicz. > > > > And how are you addressing the technical concerns about aliasing of > > cache attributes which I keep bringing up with this and you keep > > ignoring and telling me that I'm standing in your way. Just to chime in here, parisc has an identical issue. If the CPU ever sees an alias with different attributes for the same page, it will HPMC the box (that's basically the bios will kill the system as being architecturally inconsistent), so an architecture neutral solution on this point is essential to us as well. > This is of course an important issue, and it's the one item listed as > TODO in the introductory mail that sent. > > It's also a preexisting problem as far as I can tell, and it needs > to be solved in __dma_alloc for both cases, dma_alloc_from_contiguous > and __alloc_system_pages as introduced in patch 7. > > We've discussed this back and forth, and it always comes down to > one of two ugly solutions: > > 1. Put all of the MIGRATE_CMA and pages into highmem and change > __alloc_system_pages so it also allocates only from highmem pages. > The consequences of this are that we always need to build kernels > with highmem enabled and that we have less lowmem on systems that > are already small, both of which can be fairly expensive unless > you have lots of highmem already. So this would require that systems using the API have a highmem? (parisc doesn't today). > 2. Add logic to unmap pages from the linear mapping, which is > very expensive because it forces the use of small pages in the > linear mapping (or in parts of it), and possibly means walking > all page tables to remove the PTEs on alloc and put them back > in on free. > > I believe that Chunsang Jeong from Linaro is planning to > implement both variants and post them for review, so we can > decide which one to merge, or even to merge both and make > it a configuration option. See also > https://blueprints.launchpad.net/linaro-mm-sig/+spec/engr-mm-dma-mapping-2011.07 > > I don't think we need to make merging the CMA patches depending on > the other patches, it's clear that both need to be solved, and > they are independent enough. I assume from the above that ARM has a hardware page walker? The way I'd fix this on parisc, because we have a software based TLB, is to rely on the fact that a page may only be used either for DMA or for Page Cache, so the aliases should never be interleaved. Since you know the point at which the page flips from DMA to Cache (and vice versa), I'd purge the TLB entry and flush the page at that point and rely on the usage guarantees to ensure that the alias TLB entry doesn't reappear. This isn't inexpensive but the majority of the cost is the cache flush which is a requirement to clean the aliases anyway (a TLB entry purge is pretty cheap). Would this work for the ARM hardware walker as well? It would require you to have a TLB entry purge instruction as well as some architectural guarantees about not speculating the TLB. James -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 6/8] drivers: add Contiguous Memory Allocator
The Contiguous Memory Allocator is a set of helper functions for DMA mapping framework that improves allocations of contiguous memory chunks. CMA grabs memory on system boot, marks it with CMA_MIGRATE_TYPE and gives back to the system. Kernel is allowed to allocate movable pages within CMA's managed memory so that it can be used for example for page cache when DMA mapping do not use it. On dma_alloc_from_contiguous() request such pages are migrated out of CMA area to free required contiguous block and fulfill the request. This allows to allocate large contiguous chunks of memory at any time assuming that there is enough free memory available in the system. This code is heavily based on earlier works by Michal Nazarewicz. Signed-off-by: Marek Szyprowski Signed-off-by: Kyungmin Park CC: Michal Nazarewicz --- arch/Kconfig |3 + drivers/base/Kconfig | 77 drivers/base/Makefile |1 + drivers/base/dma-contiguous.c | 396 include/linux/dma-contiguous.h | 102 ++ 5 files changed, 579 insertions(+), 0 deletions(-) create mode 100644 drivers/base/dma-contiguous.c create mode 100644 include/linux/dma-contiguous.h diff --git a/arch/Kconfig b/arch/Kconfig index 26b0e23..228d761 100644 --- a/arch/Kconfig +++ b/arch/Kconfig @@ -124,6 +124,9 @@ config HAVE_ARCH_TRACEHOOK config HAVE_DMA_ATTRS bool +config HAVE_DMA_CONTIGUOUS + bool + config USE_GENERIC_SMP_HELPERS bool diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig index d57e8d0..c690d05 100644 --- a/drivers/base/Kconfig +++ b/drivers/base/Kconfig @@ -168,4 +168,81 @@ config SYS_HYPERVISOR bool default n +config CMA + bool "Contiguous Memory Allocator" + depends on HAVE_DMA_CONTIGUOUS && HAVE_MEMBLOCK + select MIGRATION + select CMA_MIGRATE_TYPE + help + This enables the Contiguous Memory Allocator which allows drivers + to allocate big physically-contiguous blocks of memory for use with + hardware components that do not support I/O map nor scatter-gather. + + For more information see . + If unsure, say "n". + +if CMA + +config CMA_DEBUG + bool "CMA debug messages (DEVELOPEMENT)" + help + Turns on debug messages in CMA. This produces KERN_DEBUG + messages for every CMA call as well as various messages while + processing calls such as dma_alloc_from_contiguous(). + This option does not affect warning and error messages. + +comment "Default contiguous memory area size:" + +config CMA_SIZE_ABSOLUTE + int "Absolute size (in MiB)" + default 16 + help + Defines the size (in MiB) of the default memory area for Contiguous + Memory Allocator. + +config CMA_SIZE_PERCENTAGE + int "Percentage of total memory" + default 10 + help + Defines the size of the default memory area for Contiguous Memory + Allocator as a percentage of the total memory in the system. + +choice + prompt "Selected region size" + default CMA_SIZE_SEL_ABSOLUTE + +config CMA_SIZE_SEL_ABSOLUTE + bool "Use absolute value only" + +config CMA_SIZE_SEL_PERCENTAGE + bool "Use percentage value only" + +config CMA_SIZE_SEL_MIN + bool "Use lower value (minimum)" + +config CMA_SIZE_SEL_MAX + bool "Use higher value (maximum)" + +endchoice + +config CMA_ALIGNMENT + int "Maximum PAGE_SIZE order of alignment for contiguous buffers" + range 4 9 + default 8 + help + DMA mapping framework by default aligns all buffers to the smallest + PAGE_SIZE order which is greater than or equal to the requested buffer + size. This works well for buffers up to a few hundreds kilobytes, but + for larger buffers it just a memory waste. With this parameter you can + specify the maximum PAGE_SIZE order for contiguous buffers. Larger + buffers will be aligned only to this specified order. The order is + expressed as a power of two multiplied by the PAGE_SIZE. + + For example, if your system defaults to 4KiB pages, the order value + of 8 means that the buffers will be aligned up to 1MiB only. + + If unsure, leave the default value "8". + +endif + endmenu diff --git a/drivers/base/Makefile b/drivers/base/Makefile index 4c5701c..be6aab4 100644 --- a/drivers/base/Makefile +++ b/drivers/base/Makefile @@ -5,6 +5,7 @@ obj-y := core.o sys.o bus.o dd.o syscore.o \ cpu.o firmware.o init.o map.o devres.o \ attribute_container.o transport_class.o obj-$(CONFIG_DEVTMPFS) += devtmpfs.o +obj-$(CONFIG_CMA) += dma-contiguous.o obj-y += power/ obj-$(CONFIG_HAS_DMA) += dma-mapping.o obj-$(CONFIG_HAVE_GENERIC_DMA_COHERENT) += dma-coherent.o diff --git a/drivers/base/dma-contiguous.c b/drive
RE: [PATCH 6/8] drivers: add Contiguous Memory Allocator
Hello, I've just found two nasty bugs in this version of CMA. Sadly, both are the results of posting the patches in a big hurry. I'm really sorry. Alignment argument was not passed correctly to the bitmap_find_next_zero_area() function and there was an ugly bug in the dma_release_from_contiguous() function. On Tuesday, July 05, 2011 9:42 AM Marek Szyprowski wrote: > The Contiguous Memory Allocator is a set of helper functions for DMA > mapping framework that improves allocations of contiguous memory chunks. > > CMA grabs memory on system boot, marks it with CMA_MIGRATE_TYPE and > gives back to the system. Kernel is allowed to allocate movable pages > within CMA's managed memory so that it can be used for example for page > cache when DMA mapping do not use it. On dma_alloc_from_contiguous() > request such pages are migrated out of CMA area to free required > contiguous block and fulfill the request. This allows to allocate large > contiguous chunks of memory at any time assuming that there is enough > free memory available in the system. > > This code is heavily based on earlier works by Michal Nazarewicz. > > Signed-off-by: Marek Szyprowski > Signed-off-by: Kyungmin Park > CC: Michal Nazarewicz > --- > drivers/base/Kconfig | 77 + > drivers/base/Makefile |1 + > drivers/base/dma-contiguous.c | 367 > > include/linux/dma-contiguous.h | 104 +++ > 4 files changed, 549 insertions(+), 0 deletions(-) > create mode 100644 drivers/base/dma-contiguous.c > create mode 100644 include/linux/dma-contiguous.h > > diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig > index d57e8d0..95ae1a7 100644 > --- a/drivers/base/Kconfig > +++ b/drivers/base/Kconfig > @@ -168,4 +168,81 @@ config SYS_HYPERVISOR > bool > default n > > +config CMA > + bool "Contiguous Memory Allocator" > + depends HAVE_DMA_CONTIGUOUS && HAVE_MEMBLOCK > + select MIGRATION > + select CMA_MIGRATE_TYPE > + help > + This enables the Contiguous Memory Allocator which allows drivers > + to allocate big physically-contiguous blocks of memory for use with > + hardware components that do not support I/O map nor scatter-gather. > + > + For more information see . > + If unsure, say "n". > + > +if CMA > + > +config CMA_DEBUG > + bool "CMA debug messages (DEVELOPEMENT)" > + help > + Turns on debug messages in CMA. This produces KERN_DEBUG > + messages for every CMA call as well as various messages while > + processing calls such as dma_alloc_from_contiguous(). > + This option does not affect warning and error messages. > + > +comment "Default contiguous memory area size:" > + > +config CMA_SIZE_ABSOLUTE > + int "Absolute size (in MiB)" > + default 16 > + help > + Defines the size (in MiB) of the default memory area for Contiguous > + Memory Allocator. > + > +config CMA_SIZE_PERCENTAGE > + int "Percentage of total memory" > + default 10 > + help > + Defines the size of the default memory area for Contiguous Memory > + Allocator as a percentage of the total memory in the system. > + > +choice > + prompt "Selected region size" > + default CMA_SIZE_SEL_ABSOLUTE > + > +config CMA_SIZE_SEL_ABSOLUTE > + bool "Use absolute value only" > + > +config CMA_SIZE_SEL_PERCENTAGE > + bool "Use percentage value only" > + > +config CMA_SIZE_SEL_MIN > + bool "Use lower value (minimum)" > + > +config CMA_SIZE_SEL_MAX > + bool "Use higher value (maximum)" > + > +endchoice > + > +config CMA_ALIGNMENT > + int "Maximum PAGE_SIZE order of alignment for contiguous buffers" > + range 4 9 > + default 8 > + help > + DMA mapping framework by default aligns all buffers to the smallest > + PAGE_SIZE order which is greater than or equal to the requested > buffer > + size. This works well for buffers up to a few hundreds kilobytes, > but > + for larger buffers it just a memory waste. With this parameter you > can > + specify the maximum PAGE_SIZE order for contiguous buffers. Larger > + buffers will be aligned only to this specified order. The order is > + expressed as a power of two multiplied by the PAGE_SIZE. > + > + For example, if your system defaults to 4KiB pages, the order value > + of 8 means that the buffers will be aligned up to 1MiB only. > + > + If unsure, leave the default value "8". > + > +endif > + > endmenu > diff --git a/drivers/base/Makefile b/drivers/base/Makefile > index 4c5701c..be6aab4 100644 > --- a/drivers/base/Makefile > +++ b/drivers/base/Makefile > @@ -5,6 +5,7 @@ obj-y := core.o sys.o bus.o dd.o syscore.o \ > cpu.o firmware.o init.o map.o devres.o \ > attribute_container.o transport_class.o > obj-$(CONFIG_DEVTMPFS) += devtmpfs.o > +obj-$(CONFIG_CMA) +=
Re: [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Friday 08 July 2011, Russell King - ARM Linux wrote: > On Tue, Jul 05, 2011 at 03:58:39PM +0200, Arnd Bergmann wrote: > > > If I'm reading your "ARM: DMA: steal memory for DMA coherent mappings" > > correctly, the idea is to have a per-platform compile-time amount > > of memory that is reserved purely for coherent allocations and > > taking out of the buddy allocator, right? > > Yes, because every time I've looked at taking out memory mappings in > the first level page tables, it's always been a major issue. > > We have a method where we can remove first level mappings on > uniprocessor systems in the ioremap code just fine - we use that so > that systems can setup section and supersection mappings. They can > tear them down as well - and we update other tasks L1 page tables > when they get switched in. > > This, however, doesn't work on SMP, because if you have a DMA allocation > (which is permitted from IRQ context) you must have some way of removing > the L1 page table entries from all CPUs TLBs and the page tables currently > in use and any future page tables which those CPUs may switch to. Ah, interesting. So there is no tlb flush broadcast operation and it always goes through IPI? > So, in a SMP system, there is no safe way to remove L1 page table entries > from IRQ context. That means if memory is mapped for the buddy allocators > using L1 page table entries, then it is fixed for that application on a > SMP system. Ok. Can we limit GFP_ATOMIC to memory that doesn't need to be remapped then? I guess we can assume that there is no regression if we just skip the dma_alloc_contiguous step in dma_alloc_coherent for any atomic callers and immediately fall back to the regular allocator. Unfortunately, this still means we have to keep both methods. I was hoping that with CMA doing dynamic remapping there would be no need for keeping a significant number of pages reserved for this. Arnd -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [Linaro-mm-sig] [PATCH 6/8] drivers: add Contiguous Memory Allocator
Hello, On Monday, July 11, 2011 9:01 PM Janusz Krzysztofik wrote: > Dnia poniedziałek, 11 lipca 2011 o 15:47:32 Marek Szyprowski napisał(a): > > Hello, > > > > On Saturday, July 09, 2011 4:57 PM Janusz Krzysztofik wrote: > > > On Wed, 6 Jul 2011 at 16:59:45 Arnd Bergmann wrote: > > > > On Wednesday 06 July 2011, Nicolas Pitre wrote: > > > > > On Wed, 6 Jul 2011, Russell King - ARM Linux wrote: > > > > > > Another issue is that when a platform has restricted DMA > > > > > > regions, they typically don't fall into the highmem zone. > > > > > > As the dmabounce code allocates from the DMA coherent > > > > > > allocator to provide it with guaranteed DMA-able memory, > > > > > > that would be rather inconvenient. > > > > > > > > > > Do we encounter this in practice i.e. do those platforms > > > > > requiring large contiguous allocations motivating this work > > > > > have such DMA restrictions? > > > > > > > > You can probably find one or two of those, but we don't have to > > > > optimize for that case. I would at least expect the maximum size > > > > of the allocation to be smaller than the DMA limit for these, > > > > and consequently mandate that they define a sufficiently large > > > > CONSISTENT_DMA_SIZE for the crazy devices, or possibly add a > > > > hack to unmap some low memory and call > > > > dma_declare_coherent_memory() for the device. > > > > > > Once found that Russell has dropped his "ARM: DMA: steal memory for > > > DMA coherent mappings" for now, let me get back to this idea of a > > > hack that would allow for safely calling > > > dma_declare_coherent_memory() in order to assign a device with a > > > block of contiguous memory for exclusive use. > > > > We tested such approach and finally with 3.0-rc1 it works fine. You > > can find an example for dma_declare_coherent() together with > > required memblock_remove() calls in the following patch series: > > http://www.spinics.net/lists/linux-samsung-soc/msg05026.html > > "[PATCH 0/3 v2] ARM: S5P: Add support for MFC device on S5PV210 and > > EXYNOS4" > > > > > Assuming there should be no problem with successfully allocating a > > > large continuous block of coherent memory at boot time with > > > dma_alloc_coherent(), this block could be reserved for the device. > > > The only problem is with the dma_declare_coherent_memory() calling > > > ioremap(), which was designed with a device's dedicated physical > > > memory in mind, but shouldn't be called on a memory already > > > mapped. > > > > All these issues with ioremap has been finally resolved in 3.0-rc1. > > Like Russell pointed me in > > http://www.spinics.net/lists/arm-kernel/msg127644.html, ioremap can > > be fixed to work on early reserved memory areas by selecting > > ARCH_HAS_HOLES_MEMORYMODEL Kconfig option. > > I'm not sure. Recently I tried to refresh my now 7 months old patch in > which I used that 'memblock_remove() then dma_declare_coherent_memery()' > method[1]. It was different from your S5P MFC example in that it didn't > punch any holes in the system memory, only stole a block of SDRAM from > its tail. But Russell reminded me again: "we should not be mapping SDRAM > using device mappings."[2]. Would defining ARCH_HAS_HOLES_MEMORYMODEL > (even if it was justified) make any diference in my case? I don't think > so. Defining ARCH_HAS_HOLES_MEMORYMODEL changes the behavior of valid_pfn() macro/function, which is used in the ioremap(). When defined, valid_pfn() checks if the selected pfn is inside system memory or not (using memblock information). If the area is removed with memblock_remove(), then a check with valid_pfn() fails and ioremap() doesn't complain about mapping system memory. > Wnat I think, after Russell, is that we still need that obligatory > ioremap() removed from dma_declare_coherent_memory(), or made it > optional, or a separate dma_declare_coherent_memory()-like function > without (obligatory) ioremap() provided by the DMA API, in order to get > the dma_declare_coherent_memery() method being accepted without any > reservations when used inside arch/arm, I'm afraid. Please check again with 3.0-rc1. ARCH_HAS_HOLES_MEMORYMODEL solution was suggested by Russell. It looks like this is the correct solution for this problem, because I don't believe that ioremap() will be removed from dma_declare_coherent() anytime soon. Best regards -- Marek Szyprowski Samsung Poland R&D Center -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Linaro-mm-sig] [PATCH 6/8] drivers: add Contiguous Memory Allocator
Dnia poniedziałek, 11 lipca 2011 o 15:47:32 Marek Szyprowski napisał(a): > Hello, > > On Saturday, July 09, 2011 4:57 PM Janusz Krzysztofik wrote: > > On Wed, 6 Jul 2011 at 16:59:45 Arnd Bergmann wrote: > > > On Wednesday 06 July 2011, Nicolas Pitre wrote: > > > > On Wed, 6 Jul 2011, Russell King - ARM Linux wrote: > > > > > Another issue is that when a platform has restricted DMA > > > > > regions, they typically don't fall into the highmem zone. > > > > > As the dmabounce code allocates from the DMA coherent > > > > > allocator to provide it with guaranteed DMA-able memory, > > > > > that would be rather inconvenient. > > > > > > > > Do we encounter this in practice i.e. do those platforms > > > > requiring large contiguous allocations motivating this work > > > > have such DMA restrictions? > > > > > > You can probably find one or two of those, but we don't have to > > > optimize for that case. I would at least expect the maximum size > > > of the allocation to be smaller than the DMA limit for these, > > > and consequently mandate that they define a sufficiently large > > > CONSISTENT_DMA_SIZE for the crazy devices, or possibly add a > > > hack to unmap some low memory and call > > > dma_declare_coherent_memory() for the device. > > > > Once found that Russell has dropped his "ARM: DMA: steal memory for > > DMA coherent mappings" for now, let me get back to this idea of a > > hack that would allow for safely calling > > dma_declare_coherent_memory() in order to assign a device with a > > block of contiguous memory for exclusive use. > > We tested such approach and finally with 3.0-rc1 it works fine. You > can find an example for dma_declare_coherent() together with > required memblock_remove() calls in the following patch series: > http://www.spinics.net/lists/linux-samsung-soc/msg05026.html > "[PATCH 0/3 v2] ARM: S5P: Add support for MFC device on S5PV210 and > EXYNOS4" > > > Assuming there should be no problem with successfully allocating a > > large continuous block of coherent memory at boot time with > > dma_alloc_coherent(), this block could be reserved for the device. > > The only problem is with the dma_declare_coherent_memory() calling > > ioremap(), which was designed with a device's dedicated physical > > memory in mind, but shouldn't be called on a memory already > > mapped. > > All these issues with ioremap has been finally resolved in 3.0-rc1. > Like Russell pointed me in > http://www.spinics.net/lists/arm-kernel/msg127644.html, ioremap can > be fixed to work on early reserved memory areas by selecting > ARCH_HAS_HOLES_MEMORYMODEL Kconfig option. I'm not sure. Recently I tried to refresh my now 7 months old patch in which I used that 'memblock_remove() then dma_declare_coherent_memery()' method[1]. It was different from your S5P MFC example in that it didn't punch any holes in the system memory, only stole a block of SDRAM from its tail. But Russell reminded me again: "we should not be mapping SDRAM using device mappings."[2]. Would defining ARCH_HAS_HOLES_MEMORYMODEL (even if it was justified) make any diference in my case? I don't think so. Wnat I think, after Russell, is that we still need that obligatory ioremap() removed from dma_declare_coherent_memory(), or made it optional, or a separate dma_declare_coherent_memory()-like function without (obligatory) ioremap() provided by the DMA API, in order to get the dma_declare_coherent_memery() method being accepted without any reservations when used inside arch/arm, I'm afraid. Thanks, Janusz [1] http://lists.infradead.org/pipermail/linux-arm-kernel/2010-December/034644.html [2] http://lists.infradead.org/pipermail/linux-arm-kernel/2011-June/052488.html -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [Linaro-mm-sig] [PATCH 6/8] drivers: add Contiguous Memory Allocator
Hello, On Saturday, July 09, 2011 4:57 PM Janusz Krzysztofik wrote: > On Wed, 6 Jul 2011 at 16:59:45 Arnd Bergmann wrote: > > On Wednesday 06 July 2011, Nicolas Pitre wrote: > > > On Wed, 6 Jul 2011, Russell King - ARM Linux wrote: > > > > Another issue is that when a platform has restricted DMA regions, > > > > they typically don't fall into the highmem zone. As the > > > > dmabounce code allocates from the DMA coherent allocator to > > > > provide it with guaranteed DMA-able memory, that would be rather > > > > inconvenient. > > > > > > Do we encounter this in practice i.e. do those platforms requiring > > > large contiguous allocations motivating this work have such DMA > > > restrictions? > > > > You can probably find one or two of those, but we don't have to > > optimize for that case. I would at least expect the maximum size of > > the allocation to be smaller than the DMA limit for these, and > > consequently mandate that they define a sufficiently large > > CONSISTENT_DMA_SIZE for the crazy devices, or possibly add a hack to > > unmap some low memory and call > > dma_declare_coherent_memory() for the device. > > Once found that Russell has dropped his "ARM: DMA: steal memory for DMA > coherent mappings" for now, let me get back to this idea of a hack that > would allow for safely calling dma_declare_coherent_memory() in order to > assign a device with a block of contiguous memory for exclusive use. We tested such approach and finally with 3.0-rc1 it works fine. You can find an example for dma_declare_coherent() together with required memblock_remove() calls in the following patch series: http://www.spinics.net/lists/linux-samsung-soc/msg05026.html "[PATCH 0/3 v2] ARM: S5P: Add support for MFC device on S5PV210 and EXYNOS4" > Assuming there should be no problem with successfully allocating a large > continuous block of coherent memory at boot time with > dma_alloc_coherent(), this block could be reserved for the device. The > only problem is with the dma_declare_coherent_memory() calling > ioremap(), which was designed with a device's dedicated physical memory > in mind, but shouldn't be called on a memory already mapped. All these issues with ioremap has been finally resolved in 3.0-rc1. Like Russell pointed me in http://www.spinics.net/lists/arm-kernel/msg127644.html, ioremap can be fixed to work on early reserved memory areas by selecting ARCH_HAS_HOLES_MEMORYMODEL Kconfig option. > There were three approaches proposed, two of them in August 2010: > http://www.spinics.net/lists/linux-media/msg22179.html, > http://www.spinics.net/lists/arm-kernel/msg96318.html, > and a third one in January 2011: > http://www.spinics.net/lists/linux-arch/msg12637.html. > > As far as I can understand the reason why both of the first two were > NAKed, it was suggested that videobuf-dma-contig shouldn't use coherent > if all it requires is a contiguous memory, and a new API should be > invented, or dma_pool API extended, for providing contiguous memory. This is another story. DMA-mapping framework definitely needs some extensions to allow more detailed specification of the allocated memory (currently we have only coherent and nearly ARM-specific writecombine). During Linaro Memory Management summit we agreed that the dma_alloc_attrs() function might be needed to clean-up the API and provide a nice way of adding new memory parameters. Having a possibility to allocate contiguous cached buffers might be one of the new DMA attributes. Here are some details of my proposal: http://www.spinics.net/lists/linux-mm/msg21235.html > The > CMA was pointed out as a new work in progress contiguous memory API. That was probably the biggest mistake at the beginning. We definitely should have learned dma-mapping framework and its internals. > Now > it turns out it's not, it's only a helper to ensure that > dma_alloc_coherent() always succeeds, and videobuf2-dma-contig is still > going to allocate buffers from coherent memory. I hope that once the dma_alloc_attrs() API will be accepted, I will add support for memory attributes to videobuf2-dma-contig allocator. > (CCing both authors, Marin Mitov and Guennadi Liakhovetski, and their > main opponent, FUJITA Tomonori) > > The third solution was not discussed much after it was pointed out as > being not very different from those two in terms of the above mentioned > rationale. > > All three solutions was different from now suggested method of unmapping > some low memory and then calling dma_declare_coherent_memory() which > ioremaps it in that those tried to reserve some boot time allocated > coherent memory, already mapped correctly, without (io)remapping it. > > If there are still problems with the CMA on one hand, and a need for a > hack to handle "crazy devices" is still seen, regardless of CMA > available and working or not, on the other, maybe we should get back to > the idea of adopting coherent API to new requirements, review those > three proposals again a
Re: [Linaro-mm-sig] [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Wed, 6 Jul 2011 at 16:59:45 Arnd Bergmann wrote: > On Wednesday 06 July 2011, Nicolas Pitre wrote: > > On Wed, 6 Jul 2011, Russell King - ARM Linux wrote: > > > Another issue is that when a platform has restricted DMA regions, > > > they typically don't fall into the highmem zone. As the > > > dmabounce code allocates from the DMA coherent allocator to > > > provide it with guaranteed DMA-able memory, that would be rather > > > inconvenient. > > > > Do we encounter this in practice i.e. do those platforms requiring > > large contiguous allocations motivating this work have such DMA > > restrictions? > > You can probably find one or two of those, but we don't have to > optimize for that case. I would at least expect the maximum size of > the allocation to be smaller than the DMA limit for these, and > consequently mandate that they define a sufficiently large > CONSISTENT_DMA_SIZE for the crazy devices, or possibly add a hack to > unmap some low memory and call > dma_declare_coherent_memory() for the device. Once found that Russell has dropped his "ARM: DMA: steal memory for DMA coherent mappings" for now, let me get back to this idea of a hack that would allow for safely calling dma_declare_coherent_memory() in order to assign a device with a block of contiguous memory for exclusive use. Assuming there should be no problem with successfully allocating a large continuous block of coherent memory at boot time with dma_alloc_coherent(), this block could be reserved for the device. The only problem is with the dma_declare_coherent_memory() calling ioremap(), which was designed with a device's dedicated physical memory in mind, but shouldn't be called on a memory already mapped. There were three approaches proposed, two of them in August 2010: http://www.spinics.net/lists/linux-media/msg22179.html, http://www.spinics.net/lists/arm-kernel/msg96318.html, and a third one in January 2011: http://www.spinics.net/lists/linux-arch/msg12637.html. As far as I can understand the reason why both of the first two were NAKed, it was suggested that videobuf-dma-contig shouldn't use coherent if all it requires is a contiguous memory, and a new API should be invented, or dma_pool API extended, for providing contiguous memory. The CMA was pointed out as a new work in progress contiguous memory API. Now it turns out it's not, it's only a helper to ensure that dma_alloc_coherent() always succeeds, and videobuf2-dma-contig is still going to allocate buffers from coherent memory. (CCing both authors, Marin Mitov and Guennadi Liakhovetski, and their main opponent, FUJITA Tomonori) The third solution was not discussed much after it was pointed out as being not very different from those two in terms of the above mentioned rationale. All three solutions was different from now suggested method of unmapping some low memory and then calling dma_declare_coherent_memory() which ioremaps it in that those tried to reserve some boot time allocated coherent memory, already mapped correctly, without (io)remapping it. If there are still problems with the CMA on one hand, and a need for a hack to handle "crazy devices" is still seen, regardless of CMA available and working or not, on the other, maybe we should get back to the idea of adopting coherent API to new requirements, review those three proposals again and select one which seems most acceptable to everyone? Being a submitter of the third, I'll be happy to refresh it if selected. Thanks, Janusz -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Tue, Jul 05, 2011 at 03:58:39PM +0200, Arnd Bergmann wrote: > Ah, sorry I missed that patch on the mailing list, found it now in > your for-next branch. I've been searching for this email to reply to for the last day or so... > If I'm reading your "ARM: DMA: steal memory for DMA coherent mappings" > correctly, the idea is to have a per-platform compile-time amount > of memory that is reserved purely for coherent allocations and > taking out of the buddy allocator, right? Yes, because every time I've looked at taking out memory mappings in the first level page tables, it's always been a major issue. We have a method where we can remove first level mappings on uniprocessor systems in the ioremap code just fine - we use that so that systems can setup section and supersection mappings. They can tear them down as well - and we update other tasks L1 page tables when they get switched in. This, however, doesn't work on SMP, because if you have a DMA allocation (which is permitted from IRQ context) you must have some way of removing the L1 page table entries from all CPUs TLBs and the page tables currently in use and any future page tables which those CPUs may switch to. The easy bit is "future page tables" - that can be done in the same way as the ioremap() code does with a generation number, checked when a new page table is switched in. The problem is the current CPUs, and as we know trying to call smp_call_function() with IRQs disabled is not permitted due to deadlock. So, in a SMP system, there is no safe way to remove L1 page table entries from IRQ context. That means if memory is mapped for the buddy allocators using L1 page table entries, then it is fixed for that application on a SMP system. However, that's not really what I wanted to find this email for. That is I'm dropping the "ARM: DMA: steal memory for DMA coherent mappings" patch for this merge window because - as I found out yesterday - it prevents the Assabet platform booting, and so would be a regression. Plus, I have a report of a regression with the streaming DMA API speculative prefetch fixes causing the IOP ADMA raid5 async offload stuff to explode - which may result in the streaming DMA API fixes being reverted (which will leave ARMv6+ vulnerable to data corruption.) As I have no time to work through the RAID5 code, async_tx code, and IOP ADMA code to get to the bottom of it (because of this flood of patches) I think a revert is looking likely - either that or I'll have to tell the bug reporter to go away, which really isn't on. It's on LKML if anyone's interested in trying to diagnose it, the "PROBLEM: ARM-dma-mapping-fix-for-speculative-prefetching cause OOPS" thread. -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Linaro-mm-sig] [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Wed, 6 Jul 2011, Arnd Bergmann wrote: > On Wednesday 06 July 2011 21:10:07 Nicolas Pitre wrote: > > If you get a highmem page, because the cache is VIPT, that page might > > still be cached even if it wasn't mapped. With a VIVT cache we must > > flush the cache whenever a highmem page is unmapped. There is no such > > restriction with VIPT i.e. ARMv6 and above. Therefore to make sure the > > highmem page you get doesn't have cache lines associated to it, you must > > first map it cacheable, then perform cache invalidation on it, and > > eventually remap it as non-cacheable. This is necessary because there > > is no way to perform cache maintenance on L1 cache using physical > > addresses unfortunately. See commit 7e5a69e83b for an example of what > > this entails (fortunately commit 3e4d3af501 made things much easier and > > therefore commit 39af22a79 greatly simplified things). > > Ok, thanks for the explanation. This definitely makes the highmem approach > much harder to get right, and slower. Let's hope then that Marek's approach > of using small pages for the contiguous memory region and changing their > attributes on the fly works out better than this. I would say that both approaches have fairly equivalent complexity. Nicolas -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Linaro-mm-sig] [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Wednesday 06 July 2011 21:10:07 Nicolas Pitre wrote: > If you get a highmem page, because the cache is VIPT, that page might > still be cached even if it wasn't mapped. With a VIVT cache we must > flush the cache whenever a highmem page is unmapped. There is no such > restriction with VIPT i.e. ARMv6 and above. Therefore to make sure the > highmem page you get doesn't have cache lines associated to it, you must > first map it cacheable, then perform cache invalidation on it, and > eventually remap it as non-cacheable. This is necessary because there > is no way to perform cache maintenance on L1 cache using physical > addresses unfortunately. See commit 7e5a69e83b for an example of what > this entails (fortunately commit 3e4d3af501 made things much easier and > therefore commit 39af22a79 greatly simplified things). Ok, thanks for the explanation. This definitely makes the highmem approach much harder to get right, and slower. Let's hope then that Marek's approach of using small pages for the contiguous memory region and changing their attributes on the fly works out better than this. Arnd -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Wed, 6 Jul 2011, Arnd Bergmann wrote: > On Wednesday 06 July 2011, Russell King - ARM Linux wrote: > > On Wed, Jul 06, 2011 at 04:51:49PM +0200, Arnd Bergmann wrote: > > > On Wednesday 06 July 2011, Russell King - ARM Linux wrote: > > > > > > I don't see how. The pages get allocated from an unmapped area > > > or memory, mapped into the kernel address space as uncached or wc > > > and then cleared. This should be the same for lowmem or highmem > > > pages. > > > > You don't want to clear them via their uncached or WC mapping, but via > > their cached mapping _before_ they get their alternative mapping, and > > flush any cached out of that mapping - both L1 and L2 caches. > > But there can't be any other mapping, which is the whole point of > the exercise to use highmem. > Quoting from the new dma_alloc_area() function: > > c = arm_vmregion_alloc(&area->vm, align, size, > gfp & ~(__GFP_DMA | __GFP_HIGHMEM)); > if (!c) > return NULL; > memset((void *)c->vm_start, 0, size); > > area->vm here points to an uncached location, which means that > we already zero the data through the uncached mapping. I don't > see how it's getting worse than it is already. If you get a highmem page, because the cache is VIPT, that page might still be cached even if it wasn't mapped. With a VIVT cache we must flush the cache whenever a highmem page is unmapped. There is no such restriction with VIPT i.e. ARMv6 and above. Therefore to make sure the highmem page you get doesn't have cache lines associated to it, you must first map it cacheable, then perform cache invalidation on it, and eventually remap it as non-cacheable. This is necessary because there is no way to perform cache maintenance on L1 cache using physical addresses unfortunately. See commit 7e5a69e83b for an example of what this entails (fortunately commit 3e4d3af501 made things much easier and therefore commit 39af22a79 greatly simplified things). Nicolas -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Wed, 6 Jul 2011, Russell King - ARM Linux wrote: > So, ARM is no different from x86, with the exception that the 16MB DMA > zone due to ISA ends up being different sizes on ARM depending on our > restrictions. Sounds good. Thank you. -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Wed, Jul 06, 2011 at 11:19:00AM -0500, Christoph Lameter wrote: > What I described is the basic memory architecture of Linux. I am not that > familiar with ARM and the issue discussed here. Only got involved because > ZONE_DMA was mentioned. The nature of ZONE_DMA is often misunderstood. > > The allocation of the memory banks for the Samsung devices has to fit > somehow into one of these zones. Its probably best to put the memory banks > into ZONE_NORMAL and not have any dependency on ZONE_DMA at all. Let me teach you about the ARM memory management on Linux. Firstly, lets go over the structure of zones in Linux. There are three zones - ZONE_DMA, ZONE_NORMAL and ZONE_HIGHMEM. These zones are filled in that order. So, ZONE_DMA starts at zero. Following on from ZONE_DMA is ZONE_NORMAL memory, and lastly ZONE_HIGHMEM. At boot, we pass all memory over to the kernel as follows: 1. If there is no DMA zone, then we pass all low memory over as ZONE_NORMAL. 2. If there is a DMA zone, by default we pass all low memory as ZONE_DMA. This is required so drivers which use GFP_DMA can work. Platforms with restricted DMA requirements can modify that layout to move memory from ZONE_DMA into ZONE_NORMAL, thereby restricting the upper address which the kernel allocators will give for GFP_DMA allocations. 3. In either case, any high memory as ZONE_HIGHMEM if configured (or memory is truncated if not.) So, when we have (eg) a platform where only the _even_ MBs of memory are DMA-able, we have a 1MB DMA zone at the beginning of system memory, and everything else in ZONE_NORMAL. This means GFP_DMA will return either memory from the first 1MB or fail if it can't. This is the behaviour we desire. Normal allocations will come from ZONE_NORMAL _first_ and then try ZONE_DMA if there's no other alternative. This is the same desired behaviour as x86. So, ARM is no different from x86, with the exception that the 16MB DMA zone due to ISA ends up being different sizes on ARM depending on our restrictions. -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Wed, Jul 06, 2011 at 11:05:00AM -0500, Christoph Lameter wrote: > On Wed, 6 Jul 2011, Russell King - ARM Linux wrote: > > > > > they typically don't fall into the highmem zone. As the dmabounce > > > > code allocates from the DMA coherent allocator to provide it with > > > > guaranteed DMA-able memory, that would be rather inconvenient. > > > > > > True. The dmabounce code would consequently have to allocate > > > the memory through an internal function that avoids the > > > contiguous allocation area and goes straight to ZONE_DMA memory > > > as it does today. > > > > CMA's whole purpose for existing is to provide _dma-able_ contiguous > > memory for things like cameras and such like found on crippled non- > > scatter-gather hardware. If that memory is not DMA-able what's the > > point? > > ZONE_DMA is a zone for memory of legacy (crippled) devices that cannot DMA > into all of memory (and so is ZONE_DMA32). Memory from ZONE_NORMAL can be > used for DMA as well and a fully capable device would be expected to > handle any memory in the system for DMA transfers. > > "guaranteed" dmaable memory? DMA abilities are device specific. Well maybe > you can call ZONE_DMA memory to be guaranteed if you guarantee that any > device must at mininum be able to perform DMA into ZONE_DMA memory. But > there may not be much of that memory around so you would want to limit > the use of that scarce resource. Precisely, which is what ZONE_DMA is all about. I *have* been a Linux kernel hacker for the last 18 years and do know these things, especially as ARM has had various issues with DMA memory limitations over those years - and have successfully had platforms working reliably given that and ZONE_DMA. -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Wednesday 06 July 2011, Russell King - ARM Linux wrote: > On Wed, Jul 06, 2011 at 04:51:49PM +0200, Arnd Bergmann wrote: > > On Wednesday 06 July 2011, Russell King - ARM Linux wrote: > > > > I don't see how. The pages get allocated from an unmapped area > > or memory, mapped into the kernel address space as uncached or wc > > and then cleared. This should be the same for lowmem or highmem > > pages. > > You don't want to clear them via their uncached or WC mapping, but via > their cached mapping _before_ they get their alternative mapping, and > flush any cached out of that mapping - both L1 and L2 caches. But there can't be any other mapping, which is the whole point of the exercise to use highmem. Quoting from the new dma_alloc_area() function: c = arm_vmregion_alloc(&area->vm, align, size, gfp & ~(__GFP_DMA | __GFP_HIGHMEM)); if (!c) return NULL; memset((void *)c->vm_start, 0, size); area->vm here points to an uncached location, which means that we already zero the data through the uncached mapping. I don't see how it's getting worse than it is already. > > > Another issue is that when a platform has restricted DMA regions, > > > they typically don't fall into the highmem zone. As the dmabounce > > > code allocates from the DMA coherent allocator to provide it with > > > guaranteed DMA-able memory, that would be rather inconvenient. > > > > True. The dmabounce code would consequently have to allocate > > the memory through an internal function that avoids the > > contiguous allocation area and goes straight to ZONE_DMA memory > > as it does today. > > CMA's whole purpose for existing is to provide _dma-able_ contiguous > memory for things like cameras and such like found on crippled non- > scatter-gather hardware. If that memory is not DMA-able what's the > point? I mean not any ZONE_DMA memory, but the memory backing coherent_areas[], which is by definition DMA-able from any device and is what is currently being used for the purpose. Arnd -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Wed, 6 Jul 2011, Michal Nazarewicz wrote: > On Wed, 06 Jul 2011 18:05:00 +0200, Christoph Lameter wrote: > > ZONE_DMA is a zone for memory of legacy (crippled) devices that cannot DMA > > into all of memory (and so is ZONE_DMA32). Memory from ZONE_NORMAL > > can be used for DMA as well and a fully capable device would be expected > > to handle any memory in the system for DMA transfers. > > > > "guaranteed" dmaable memory? DMA abilities are device specific. Well maybe > > you can call ZONE_DMA memory to be guaranteed if you guarantee > > that any device must at mininum be able to perform DMA into ZONE_DMA > > memory. But there may not be much of that memory around so you would > > want to limit the use of that scarce resource. > > As pointed in Marek's other mail, this reasoning is not helping in any > way. In case of video codec on various Samsung devices (and from some > other threads this is not limited to Samsung), the codec needs separate > buffers in separate memory banks. What I described is the basic memory architecture of Linux. I am not that familiar with ARM and the issue discussed here. Only got involved because ZONE_DMA was mentioned. The nature of ZONE_DMA is often misunderstood. The allocation of the memory banks for the Samsung devices has to fit somehow into one of these zones. Its probably best to put the memory banks into ZONE_NORMAL and not have any dependency on ZONE_DMA at all. -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Wed, 6 Jul 2011, Russell King - ARM Linux wrote: > > > they typically don't fall into the highmem zone. As the dmabounce > > > code allocates from the DMA coherent allocator to provide it with > > > guaranteed DMA-able memory, that would be rather inconvenient. > > > > True. The dmabounce code would consequently have to allocate > > the memory through an internal function that avoids the > > contiguous allocation area and goes straight to ZONE_DMA memory > > as it does today. > > CMA's whole purpose for existing is to provide _dma-able_ contiguous > memory for things like cameras and such like found on crippled non- > scatter-gather hardware. If that memory is not DMA-able what's the > point? ZONE_DMA is a zone for memory of legacy (crippled) devices that cannot DMA into all of memory (and so is ZONE_DMA32). Memory from ZONE_NORMAL can be used for DMA as well and a fully capable device would be expected to handle any memory in the system for DMA transfers. "guaranteed" dmaable memory? DMA abilities are device specific. Well maybe you can call ZONE_DMA memory to be guaranteed if you guarantee that any device must at mininum be able to perform DMA into ZONE_DMA memory. But there may not be much of that memory around so you would want to limit the use of that scarce resource. -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Wed, 06 Jul 2011 18:05:00 +0200, Christoph Lameter wrote: ZONE_DMA is a zone for memory of legacy (crippled) devices that cannot DMA into all of memory (and so is ZONE_DMA32). Memory from ZONE_NORMAL can be used for DMA as well and a fully capable device would be expected to handle any memory in the system for DMA transfers. "guaranteed" dmaable memory? DMA abilities are device specific. Well maybe you can call ZONE_DMA memory to be guaranteed if you guarantee that any device must at mininum be able to perform DMA into ZONE_DMA memory. But there may not be much of that memory around so you would want to limit the use of that scarce resource. As pointed in Marek's other mail, this reasoning is not helping in any way. In case of video codec on various Samsung devices (and from some other threads this is not limited to Samsung), the codec needs separate buffers in separate memory banks. -- Best regards, _ _ .o. | Liege of Serenely Enlightened Majesty of o' \,=./ `o ..o | Computer Science, Michal "mina86" Nazarewicz(o o) ooo +--ooO--(_)--Ooo-- -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Wed, Jul 06, 2011 at 04:51:49PM +0200, Arnd Bergmann wrote: > On Wednesday 06 July 2011, Russell King - ARM Linux wrote: > > On Wed, Jul 06, 2011 at 04:09:29PM +0200, Arnd Bergmann wrote: > > > Maybe you can simply adapt the default location of the contiguous memory > > > are like this: > > > - make CONFIG_CMA depend on CONFIG_HIGHMEM on ARM, at compile time > > > - if ZONE_HIGHMEM exist during boot, put the CMA area in there > > > - otherwise, put the CMA area at the top end of lowmem, and change > > > the zone sizes so ZONE_HIGHMEM stretches over all of the CMA memory. > > > > One of the requirements of the allocator is that the returned memory > > should be zero'd (because it can be exposed to userspace via ALSA > > and frame buffers.) > > > > Zeroing the memory from all the contexts which dma_alloc_coherent > > is called from is a trivial matter if its in lowmem, but highmem is > > harder. > > I don't see how. The pages get allocated from an unmapped area > or memory, mapped into the kernel address space as uncached or wc > and then cleared. This should be the same for lowmem or highmem > pages. You don't want to clear them via their uncached or WC mapping, but via their cached mapping _before_ they get their alternative mapping, and flush any cached out of that mapping - both L1 and L2 caches. For lowmem pages, that's easy. For highmem pages, they need to be individually kmap'd to zero them etc. (alloc_pages() warns on GFP_HIGHMEM + GFP_ZERO from atomic contexts - and dma_alloc_coherent must be callable from such contexts.) That may be easier now that we don't have the explicit indicies for kmap_atomics, but at that time it wasn't easily possible. > > Another issue is that when a platform has restricted DMA regions, > > they typically don't fall into the highmem zone. As the dmabounce > > code allocates from the DMA coherent allocator to provide it with > > guaranteed DMA-able memory, that would be rather inconvenient. > > True. The dmabounce code would consequently have to allocate > the memory through an internal function that avoids the > contiguous allocation area and goes straight to ZONE_DMA memory > as it does today. CMA's whole purpose for existing is to provide _dma-able_ contiguous memory for things like cameras and such like found on crippled non- scatter-gather hardware. If that memory is not DMA-able what's the point? -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 6/8] drivers: add Contiguous Memory Allocator
Hello, On Wednesday, July 06, 2011 5:37 PM Russell King - ARM Linux wrote: > On Wed, Jul 06, 2011 at 04:56:23PM +0200, Marek Szyprowski wrote: > > This will not solve our problems. We need CMA also to create at least one > > device private area that for sure will be in low memory (video codec). > > You make these statements but you don't say why. Can you please > explain why the video codec needs low memory - does it have a > restricted number of memory address bits which it can manipulate? Nope, it only needs to put some type of memory buffers in first bank (effectively in 3000-34ff area) and the others in the second bank (4000-57ff area). The values are given for Samsung GONI board. Best regards -- Marek Szyprowski Samsung Poland R&D Center -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Wed, Jul 06, 2011 at 04:56:23PM +0200, Marek Szyprowski wrote: > This will not solve our problems. We need CMA also to create at least one > device private area that for sure will be in low memory (video codec). You make these statements but you don't say why. Can you please explain why the video codec needs low memory - does it have a restricted number of memory address bits which it can manipulate? -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Linaro-mm-sig] [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Wednesday 06 July 2011, Nicolas Pitre wrote: > On Wed, 6 Jul 2011, Russell King - ARM Linux wrote: > > > Another issue is that when a platform has restricted DMA regions, > > they typically don't fall into the highmem zone. As the dmabounce > > code allocates from the DMA coherent allocator to provide it with > > guaranteed DMA-able memory, that would be rather inconvenient. > > Do we encounter this in practice i.e. do those platforms requiring large > contiguous allocations motivating this work have such DMA restrictions? You can probably find one or two of those, but we don't have to optimize for that case. I would at least expect the maximum size of the allocation to be smaller than the DMA limit for these, and consequently mandate that they define a sufficiently large CONSISTENT_DMA_SIZE for the crazy devices, or possibly add a hack to unmap some low memory and call dma_declare_coherent_memory() for the device. Arnd -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 6/8] drivers: add Contiguous Memory Allocator
Hello, On Wednesday, July 06, 2011 4:09 PM Arnd Bergmann wrote: > On Wednesday 06 July 2011, Marek Szyprowski wrote: > > The only problem that might need to be resolved is GFP_ATOMIC allocation > > (updating page properties probably requires some locking), but it can be > > served from a special area which is created on boot without low-memory > > mapping at all. None sane driver will call dma_alloc_coherent(GFP_ATOMIC) > > for large buffers anyway. > > Would it be easier to start with a version that only allocated from memory > without a low-memory mapping at first? > > This would be similar to the approach that Russell's fix for the regular > dma_alloc_coherent has taken, except that you need to also allow the memory > to be used as highmem user pages. > > Maybe you can simply adapt the default location of the contiguous memory > are like this: > - make CONFIG_CMA depend on CONFIG_HIGHMEM on ARM, at compile time > - if ZONE_HIGHMEM exist during boot, put the CMA area in there > - otherwise, put the CMA area at the top end of lowmem, and change > the zone sizes so ZONE_HIGHMEM stretches over all of the CMA memory. This will not solve our problems. We need CMA also to create at least one device private area that for sure will be in low memory (video codec). I will rewrite ARM dma-mapping & CMA integration patch basing on the latest ARM for-next patches and add proof-of-concept of the solution presented in my previous mail (2-level page tables and unmapping pages from low-mem). Best regards -- Marek Szyprowski Samsung Poland R&D Center -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Wednesday 06 July 2011, Russell King - ARM Linux wrote: > On Wed, Jul 06, 2011 at 04:09:29PM +0200, Arnd Bergmann wrote: > > Maybe you can simply adapt the default location of the contiguous memory > > are like this: > > - make CONFIG_CMA depend on CONFIG_HIGHMEM on ARM, at compile time > > - if ZONE_HIGHMEM exist during boot, put the CMA area in there > > - otherwise, put the CMA area at the top end of lowmem, and change > > the zone sizes so ZONE_HIGHMEM stretches over all of the CMA memory. > > One of the requirements of the allocator is that the returned memory > should be zero'd (because it can be exposed to userspace via ALSA > and frame buffers.) > > Zeroing the memory from all the contexts which dma_alloc_coherent > is called from is a trivial matter if its in lowmem, but highmem is > harder. I don't see how. The pages get allocated from an unmapped area or memory, mapped into the kernel address space as uncached or wc and then cleared. This should be the same for lowmem or highmem pages. What am I missing? > Another issue is that when a platform has restricted DMA regions, > they typically don't fall into the highmem zone. As the dmabounce > code allocates from the DMA coherent allocator to provide it with > guaranteed DMA-able memory, that would be rather inconvenient. True. The dmabounce code would consequently have to allocate the memory through an internal function that avoids the contiguous allocation area and goes straight to ZONE_DMA memory as it does today. Arnd -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Linaro-mm-sig] [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Wed, 6 Jul 2011, Russell King - ARM Linux wrote: > Another issue is that when a platform has restricted DMA regions, > they typically don't fall into the highmem zone. As the dmabounce > code allocates from the DMA coherent allocator to provide it with > guaranteed DMA-able memory, that would be rather inconvenient. Do we encounter this in practice i.e. do those platforms requiring large contiguous allocations motivating this work have such DMA restrictions? Nicolas -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Wed, Jul 06, 2011 at 04:09:29PM +0200, Arnd Bergmann wrote: > Maybe you can simply adapt the default location of the contiguous memory > are like this: > - make CONFIG_CMA depend on CONFIG_HIGHMEM on ARM, at compile time > - if ZONE_HIGHMEM exist during boot, put the CMA area in there > - otherwise, put the CMA area at the top end of lowmem, and change > the zone sizes so ZONE_HIGHMEM stretches over all of the CMA memory. One of the requirements of the allocator is that the returned memory should be zero'd (because it can be exposed to userspace via ALSA and frame buffers.) Zeroing the memory from all the contexts which dma_alloc_coherent is called from is a trivial matter if its in lowmem, but highmem is harder. Another issue is that when a platform has restricted DMA regions, they typically don't fall into the highmem zone. As the dmabounce code allocates from the DMA coherent allocator to provide it with guaranteed DMA-able memory, that would be rather inconvenient. -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Wednesday 06 July 2011, Marek Szyprowski wrote: > The only problem that might need to be resolved is GFP_ATOMIC allocation > (updating page properties probably requires some locking), but it can be > served from a special area which is created on boot without low-memory > mapping at all. None sane driver will call dma_alloc_coherent(GFP_ATOMIC) > for large buffers anyway. Would it be easier to start with a version that only allocated from memory without a low-memory mapping at first? This would be similar to the approach that Russell's fix for the regular dma_alloc_coherent has taken, except that you need to also allow the memory to be used as highmem user pages. Maybe you can simply adapt the default location of the contiguous memory are like this: - make CONFIG_CMA depend on CONFIG_HIGHMEM on ARM, at compile time - if ZONE_HIGHMEM exist during boot, put the CMA area in there - otherwise, put the CMA area at the top end of lowmem, and change the zone sizes so ZONE_HIGHMEM stretches over all of the CMA memory. Arnd -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 6/8] drivers: add Contiguous Memory Allocator
Hello, On Tuesday, July 05, 2011 1:34 PM Russell King - ARM Linux wrote: > On Tue, Jul 05, 2011 at 09:41:48AM +0200, Marek Szyprowski wrote: > > The Contiguous Memory Allocator is a set of helper functions for DMA > > mapping framework that improves allocations of contiguous memory chunks. > > > > CMA grabs memory on system boot, marks it with CMA_MIGRATE_TYPE and > > gives back to the system. Kernel is allowed to allocate movable pages > > within CMA's managed memory so that it can be used for example for page > > cache when DMA mapping do not use it. On dma_alloc_from_contiguous() > > request such pages are migrated out of CMA area to free required > > contiguous block and fulfill the request. This allows to allocate large > > contiguous chunks of memory at any time assuming that there is enough > > free memory available in the system. > > > > This code is heavily based on earlier works by Michal Nazarewicz. > > And how are you addressing the technical concerns about aliasing of > cache attributes which I keep bringing up with this and you keep > ignoring and telling me that I'm standing in your way. I'm perfectly aware of the issues with aliasing of cache attributes. My idea is to change low memory linear mapping for all CMA areas on boot time to use 2 level page tables (4KiB mappings instead of super-section mappings). This way the page properties for a single page in CMA area can be changed/updated at any time to match required coherent/writecombine attributes. Linear mapping can be even removed completely if we want to create the it elsewhere in the address space. The only problem that might need to be resolved is GFP_ATOMIC allocation (updating page properties probably requires some locking), but it can be served from a special area which is created on boot without low-memory mapping at all. None sane driver will call dma_alloc_coherent(GFP_ATOMIC) for large buffers anyway. CMA limits the memory area from which coherent pages are being taken quite well, so the change in the linear mapping method should have no significant impact on the system performance. I didn't implement such solution yet, because it is really hard to handle all issues at the same time and creating the allocator was just a first step. Best regards -- Marek Szyprowski Samsung Poland R&D Center -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Tuesday 05 July 2011, Russell King - ARM Linux wrote: > On Tue, Jul 05, 2011 at 02:27:44PM +0200, Arnd Bergmann wrote: > > It's also a preexisting problem as far as I can tell, and it needs > > to be solved in __dma_alloc for both cases, dma_alloc_from_contiguous > > and __alloc_system_pages as introduced in patch 7. > > Which is now resolved in linux-next, and has been through this cycle > as previously discussed. > > It's taken some time because the guy who tested the patch for me said > he'd review other platforms but never did, so I've just about given up > waiting and stuffed it in ready for the 3.1 merge window irrespective > of anything else. Ah, sorry I missed that patch on the mailing list, found it now in your for-next branch. If I'm reading your "ARM: DMA: steal memory for DMA coherent mappings" correctly, the idea is to have a per-platform compile-time amount of memory that is reserved purely for coherent allocations and taking out of the buddy allocator, right? As you say, this solves the problem for the non-CMA case, and does not apply to CMA because the entire point of CMA is not to remove the pages from the buddy allocator in order to preserve memory. So with your patch getting merged, patch 7/8 obviously has both a conflict and introduces a regression against the fix you did. Consequently that patch needs to be redone in a way that fits on top of your patch and avoids the double-mapping problem. What about the rest? As I mentioned in private, adding invasive features to core code is obviously not nice if it can be avoided, but my feeling is that we can no longer claim that there is no need for this with so much hardware relying on large contiguous memory ranges for DMA. The patches have come a long way since the first version, especially regarding the device driver interface and I think they are about as good as it gets in that regard. I do understand that without patch 7, there isn't a single architecture using the feature, which is somewhat silly, but I'm also convinced that other architectures will start using it, and that a solution for the double mapping in the ways I mentioned in my previous mail is going to happen. Probably not in 3.1 then, but we could put the patches into -mm anyway until we get there. Arnd -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Tue, Jul 05, 2011 at 02:27:44PM +0200, Arnd Bergmann wrote: > It's also a preexisting problem as far as I can tell, and it needs > to be solved in __dma_alloc for both cases, dma_alloc_from_contiguous > and __alloc_system_pages as introduced in patch 7. Which is now resolved in linux-next, and has been through this cycle as previously discussed. It's taken some time because the guy who tested the patch for me said he'd review other platforms but never did, so I've just about given up waiting and stuffed it in ready for the 3.1 merge window irrespective of anything else. -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Tuesday 05 July 2011, Russell King - ARM Linux wrote: > On Tue, Jul 05, 2011 at 09:41:48AM +0200, Marek Szyprowski wrote: > > The Contiguous Memory Allocator is a set of helper functions for DMA > > mapping framework that improves allocations of contiguous memory chunks. > > > > CMA grabs memory on system boot, marks it with CMA_MIGRATE_TYPE and > > gives back to the system. Kernel is allowed to allocate movable pages > > within CMA's managed memory so that it can be used for example for page > > cache when DMA mapping do not use it. On dma_alloc_from_contiguous() > > request such pages are migrated out of CMA area to free required > > contiguous block and fulfill the request. This allows to allocate large > > contiguous chunks of memory at any time assuming that there is enough > > free memory available in the system. > > > > This code is heavily based on earlier works by Michal Nazarewicz. > > And how are you addressing the technical concerns about aliasing of > cache attributes which I keep bringing up with this and you keep > ignoring and telling me that I'm standing in your way. This is of course an important issue, and it's the one item listed as TODO in the introductory mail that sent. It's also a preexisting problem as far as I can tell, and it needs to be solved in __dma_alloc for both cases, dma_alloc_from_contiguous and __alloc_system_pages as introduced in patch 7. We've discussed this back and forth, and it always comes down to one of two ugly solutions: 1. Put all of the MIGRATE_CMA and pages into highmem and change __alloc_system_pages so it also allocates only from highmem pages. The consequences of this are that we always need to build kernels with highmem enabled and that we have less lowmem on systems that are already small, both of which can be fairly expensive unless you have lots of highmem already. 2. Add logic to unmap pages from the linear mapping, which is very expensive because it forces the use of small pages in the linear mapping (or in parts of it), and possibly means walking all page tables to remove the PTEs on alloc and put them back in on free. I believe that Chunsang Jeong from Linaro is planning to implement both variants and post them for review, so we can decide which one to merge, or even to merge both and make it a configuration option. See also https://blueprints.launchpad.net/linaro-mm-sig/+spec/engr-mm-dma-mapping-2011.07 I don't think we need to make merging the CMA patches depending on the other patches, it's clear that both need to be solved, and they are independent enough. Arnd -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 6/8] drivers: add Contiguous Memory Allocator
On Tue, Jul 05, 2011 at 09:41:48AM +0200, Marek Szyprowski wrote: > The Contiguous Memory Allocator is a set of helper functions for DMA > mapping framework that improves allocations of contiguous memory chunks. > > CMA grabs memory on system boot, marks it with CMA_MIGRATE_TYPE and > gives back to the system. Kernel is allowed to allocate movable pages > within CMA's managed memory so that it can be used for example for page > cache when DMA mapping do not use it. On dma_alloc_from_contiguous() > request such pages are migrated out of CMA area to free required > contiguous block and fulfill the request. This allows to allocate large > contiguous chunks of memory at any time assuming that there is enough > free memory available in the system. > > This code is heavily based on earlier works by Michal Nazarewicz. And how are you addressing the technical concerns about aliasing of cache attributes which I keep bringing up with this and you keep ignoring and telling me that I'm standing in your way. -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [PATCH 6/8] drivers: add Contiguous Memory Allocator
Hello, On Tuesday, July 05, 2011 9:42 AM Marek Szyprowski wrote: > The Contiguous Memory Allocator is a set of helper functions for DMA > mapping framework that improves allocations of contiguous memory chunks. > > CMA grabs memory on system boot, marks it with CMA_MIGRATE_TYPE and > gives back to the system. Kernel is allowed to allocate movable pages > within CMA's managed memory so that it can be used for example for page > cache when DMA mapping do not use it. On dma_alloc_from_contiguous() > request such pages are migrated out of CMA area to free required > contiguous block and fulfill the request. This allows to allocate large > contiguous chunks of memory at any time assuming that there is enough > free memory available in the system. > > This code is heavily based on earlier works by Michal Nazarewicz. > > Signed-off-by: Marek Szyprowski > Signed-off-by: Kyungmin Park > CC: Michal Nazarewicz > --- > drivers/base/Kconfig | 77 + > drivers/base/Makefile |1 + > drivers/base/dma-contiguous.c | 367 > > include/linux/dma-contiguous.h | 104 +++ > 4 files changed, 549 insertions(+), 0 deletions(-) > create mode 100644 drivers/base/dma-contiguous.c > create mode 100644 include/linux/dma-contiguous.h > > diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig > index d57e8d0..95ae1a7 100644 > --- a/drivers/base/Kconfig > +++ b/drivers/base/Kconfig > @@ -168,4 +168,81 @@ config SYS_HYPERVISOR > bool > default n > > +config CMA > + bool "Contiguous Memory Allocator" > + depends HAVE_DMA_CONTIGUOUS && HAVE_MEMBLOCK The above line should be obviously "depends on HAVE_DMA_CONTIGUOUS && HAVE_MEMBLOCK". I'm sorry for posting broken version. (snipped) Best regards -- Marek Szyprowski Samsung Poland R&D Center -- To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 6/8] drivers: add Contiguous Memory Allocator
The Contiguous Memory Allocator is a set of helper functions for DMA mapping framework that improves allocations of contiguous memory chunks. CMA grabs memory on system boot, marks it with CMA_MIGRATE_TYPE and gives back to the system. Kernel is allowed to allocate movable pages within CMA's managed memory so that it can be used for example for page cache when DMA mapping do not use it. On dma_alloc_from_contiguous() request such pages are migrated out of CMA area to free required contiguous block and fulfill the request. This allows to allocate large contiguous chunks of memory at any time assuming that there is enough free memory available in the system. This code is heavily based on earlier works by Michal Nazarewicz. Signed-off-by: Marek Szyprowski Signed-off-by: Kyungmin Park CC: Michal Nazarewicz --- drivers/base/Kconfig | 77 + drivers/base/Makefile |1 + drivers/base/dma-contiguous.c | 367 include/linux/dma-contiguous.h | 104 +++ 4 files changed, 549 insertions(+), 0 deletions(-) create mode 100644 drivers/base/dma-contiguous.c create mode 100644 include/linux/dma-contiguous.h diff --git a/drivers/base/Kconfig b/drivers/base/Kconfig index d57e8d0..95ae1a7 100644 --- a/drivers/base/Kconfig +++ b/drivers/base/Kconfig @@ -168,4 +168,81 @@ config SYS_HYPERVISOR bool default n +config CMA + bool "Contiguous Memory Allocator" + depends HAVE_DMA_CONTIGUOUS && HAVE_MEMBLOCK + select MIGRATION + select CMA_MIGRATE_TYPE + help + This enables the Contiguous Memory Allocator which allows drivers + to allocate big physically-contiguous blocks of memory for use with + hardware components that do not support I/O map nor scatter-gather. + + For more information see . + If unsure, say "n". + +if CMA + +config CMA_DEBUG + bool "CMA debug messages (DEVELOPEMENT)" + help + Turns on debug messages in CMA. This produces KERN_DEBUG + messages for every CMA call as well as various messages while + processing calls such as dma_alloc_from_contiguous(). + This option does not affect warning and error messages. + +comment "Default contiguous memory area size:" + +config CMA_SIZE_ABSOLUTE + int "Absolute size (in MiB)" + default 16 + help + Defines the size (in MiB) of the default memory area for Contiguous + Memory Allocator. + +config CMA_SIZE_PERCENTAGE + int "Percentage of total memory" + default 10 + help + Defines the size of the default memory area for Contiguous Memory + Allocator as a percentage of the total memory in the system. + +choice + prompt "Selected region size" + default CMA_SIZE_SEL_ABSOLUTE + +config CMA_SIZE_SEL_ABSOLUTE + bool "Use absolute value only" + +config CMA_SIZE_SEL_PERCENTAGE + bool "Use percentage value only" + +config CMA_SIZE_SEL_MIN + bool "Use lower value (minimum)" + +config CMA_SIZE_SEL_MAX + bool "Use higher value (maximum)" + +endchoice + +config CMA_ALIGNMENT + int "Maximum PAGE_SIZE order of alignment for contiguous buffers" + range 4 9 + default 8 + help + DMA mapping framework by default aligns all buffers to the smallest + PAGE_SIZE order which is greater than or equal to the requested buffer + size. This works well for buffers up to a few hundreds kilobytes, but + for larger buffers it just a memory waste. With this parameter you can + specify the maximum PAGE_SIZE order for contiguous buffers. Larger + buffers will be aligned only to this specified order. The order is + expressed as a power of two multiplied by the PAGE_SIZE. + + For example, if your system defaults to 4KiB pages, the order value + of 8 means that the buffers will be aligned up to 1MiB only. + + If unsure, leave the default value "8". + +endif + endmenu diff --git a/drivers/base/Makefile b/drivers/base/Makefile index 4c5701c..be6aab4 100644 --- a/drivers/base/Makefile +++ b/drivers/base/Makefile @@ -5,6 +5,7 @@ obj-y := core.o sys.o bus.o dd.o syscore.o \ cpu.o firmware.o init.o map.o devres.o \ attribute_container.o transport_class.o obj-$(CONFIG_DEVTMPFS) += devtmpfs.o +obj-$(CONFIG_CMA) += dma-contiguous.o obj-y += power/ obj-$(CONFIG_HAS_DMA) += dma-mapping.o obj-$(CONFIG_HAVE_GENERIC_DMA_COHERENT) += dma-coherent.o diff --git a/drivers/base/dma-contiguous.c b/drivers/base/dma-contiguous.c new file mode 100644 index 000..707b901 --- /dev/null +++ b/drivers/base/dma-contiguous.c @@ -0,0 +1,367 @@ +/* + * Contiguous Memory Allocator for DMA mapping framework + * Copyright (c) 2010-2011 by Samsung Electronics. + * Written by: + * Marek Szyprowski + * Michal Nazarewicz + * +
