Re: dma_declare_coherent_memory fails for RAM allocated memory

2016-06-04 Thread Ivaylo Dimitrov



On  2.06.2016 20:14, Ivaylo Dimitrov wrote:

Hi,

On  1.06.2016 14:18, Brian Starkey wrote:

Hi Ivo,

On Sun, May 29, 2016 at 05:56:02PM +0300, Ivaylo Dimitrov wrote:

Hi,

When trying to declare and use DT reserved memory region on ARM
(OMAP3), dma_declare_coherent_memory() fails in memremap(). This is
from today's master:

[ cut here ]
WARNING: CPU: 0 PID: 1 at kernel/memremap.c:111 memremap+0x118/0x194
memremap attempted on ram 0x8f80 size: 0x70
Modules linked in:
CPU: 0 PID: 1 Comm: swapper Not tainted 4.6.0+ #15
Hardware name: Nokia RX-51 board
[] (unwind_backtrace) from [] (show_stack+0x10/0x14)
[] (show_stack) from [] (__warn+0xcc/0xf8)
[] (__warn) from [] (warn_slowpath_fmt+0x34/0x44)
[] (warn_slowpath_fmt) from []
(memremap+0x118/0x194)
[] (memremap) from []
(dma_init_coherent_memory+0x48/0x104)
[] (dma_init_coherent_memory) from []
(dma_declare_coherent_memory+0x2c/0x68)
[] (dma_declare_coherent_memory) from []
(rmem_omapfb_device_init+0x34/0x64)
[] (rmem_omapfb_device_init) from []
(of_reserved_mem_device_init+0x94/0xd8)
[] (of_reserved_mem_device_init) from []
(omapdss_init_of+0xe4/0x154)
[] (omapdss_init_of) from []
(customize_machine+0x20/0x44)
[] (customize_machine) from []
(do_one_initcall+0xac/0x158)
[] (do_one_initcall) from []
(kernel_init_freeable+0xf8/0x1c8)
[] (kernel_init_freeable) from []
(kernel_init+0x8/0x110)
[] (kernel_init) from [] (ret_from_fork+0x14/0x3c)
---[ end trace 73a8c076df72166b ]---
omapfb: dma_declare_coherent_memory failed


The failing code looks like:
.
.
.
static int rmem_omapfb_device_init(struct reserved_mem *rmem, struct
device *dev)
{
int dma;

if (rmem->priv)
return 0;

dma = dma_declare_coherent_memory(_fb_device.dev, rmem->base,
  rmem->base, rmem->size,
  DMA_MEMORY_MAP |
  DMA_MEMORY_EXCLUSIVE);
if (!(dma & DMA_MEMORY_MAP)) {
pr_err("omapfb: dma_declare_coherent_memory failed\n");
return -ENOMEM;
}
else
rmem->priv = omap_fb_device.dev.dma_mem;

return 0;
}

static void rmem_omapfb_device_release(struct reserved_mem *rmem,
   struct device *dev)
{
dma_release_declared_memory(_fb_device.dev);
}

static const struct reserved_mem_ops rmem_omapfb_ops = {
.device_init= rmem_omapfb_device_init,
.device_release = rmem_omapfb_device_release,
};

static int __init rmem_omapfb_setup(struct reserved_mem *rmem)
{
rmem->ops = _omapfb_ops;
pr_info("omapfb: reserved %d bytes at %pa\n", rmem->size,
>base);

return 0;
}

RESERVEDMEM_OF_DECLARE(dss, "ti,omapfb-memsize", rmem_omapfb_setup);


It turns out that dma_init_coherent_memory calls memremap with
MEMREMAP_WC flag, which is disallowed for RAM IIUC.



Right. If you want to use a memory region as coherent DMA memory, then
that same region can't be System RAM, because System RAM is usually
mapped in a non-coherent fashion.


I quickly hacked some code to fix the issue, but as memremap API is
relatively new(esp to me), I wonder if this is the correct way to go:

diff --git a/drivers/base/dma-coherent.c b/drivers/base/dma-coherent.c
index bdf28f7..04b1687 100644
--- a/drivers/base/dma-coherent.c
+++ b/drivers/base/dma-coherent.c
@@ -5,6 +5,7 @@
#include 
#include 
#include 
+#include 
#include 
#include 

@@ -32,8 +33,12 @@ static bool dma_init_coherent_memory(
   if (!size)
   goto out;

-   if (flags & DMA_MEMORY_MAP)
-   mem_base = memremap(phys_addr, size, MEMREMAP_WC);
+   if (flags & DMA_MEMORY_MAP) {
+   unsigned long map_type =
memblock_is_map_memory(phys_addr) ?
+MEMREMAP_WB :
MEMREMAP_WC;
+
+   mem_base = memremap(phys_addr, size, map_type);
+   }
   else
   mem_base = ioremap(phys_addr, size);
   if (!mem_base)

Does the above code looks sane? How to fix the problem if not?


AFAIK dma_declare_coherent_memory() should only be used on physical
addresses which don't already have a CPU virtual mapping. Certainly
returning (potentially) cached memory as DMA coherent is asking for
trouble.

I think what you want to do is add a "no-map;" property to your
reserved-memory region in DT. That will stop the kernel from using it
as System RAM, leaving it free for your framebuffer.



I tried to use "no-map", but for some reason it doesn't allow me to
allocate more than 2MB, anything above results in:

qemu: fatal: Trying to execute code outside RAM or ROM at 0x0010

R00=2220 R01=cf901000 R02=cfa4 R03=e000
R04=0008ff80 R05=0009 R06= R07=c096333c
R08=c0904dc0 R09=c093a23c R10=0001 R11=c0727b3b
R12=ff80 R13=c093af8c R14=c082aba0 R15=0010
PSR=a1d7 N-C- A abt32
Aborted (core dumped)

I was able to track the code to memblock_isolate_range() - if I return
-ENOMEM on http://lxr.free-electrons.com/source/mm/memblock.c#L671, the
above crash does not 

Re: dma_declare_coherent_memory fails for RAM allocated memory

2016-06-04 Thread Ivaylo Dimitrov



On  2.06.2016 20:14, Ivaylo Dimitrov wrote:

Hi,

On  1.06.2016 14:18, Brian Starkey wrote:

Hi Ivo,

On Sun, May 29, 2016 at 05:56:02PM +0300, Ivaylo Dimitrov wrote:

Hi,

When trying to declare and use DT reserved memory region on ARM
(OMAP3), dma_declare_coherent_memory() fails in memremap(). This is
from today's master:

[ cut here ]
WARNING: CPU: 0 PID: 1 at kernel/memremap.c:111 memremap+0x118/0x194
memremap attempted on ram 0x8f80 size: 0x70
Modules linked in:
CPU: 0 PID: 1 Comm: swapper Not tainted 4.6.0+ #15
Hardware name: Nokia RX-51 board
[] (unwind_backtrace) from [] (show_stack+0x10/0x14)
[] (show_stack) from [] (__warn+0xcc/0xf8)
[] (__warn) from [] (warn_slowpath_fmt+0x34/0x44)
[] (warn_slowpath_fmt) from []
(memremap+0x118/0x194)
[] (memremap) from []
(dma_init_coherent_memory+0x48/0x104)
[] (dma_init_coherent_memory) from []
(dma_declare_coherent_memory+0x2c/0x68)
[] (dma_declare_coherent_memory) from []
(rmem_omapfb_device_init+0x34/0x64)
[] (rmem_omapfb_device_init) from []
(of_reserved_mem_device_init+0x94/0xd8)
[] (of_reserved_mem_device_init) from []
(omapdss_init_of+0xe4/0x154)
[] (omapdss_init_of) from []
(customize_machine+0x20/0x44)
[] (customize_machine) from []
(do_one_initcall+0xac/0x158)
[] (do_one_initcall) from []
(kernel_init_freeable+0xf8/0x1c8)
[] (kernel_init_freeable) from []
(kernel_init+0x8/0x110)
[] (kernel_init) from [] (ret_from_fork+0x14/0x3c)
---[ end trace 73a8c076df72166b ]---
omapfb: dma_declare_coherent_memory failed


The failing code looks like:
.
.
.
static int rmem_omapfb_device_init(struct reserved_mem *rmem, struct
device *dev)
{
int dma;

if (rmem->priv)
return 0;

dma = dma_declare_coherent_memory(_fb_device.dev, rmem->base,
  rmem->base, rmem->size,
  DMA_MEMORY_MAP |
  DMA_MEMORY_EXCLUSIVE);
if (!(dma & DMA_MEMORY_MAP)) {
pr_err("omapfb: dma_declare_coherent_memory failed\n");
return -ENOMEM;
}
else
rmem->priv = omap_fb_device.dev.dma_mem;

return 0;
}

static void rmem_omapfb_device_release(struct reserved_mem *rmem,
   struct device *dev)
{
dma_release_declared_memory(_fb_device.dev);
}

static const struct reserved_mem_ops rmem_omapfb_ops = {
.device_init= rmem_omapfb_device_init,
.device_release = rmem_omapfb_device_release,
};

static int __init rmem_omapfb_setup(struct reserved_mem *rmem)
{
rmem->ops = _omapfb_ops;
pr_info("omapfb: reserved %d bytes at %pa\n", rmem->size,
>base);

return 0;
}

RESERVEDMEM_OF_DECLARE(dss, "ti,omapfb-memsize", rmem_omapfb_setup);


It turns out that dma_init_coherent_memory calls memremap with
MEMREMAP_WC flag, which is disallowed for RAM IIUC.



Right. If you want to use a memory region as coherent DMA memory, then
that same region can't be System RAM, because System RAM is usually
mapped in a non-coherent fashion.


I quickly hacked some code to fix the issue, but as memremap API is
relatively new(esp to me), I wonder if this is the correct way to go:

diff --git a/drivers/base/dma-coherent.c b/drivers/base/dma-coherent.c
index bdf28f7..04b1687 100644
--- a/drivers/base/dma-coherent.c
+++ b/drivers/base/dma-coherent.c
@@ -5,6 +5,7 @@
#include 
#include 
#include 
+#include 
#include 
#include 

@@ -32,8 +33,12 @@ static bool dma_init_coherent_memory(
   if (!size)
   goto out;

-   if (flags & DMA_MEMORY_MAP)
-   mem_base = memremap(phys_addr, size, MEMREMAP_WC);
+   if (flags & DMA_MEMORY_MAP) {
+   unsigned long map_type =
memblock_is_map_memory(phys_addr) ?
+MEMREMAP_WB :
MEMREMAP_WC;
+
+   mem_base = memremap(phys_addr, size, map_type);
+   }
   else
   mem_base = ioremap(phys_addr, size);
   if (!mem_base)

Does the above code looks sane? How to fix the problem if not?


AFAIK dma_declare_coherent_memory() should only be used on physical
addresses which don't already have a CPU virtual mapping. Certainly
returning (potentially) cached memory as DMA coherent is asking for
trouble.

I think what you want to do is add a "no-map;" property to your
reserved-memory region in DT. That will stop the kernel from using it
as System RAM, leaving it free for your framebuffer.



I tried to use "no-map", but for some reason it doesn't allow me to
allocate more than 2MB, anything above results in:

qemu: fatal: Trying to execute code outside RAM or ROM at 0x0010

R00=2220 R01=cf901000 R02=cfa4 R03=e000
R04=0008ff80 R05=0009 R06= R07=c096333c
R08=c0904dc0 R09=c093a23c R10=0001 R11=c0727b3b
R12=ff80 R13=c093af8c R14=c082aba0 R15=0010
PSR=a1d7 N-C- A abt32
Aborted (core dumped)

I was able to track the code to memblock_isolate_range() - if I return
-ENOMEM on http://lxr.free-electrons.com/source/mm/memblock.c#L671, the
above crash does not 

Re: dma_declare_coherent_memory fails for RAM allocated memory

2016-06-02 Thread Ivaylo Dimitrov

Hi,

On  1.06.2016 14:18, Brian Starkey wrote:

Hi Ivo,

On Sun, May 29, 2016 at 05:56:02PM +0300, Ivaylo Dimitrov wrote:

Hi,

When trying to declare and use DT reserved memory region on ARM
(OMAP3), dma_declare_coherent_memory() fails in memremap(). This is
from today's master:

[ cut here ]
WARNING: CPU: 0 PID: 1 at kernel/memremap.c:111 memremap+0x118/0x194
memremap attempted on ram 0x8f80 size: 0x70
Modules linked in:
CPU: 0 PID: 1 Comm: swapper Not tainted 4.6.0+ #15
Hardware name: Nokia RX-51 board
[] (unwind_backtrace) from [] (show_stack+0x10/0x14)
[] (show_stack) from [] (__warn+0xcc/0xf8)
[] (__warn) from [] (warn_slowpath_fmt+0x34/0x44)
[] (warn_slowpath_fmt) from [] (memremap+0x118/0x194)
[] (memremap) from []
(dma_init_coherent_memory+0x48/0x104)
[] (dma_init_coherent_memory) from []
(dma_declare_coherent_memory+0x2c/0x68)
[] (dma_declare_coherent_memory) from []
(rmem_omapfb_device_init+0x34/0x64)
[] (rmem_omapfb_device_init) from []
(of_reserved_mem_device_init+0x94/0xd8)
[] (of_reserved_mem_device_init) from []
(omapdss_init_of+0xe4/0x154)
[] (omapdss_init_of) from []
(customize_machine+0x20/0x44)
[] (customize_machine) from []
(do_one_initcall+0xac/0x158)
[] (do_one_initcall) from []
(kernel_init_freeable+0xf8/0x1c8)
[] (kernel_init_freeable) from []
(kernel_init+0x8/0x110)
[] (kernel_init) from [] (ret_from_fork+0x14/0x3c)
---[ end trace 73a8c076df72166b ]---
omapfb: dma_declare_coherent_memory failed


The failing code looks like:
.
.
.
static int rmem_omapfb_device_init(struct reserved_mem *rmem, struct
device *dev)
{
int dma;

if (rmem->priv)
return 0;

dma = dma_declare_coherent_memory(_fb_device.dev, rmem->base,
  rmem->base, rmem->size,
  DMA_MEMORY_MAP |
  DMA_MEMORY_EXCLUSIVE);
if (!(dma & DMA_MEMORY_MAP)) {
pr_err("omapfb: dma_declare_coherent_memory failed\n");
return -ENOMEM;
}
else
rmem->priv = omap_fb_device.dev.dma_mem;

return 0;
}

static void rmem_omapfb_device_release(struct reserved_mem *rmem,
   struct device *dev)
{
dma_release_declared_memory(_fb_device.dev);
}

static const struct reserved_mem_ops rmem_omapfb_ops = {
.device_init= rmem_omapfb_device_init,
.device_release = rmem_omapfb_device_release,
};

static int __init rmem_omapfb_setup(struct reserved_mem *rmem)
{
rmem->ops = _omapfb_ops;
pr_info("omapfb: reserved %d bytes at %pa\n", rmem->size,
>base);

return 0;
}

RESERVEDMEM_OF_DECLARE(dss, "ti,omapfb-memsize", rmem_omapfb_setup);


It turns out that dma_init_coherent_memory calls memremap with
MEMREMAP_WC flag, which is disallowed for RAM IIUC.



Right. If you want to use a memory region as coherent DMA memory, then
that same region can't be System RAM, because System RAM is usually
mapped in a non-coherent fashion.


I quickly hacked some code to fix the issue, but as memremap API is
relatively new(esp to me), I wonder if this is the correct way to go:

diff --git a/drivers/base/dma-coherent.c b/drivers/base/dma-coherent.c
index bdf28f7..04b1687 100644
--- a/drivers/base/dma-coherent.c
+++ b/drivers/base/dma-coherent.c
@@ -5,6 +5,7 @@
#include 
#include 
#include 
+#include 
#include 
#include 

@@ -32,8 +33,12 @@ static bool dma_init_coherent_memory(
   if (!size)
   goto out;

-   if (flags & DMA_MEMORY_MAP)
-   mem_base = memremap(phys_addr, size, MEMREMAP_WC);
+   if (flags & DMA_MEMORY_MAP) {
+   unsigned long map_type =
memblock_is_map_memory(phys_addr) ?
+MEMREMAP_WB :
MEMREMAP_WC;
+
+   mem_base = memremap(phys_addr, size, map_type);
+   }
   else
   mem_base = ioremap(phys_addr, size);
   if (!mem_base)

Does the above code looks sane? How to fix the problem if not?


AFAIK dma_declare_coherent_memory() should only be used on physical
addresses which don't already have a CPU virtual mapping. Certainly
returning (potentially) cached memory as DMA coherent is asking for
trouble.

I think what you want to do is add a "no-map;" property to your
reserved-memory region in DT. That will stop the kernel from using it
as System RAM, leaving it free for your framebuffer.



I tried to use "no-map", but for some reason it doesn't allow me to 
allocate more than 2MB, anything above results in:


qemu: fatal: Trying to execute code outside RAM or ROM at 0x0010

R00=2220 R01=cf901000 R02=cfa4 R03=e000
R04=0008ff80 R05=0009 R06= R07=c096333c
R08=c0904dc0 R09=c093a23c R10=0001 R11=c0727b3b
R12=ff80 R13=c093af8c R14=c082aba0 R15=0010
PSR=a1d7 N-C- A abt32
Aborted (core dumped)

I was able to track the code to memblock_isolate_range() - if I return 
-ENOMEM on http://lxr.free-electrons.com/source/mm/memblock.c#L671, the 
above crash does not occur(however the allocation fails as 

Re: dma_declare_coherent_memory fails for RAM allocated memory

2016-06-02 Thread Ivaylo Dimitrov

Hi,

On  1.06.2016 14:18, Brian Starkey wrote:

Hi Ivo,

On Sun, May 29, 2016 at 05:56:02PM +0300, Ivaylo Dimitrov wrote:

Hi,

When trying to declare and use DT reserved memory region on ARM
(OMAP3), dma_declare_coherent_memory() fails in memremap(). This is
from today's master:

[ cut here ]
WARNING: CPU: 0 PID: 1 at kernel/memremap.c:111 memremap+0x118/0x194
memremap attempted on ram 0x8f80 size: 0x70
Modules linked in:
CPU: 0 PID: 1 Comm: swapper Not tainted 4.6.0+ #15
Hardware name: Nokia RX-51 board
[] (unwind_backtrace) from [] (show_stack+0x10/0x14)
[] (show_stack) from [] (__warn+0xcc/0xf8)
[] (__warn) from [] (warn_slowpath_fmt+0x34/0x44)
[] (warn_slowpath_fmt) from [] (memremap+0x118/0x194)
[] (memremap) from []
(dma_init_coherent_memory+0x48/0x104)
[] (dma_init_coherent_memory) from []
(dma_declare_coherent_memory+0x2c/0x68)
[] (dma_declare_coherent_memory) from []
(rmem_omapfb_device_init+0x34/0x64)
[] (rmem_omapfb_device_init) from []
(of_reserved_mem_device_init+0x94/0xd8)
[] (of_reserved_mem_device_init) from []
(omapdss_init_of+0xe4/0x154)
[] (omapdss_init_of) from []
(customize_machine+0x20/0x44)
[] (customize_machine) from []
(do_one_initcall+0xac/0x158)
[] (do_one_initcall) from []
(kernel_init_freeable+0xf8/0x1c8)
[] (kernel_init_freeable) from []
(kernel_init+0x8/0x110)
[] (kernel_init) from [] (ret_from_fork+0x14/0x3c)
---[ end trace 73a8c076df72166b ]---
omapfb: dma_declare_coherent_memory failed


The failing code looks like:
.
.
.
static int rmem_omapfb_device_init(struct reserved_mem *rmem, struct
device *dev)
{
int dma;

if (rmem->priv)
return 0;

dma = dma_declare_coherent_memory(_fb_device.dev, rmem->base,
  rmem->base, rmem->size,
  DMA_MEMORY_MAP |
  DMA_MEMORY_EXCLUSIVE);
if (!(dma & DMA_MEMORY_MAP)) {
pr_err("omapfb: dma_declare_coherent_memory failed\n");
return -ENOMEM;
}
else
rmem->priv = omap_fb_device.dev.dma_mem;

return 0;
}

static void rmem_omapfb_device_release(struct reserved_mem *rmem,
   struct device *dev)
{
dma_release_declared_memory(_fb_device.dev);
}

static const struct reserved_mem_ops rmem_omapfb_ops = {
.device_init= rmem_omapfb_device_init,
.device_release = rmem_omapfb_device_release,
};

static int __init rmem_omapfb_setup(struct reserved_mem *rmem)
{
rmem->ops = _omapfb_ops;
pr_info("omapfb: reserved %d bytes at %pa\n", rmem->size,
>base);

return 0;
}

RESERVEDMEM_OF_DECLARE(dss, "ti,omapfb-memsize", rmem_omapfb_setup);


It turns out that dma_init_coherent_memory calls memremap with
MEMREMAP_WC flag, which is disallowed for RAM IIUC.



Right. If you want to use a memory region as coherent DMA memory, then
that same region can't be System RAM, because System RAM is usually
mapped in a non-coherent fashion.


I quickly hacked some code to fix the issue, but as memremap API is
relatively new(esp to me), I wonder if this is the correct way to go:

diff --git a/drivers/base/dma-coherent.c b/drivers/base/dma-coherent.c
index bdf28f7..04b1687 100644
--- a/drivers/base/dma-coherent.c
+++ b/drivers/base/dma-coherent.c
@@ -5,6 +5,7 @@
#include 
#include 
#include 
+#include 
#include 
#include 

@@ -32,8 +33,12 @@ static bool dma_init_coherent_memory(
   if (!size)
   goto out;

-   if (flags & DMA_MEMORY_MAP)
-   mem_base = memremap(phys_addr, size, MEMREMAP_WC);
+   if (flags & DMA_MEMORY_MAP) {
+   unsigned long map_type =
memblock_is_map_memory(phys_addr) ?
+MEMREMAP_WB :
MEMREMAP_WC;
+
+   mem_base = memremap(phys_addr, size, map_type);
+   }
   else
   mem_base = ioremap(phys_addr, size);
   if (!mem_base)

Does the above code looks sane? How to fix the problem if not?


AFAIK dma_declare_coherent_memory() should only be used on physical
addresses which don't already have a CPU virtual mapping. Certainly
returning (potentially) cached memory as DMA coherent is asking for
trouble.

I think what you want to do is add a "no-map;" property to your
reserved-memory region in DT. That will stop the kernel from using it
as System RAM, leaving it free for your framebuffer.



I tried to use "no-map", but for some reason it doesn't allow me to 
allocate more than 2MB, anything above results in:


qemu: fatal: Trying to execute code outside RAM or ROM at 0x0010

R00=2220 R01=cf901000 R02=cfa4 R03=e000
R04=0008ff80 R05=0009 R06= R07=c096333c
R08=c0904dc0 R09=c093a23c R10=0001 R11=c0727b3b
R12=ff80 R13=c093af8c R14=c082aba0 R15=0010
PSR=a1d7 N-C- A abt32
Aborted (core dumped)

I was able to track the code to memblock_isolate_range() - if I return 
-ENOMEM on http://lxr.free-electrons.com/source/mm/memblock.c#L671, the 
above crash does not occur(however the allocation fails as 

Re: dma_declare_coherent_memory fails for RAM allocated memory

2016-06-01 Thread Brian Starkey

Hi Ivo,

On Sun, May 29, 2016 at 05:56:02PM +0300, Ivaylo Dimitrov wrote:

Hi,

When trying to declare and use DT reserved memory region on ARM 
(OMAP3), dma_declare_coherent_memory() fails in memremap(). This is 
from today's master:


[ cut here ]
WARNING: CPU: 0 PID: 1 at kernel/memremap.c:111 memremap+0x118/0x194
memremap attempted on ram 0x8f80 size: 0x70
Modules linked in:
CPU: 0 PID: 1 Comm: swapper Not tainted 4.6.0+ #15
Hardware name: Nokia RX-51 board
[] (unwind_backtrace) from [] (show_stack+0x10/0x14)
[] (show_stack) from [] (__warn+0xcc/0xf8)
[] (__warn) from [] (warn_slowpath_fmt+0x34/0x44)
[] (warn_slowpath_fmt) from [] (memremap+0x118/0x194)
[] (memremap) from [] 
(dma_init_coherent_memory+0x48/0x104)
[] (dma_init_coherent_memory) from [] 
(dma_declare_coherent_memory+0x2c/0x68)
[] (dma_declare_coherent_memory) from [] 
(rmem_omapfb_device_init+0x34/0x64)
[] (rmem_omapfb_device_init) from [] 
(of_reserved_mem_device_init+0x94/0xd8)
[] (of_reserved_mem_device_init) from [] 
(omapdss_init_of+0xe4/0x154)
[] (omapdss_init_of) from [] 
(customize_machine+0x20/0x44)
[] (customize_machine) from [] 
(do_one_initcall+0xac/0x158)
[] (do_one_initcall) from [] 
(kernel_init_freeable+0xf8/0x1c8)
[] (kernel_init_freeable) from [] 
(kernel_init+0x8/0x110)

[] (kernel_init) from [] (ret_from_fork+0x14/0x3c)
---[ end trace 73a8c076df72166b ]---
omapfb: dma_declare_coherent_memory failed


The failing code looks like:
.
.
.
static int rmem_omapfb_device_init(struct reserved_mem *rmem, struct 
device *dev)

{
int dma;

if (rmem->priv)
return 0;

dma = dma_declare_coherent_memory(_fb_device.dev, rmem->base,
  rmem->base, rmem->size,
  DMA_MEMORY_MAP |
  DMA_MEMORY_EXCLUSIVE);
if (!(dma & DMA_MEMORY_MAP)) {
pr_err("omapfb: dma_declare_coherent_memory failed\n");
return -ENOMEM;
}
else
rmem->priv = omap_fb_device.dev.dma_mem;

return 0;
}

static void rmem_omapfb_device_release(struct reserved_mem *rmem,
   struct device *dev)
{
dma_release_declared_memory(_fb_device.dev);
}

static const struct reserved_mem_ops rmem_omapfb_ops = {
.device_init= rmem_omapfb_device_init,
.device_release = rmem_omapfb_device_release,
};

static int __init rmem_omapfb_setup(struct reserved_mem *rmem)
{
rmem->ops = _omapfb_ops;
pr_info("omapfb: reserved %d bytes at %pa\n", rmem->size, >base);

return 0;
}

RESERVEDMEM_OF_DECLARE(dss, "ti,omapfb-memsize", rmem_omapfb_setup);


It turns out that dma_init_coherent_memory calls memremap with 
MEMREMAP_WC flag, which is disallowed for RAM IIUC.




Right. If you want to use a memory region as coherent DMA memory, then
that same region can't be System RAM, because System RAM is usually
mapped in a non-coherent fashion.

I quickly hacked some code to fix the issue, but as memremap API is 
relatively new(esp to me), I wonder if this is the correct way to go:


diff --git a/drivers/base/dma-coherent.c b/drivers/base/dma-coherent.c
index bdf28f7..04b1687 100644
--- a/drivers/base/dma-coherent.c
+++ b/drivers/base/dma-coherent.c
@@ -5,6 +5,7 @@
#include 
#include 
#include 
+#include 
#include 
#include 

@@ -32,8 +33,12 @@ static bool dma_init_coherent_memory(
   if (!size)
   goto out;

-   if (flags & DMA_MEMORY_MAP)
-   mem_base = memremap(phys_addr, size, MEMREMAP_WC);
+   if (flags & DMA_MEMORY_MAP) {
+   unsigned long map_type = memblock_is_map_memory(phys_addr) ?
+MEMREMAP_WB : MEMREMAP_WC;
+
+   mem_base = memremap(phys_addr, size, map_type);
+   }
   else
   mem_base = ioremap(phys_addr, size);
   if (!mem_base)

Does the above code looks sane? How to fix the problem if not?


AFAIK dma_declare_coherent_memory() should only be used on physical
addresses which don't already have a CPU virtual mapping. Certainly
returning (potentially) cached memory as DMA coherent is asking for
trouble.

I think what you want to do is add a "no-map;" property to your
reserved-memory region in DT. That will stop the kernel from using it
as System RAM, leaving it free for your framebuffer.

Cheers,
-Brian



Thanks,
Ivo



Re: dma_declare_coherent_memory fails for RAM allocated memory

2016-06-01 Thread Brian Starkey

Hi Ivo,

On Sun, May 29, 2016 at 05:56:02PM +0300, Ivaylo Dimitrov wrote:

Hi,

When trying to declare and use DT reserved memory region on ARM 
(OMAP3), dma_declare_coherent_memory() fails in memremap(). This is 
from today's master:


[ cut here ]
WARNING: CPU: 0 PID: 1 at kernel/memremap.c:111 memremap+0x118/0x194
memremap attempted on ram 0x8f80 size: 0x70
Modules linked in:
CPU: 0 PID: 1 Comm: swapper Not tainted 4.6.0+ #15
Hardware name: Nokia RX-51 board
[] (unwind_backtrace) from [] (show_stack+0x10/0x14)
[] (show_stack) from [] (__warn+0xcc/0xf8)
[] (__warn) from [] (warn_slowpath_fmt+0x34/0x44)
[] (warn_slowpath_fmt) from [] (memremap+0x118/0x194)
[] (memremap) from [] 
(dma_init_coherent_memory+0x48/0x104)
[] (dma_init_coherent_memory) from [] 
(dma_declare_coherent_memory+0x2c/0x68)
[] (dma_declare_coherent_memory) from [] 
(rmem_omapfb_device_init+0x34/0x64)
[] (rmem_omapfb_device_init) from [] 
(of_reserved_mem_device_init+0x94/0xd8)
[] (of_reserved_mem_device_init) from [] 
(omapdss_init_of+0xe4/0x154)
[] (omapdss_init_of) from [] 
(customize_machine+0x20/0x44)
[] (customize_machine) from [] 
(do_one_initcall+0xac/0x158)
[] (do_one_initcall) from [] 
(kernel_init_freeable+0xf8/0x1c8)
[] (kernel_init_freeable) from [] 
(kernel_init+0x8/0x110)

[] (kernel_init) from [] (ret_from_fork+0x14/0x3c)
---[ end trace 73a8c076df72166b ]---
omapfb: dma_declare_coherent_memory failed


The failing code looks like:
.
.
.
static int rmem_omapfb_device_init(struct reserved_mem *rmem, struct 
device *dev)

{
int dma;

if (rmem->priv)
return 0;

dma = dma_declare_coherent_memory(_fb_device.dev, rmem->base,
  rmem->base, rmem->size,
  DMA_MEMORY_MAP |
  DMA_MEMORY_EXCLUSIVE);
if (!(dma & DMA_MEMORY_MAP)) {
pr_err("omapfb: dma_declare_coherent_memory failed\n");
return -ENOMEM;
}
else
rmem->priv = omap_fb_device.dev.dma_mem;

return 0;
}

static void rmem_omapfb_device_release(struct reserved_mem *rmem,
   struct device *dev)
{
dma_release_declared_memory(_fb_device.dev);
}

static const struct reserved_mem_ops rmem_omapfb_ops = {
.device_init= rmem_omapfb_device_init,
.device_release = rmem_omapfb_device_release,
};

static int __init rmem_omapfb_setup(struct reserved_mem *rmem)
{
rmem->ops = _omapfb_ops;
pr_info("omapfb: reserved %d bytes at %pa\n", rmem->size, >base);

return 0;
}

RESERVEDMEM_OF_DECLARE(dss, "ti,omapfb-memsize", rmem_omapfb_setup);


It turns out that dma_init_coherent_memory calls memremap with 
MEMREMAP_WC flag, which is disallowed for RAM IIUC.




Right. If you want to use a memory region as coherent DMA memory, then
that same region can't be System RAM, because System RAM is usually
mapped in a non-coherent fashion.

I quickly hacked some code to fix the issue, but as memremap API is 
relatively new(esp to me), I wonder if this is the correct way to go:


diff --git a/drivers/base/dma-coherent.c b/drivers/base/dma-coherent.c
index bdf28f7..04b1687 100644
--- a/drivers/base/dma-coherent.c
+++ b/drivers/base/dma-coherent.c
@@ -5,6 +5,7 @@
#include 
#include 
#include 
+#include 
#include 
#include 

@@ -32,8 +33,12 @@ static bool dma_init_coherent_memory(
   if (!size)
   goto out;

-   if (flags & DMA_MEMORY_MAP)
-   mem_base = memremap(phys_addr, size, MEMREMAP_WC);
+   if (flags & DMA_MEMORY_MAP) {
+   unsigned long map_type = memblock_is_map_memory(phys_addr) ?
+MEMREMAP_WB : MEMREMAP_WC;
+
+   mem_base = memremap(phys_addr, size, map_type);
+   }
   else
   mem_base = ioremap(phys_addr, size);
   if (!mem_base)

Does the above code looks sane? How to fix the problem if not?


AFAIK dma_declare_coherent_memory() should only be used on physical
addresses which don't already have a CPU virtual mapping. Certainly
returning (potentially) cached memory as DMA coherent is asking for
trouble.

I think what you want to do is add a "no-map;" property to your
reserved-memory region in DT. That will stop the kernel from using it
as System RAM, leaving it free for your framebuffer.

Cheers,
-Brian



Thanks,
Ivo