RE: mainline 4.14-rc4: renesas_sdhi_internal_dmac ee100000.sd: swiotlb buffer is full?
Hi Dirk-san, > From: Dirk Behme, Sent: Tuesday, October 17, 2017 2:08 PM > > Limiting the thread to Renesas only, again, to discuss the other issue: > > On 17.10.2017 06:51, Yoshihiro Shimoda wrote: > > Hi Linus-san, > > > >> From: Linus Walleij, Sent: Monday, October 16, 2017 9:22 PM > >> > >> On Mon, Oct 16, 2017 at 2:07 PM, Geert Uytterhoeven > >> wrote: > >>> On Mon, Oct 16, 2017 at 1:41 PM, Linus Walleij > >>> wrote: > On Mon, Oct 16, 2017 at 11:02 AM, Geert Uytterhoeven > wrote: > >> IIUC, > >> - Since the MMC_BOUNCE disappeared in v4.14-rc1, the MMC core will > >> request a command with 64kbyte+ size. > >> - So, swiotlb will got request 64kbyte+ size. This is different with > >> previous version. > > Other people reported problems with that. Sorry about this... > > This problem should be solved by upstream > commit de3ee99b097dd51938276e3af388cd4ad0f2750a > "mmc: Delete bounce buffer handling" > > I just killed off the bounce buffers so they can't hurt anyone anymore. > > Can you check that you have this patch in your tree? > >>> > >>> Given Dirk saw the issue with v4.14-rc4, I'm afraid that patch is already > >>> included :-( > >> > >> So the first issue is definately gone. > >> > >> Then it seems the driver is specifying to high ->max_seg_size > >> since the buffer cannot be mapped? > >> > >> Unless you're using ADMA drivers/mmc/host/sdhci.c sets it as: > >> mmc->max_seg_size = mmc->max_req_size; > > > > Thank you for the comment! > > On R-Car Gen3, it's using drivers/mmc/host/renesas_sdhi_internal_dmac.c, > > and sets the max_seg_size to very big size (0x * 512) now... > > > > So, I modified the value to 262144 and rebase the kernel to v4.14-rc5 > > (-rc4 seems have other issue in block driver.), the "swiotlb buffer is > > full" issue disappeared. > > > >> And: > >> mmc->max_req_size = 524288; > >> > renesas_sdhi_internal_dmac ee10.sd: swiotlb_tbl_map_single: test: > size modified to 524288 bytes > renesas_sdhi_internal_dmac ee10.sd: swiotlb buffer is full (sz: > 524288 bytes) > >> > >> So if your driver cannot handle 524288 bytes, why is it saying it can > >> handle 524288 bytes? > > > > I think the "swiotlb buffer is full" came from lib/swiotlb.c. > > I'm not sure the detail of swiotlb.c, but IIUC, it has a memory size > > limitation. > > > > - In lib/swiotlb.c, it calculates nslots. > > - The size of one slot is 2048 bytes because IO_TLB_SHIFT in > > include/linux/swiotlb.h is set to 11. > > - IO_TLB_SEGSIZE is set to 128 in include/linux/swiotlb.h. > > - This value is number of maximum slots. > >- So, maximum size for a single map of swiotlb is 256kbytes (2048 * 128). > > > > So, I will submit a patch to set the max_req_size to 256kbytes today. > > > Great, many thanks! I'll try to test that in the next days, then. > > While above fixes the swiotlb full issue, we still have the issue that > it looks like the driver doesn't handle that issue correctly and > crashes, then: > > renesas_sdhi_internal_dmac ee10.sd: DMA: Out of SW-IOMMU space for > 524288 bytes > -> Unable to handle kernel paging request at virtual address > c000 <- > > I'd propose to fix that, too. Have you looked into that issue, too? > > I'm no expert on this, but it looks to me that other drivers are > checking the return value of dma_map_sg() differently? > > ret = dma_map_sg(&host->pdev->dev, sg, host->sg_len, dir); > if (ret < 0) > return; > > https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/mmc/host/renesas_sdhi_internal_dmac. > c#n165 > > They check for (ret == 0)? > > Sorry if I'm wrong on that ;) Thank you for the comments! You're correct. We should check for (ret == 0) at least. I also think we should set host->force_pio = true and call renesas_sdhi_internal_dmac_enable_dma(host, false) to do PIO if "ret == 0". I will make such a patch and submit it. Best regards, Yoshihiro Shimoda > Best regards > > Dirk
Re: mainline 4.14-rc4: renesas_sdhi_internal_dmac ee100000.sd: swiotlb buffer is full?
Limiting the thread to Renesas only, again, to discuss the other issue: On 17.10.2017 06:51, Yoshihiro Shimoda wrote: Hi Linus-san, From: Linus Walleij, Sent: Monday, October 16, 2017 9:22 PM On Mon, Oct 16, 2017 at 2:07 PM, Geert Uytterhoeven wrote: On Mon, Oct 16, 2017 at 1:41 PM, Linus Walleij wrote: On Mon, Oct 16, 2017 at 11:02 AM, Geert Uytterhoeven wrote: IIUC, - Since the MMC_BOUNCE disappeared in v4.14-rc1, the MMC core will request a command with 64kbyte+ size. - So, swiotlb will got request 64kbyte+ size. This is different with previous version. Other people reported problems with that. Sorry about this... This problem should be solved by upstream commit de3ee99b097dd51938276e3af388cd4ad0f2750a "mmc: Delete bounce buffer handling" I just killed off the bounce buffers so they can't hurt anyone anymore. Can you check that you have this patch in your tree? Given Dirk saw the issue with v4.14-rc4, I'm afraid that patch is already included :-( So the first issue is definately gone. Then it seems the driver is specifying to high ->max_seg_size since the buffer cannot be mapped? Unless you're using ADMA drivers/mmc/host/sdhci.c sets it as: mmc->max_seg_size = mmc->max_req_size; Thank you for the comment! On R-Car Gen3, it's using drivers/mmc/host/renesas_sdhi_internal_dmac.c, and sets the max_seg_size to very big size (0x * 512) now... So, I modified the value to 262144 and rebase the kernel to v4.14-rc5 (-rc4 seems have other issue in block driver.), the "swiotlb buffer is full" issue disappeared. And: mmc->max_req_size = 524288; renesas_sdhi_internal_dmac ee10.sd: swiotlb_tbl_map_single: test: size modified to 524288 bytes renesas_sdhi_internal_dmac ee10.sd: swiotlb buffer is full (sz: 524288 bytes) So if your driver cannot handle 524288 bytes, why is it saying it can handle 524288 bytes? I think the "swiotlb buffer is full" came from lib/swiotlb.c. I'm not sure the detail of swiotlb.c, but IIUC, it has a memory size limitation. - In lib/swiotlb.c, it calculates nslots. - The size of one slot is 2048 bytes because IO_TLB_SHIFT in include/linux/swiotlb.h is set to 11. - IO_TLB_SEGSIZE is set to 128 in include/linux/swiotlb.h. - This value is number of maximum slots. - So, maximum size for a single map of swiotlb is 256kbytes (2048 * 128). So, I will submit a patch to set the max_req_size to 256kbytes today. Great, many thanks! I'll try to test that in the next days, then. While above fixes the swiotlb full issue, we still have the issue that it looks like the driver doesn't handle that issue correctly and crashes, then: renesas_sdhi_internal_dmac ee10.sd: DMA: Out of SW-IOMMU space for 524288 bytes -> Unable to handle kernel paging request at virtual address c000 <- I'd propose to fix that, too. Have you looked into that issue, too? I'm no expert on this, but it looks to me that other drivers are checking the return value of dma_map_sg() differently? ret = dma_map_sg(&host->pdev->dev, sg, host->sg_len, dir); if (ret < 0) return; https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/mmc/host/renesas_sdhi_internal_dmac.c#n165 They check for (ret == 0)? Sorry if I'm wrong on that ;) Best regards Dirk
RE: mainline 4.14-rc4: renesas_sdhi_internal_dmac ee100000.sd: swiotlb buffer is full?
Hi Linus-san, > From: Linus Walleij, Sent: Monday, October 16, 2017 9:22 PM > > On Mon, Oct 16, 2017 at 2:07 PM, Geert Uytterhoeven > wrote: > > On Mon, Oct 16, 2017 at 1:41 PM, Linus Walleij > > wrote: > >> On Mon, Oct 16, 2017 at 11:02 AM, Geert Uytterhoeven > >> wrote: > IIUC, > - Since the MMC_BOUNCE disappeared in v4.14-rc1, the MMC core will > request a command with 64kbyte+ size. > - So, swiotlb will got request 64kbyte+ size. This is different with > previous version. > >> > >> Other people reported problems with that. Sorry about this... > >> > >> This problem should be solved by upstream > >> commit de3ee99b097dd51938276e3af388cd4ad0f2750a > >> "mmc: Delete bounce buffer handling" > >> > >> I just killed off the bounce buffers so they can't hurt anyone anymore. > >> > >> Can you check that you have this patch in your tree? > > > > Given Dirk saw the issue with v4.14-rc4, I'm afraid that patch is already > > included :-( > > So the first issue is definately gone. > > Then it seems the driver is specifying to high ->max_seg_size > since the buffer cannot be mapped? > > Unless you're using ADMA drivers/mmc/host/sdhci.c sets it as: > mmc->max_seg_size = mmc->max_req_size; Thank you for the comment! On R-Car Gen3, it's using drivers/mmc/host/renesas_sdhi_internal_dmac.c, and sets the max_seg_size to very big size (0x * 512) now... So, I modified the value to 262144 and rebase the kernel to v4.14-rc5 (-rc4 seems have other issue in block driver.), the "swiotlb buffer is full" issue disappeared. > And: > mmc->max_req_size = 524288; > > >> renesas_sdhi_internal_dmac ee10.sd: swiotlb_tbl_map_single: test: > >> size modified to 524288 bytes > >> renesas_sdhi_internal_dmac ee10.sd: swiotlb buffer is full (sz: > >> 524288 bytes) > > So if your driver cannot handle 524288 bytes, why is it saying it can > handle 524288 bytes? I think the "swiotlb buffer is full" came from lib/swiotlb.c. I'm not sure the detail of swiotlb.c, but IIUC, it has a memory size limitation. - In lib/swiotlb.c, it calculates nslots. - The size of one slot is 2048 bytes because IO_TLB_SHIFT in include/linux/swiotlb.h is set to 11. - IO_TLB_SEGSIZE is set to 128 in include/linux/swiotlb.h. - This value is number of maximum slots. - So, maximum size for a single map of swiotlb is 256kbytes (2048 * 128). So, I will submit a patch to set the max_req_size to 256kbytes today. Best regards, Yoshihiro Shimoda
Re: mainline 4.14-rc4: renesas_sdhi_internal_dmac ee100000.sd: swiotlb buffer is full?
On Mon, Oct 16, 2017 at 2:07 PM, Geert Uytterhoeven wrote: > On Mon, Oct 16, 2017 at 1:41 PM, Linus Walleij > wrote: >> On Mon, Oct 16, 2017 at 11:02 AM, Geert Uytterhoeven >> wrote: IIUC, - Since the MMC_BOUNCE disappeared in v4.14-rc1, the MMC core will request a command with 64kbyte+ size. - So, swiotlb will got request 64kbyte+ size. This is different with previous version. >> >> Other people reported problems with that. Sorry about this... >> >> This problem should be solved by upstream >> commit de3ee99b097dd51938276e3af388cd4ad0f2750a >> "mmc: Delete bounce buffer handling" >> >> I just killed off the bounce buffers so they can't hurt anyone anymore. >> >> Can you check that you have this patch in your tree? > > Given Dirk saw the issue with v4.14-rc4, I'm afraid that patch is already > included :-( So the first issue is definately gone. Then it seems the driver is specifying to high ->max_seg_size since the buffer cannot be mapped? Unless you're using ADMA drivers/mmc/host/sdhci.c sets it as: mmc->max_seg_size = mmc->max_req_size; And: mmc->max_req_size = 524288; >> renesas_sdhi_internal_dmac ee10.sd: swiotlb_tbl_map_single: test: >> size modified to 524288 bytes >> renesas_sdhi_internal_dmac ee10.sd: swiotlb buffer is full (sz: >> 524288 bytes) So if your driver cannot handle 524288 bytes, why is it saying it can handle 524288 bytes? The MMC core will believe the claim is true, I guess maybe something in the current tree (such as coalesced blocks or whatever) is finally making this size of requests hit the driver (which is a good thing...) just that it was never doing that before and now it turns out it can't really handle that because of how the memory architecture or something works. I guess you need to figure out how big requests/segments this hardware can really handle and pass that to mmc->mas_req_size, mmc->max_seg_size. Sorry if my reasoning is dumb... need more coffee. Yours, Linus Walleij
Re: mainline 4.14-rc4: renesas_sdhi_internal_dmac ee100000.sd: swiotlb buffer is full?
Hi Linus,
On Mon, Oct 16, 2017 at 1:41 PM, Linus Walleij wrote:
> On Mon, Oct 16, 2017 at 11:02 AM, Geert Uytterhoeven
> wrote:
>>> IIUC,
>>> - Since the MMC_BOUNCE disappeared in v4.14-rc1, the MMC core will request
>>> a command with 64kbyte+ size.
>>> - So, swiotlb will got request 64kbyte+ size. This is different with
>>> previous version.
>
> Other people reported problems with that. Sorry about this...
>
> This problem should be solved by upstream
> commit de3ee99b097dd51938276e3af388cd4ad0f2750a
> "mmc: Delete bounce buffer handling"
>
> I just killed off the bounce buffers so they can't hurt anyone anymore.
>
> Can you check that you have this patch in your tree?
Given Dirk saw the issue with v4.14-rc4, I'm afraid that patch is already
included :-(
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
Re: mainline 4.14-rc4: renesas_sdhi_internal_dmac ee100000.sd: swiotlb buffer is full?
On Mon, Oct 16, 2017 at 11:02 AM, Geert Uytterhoeven wrote: >> IIUC, >> - Since the MMC_BOUNCE disappeared in v4.14-rc1, the MMC core will request a >> command with 64kbyte+ size. >> - So, swiotlb will got request 64kbyte+ size. This is different with >> previous version. Other people reported problems with that. Sorry about this... This problem should be solved by upstream commit de3ee99b097dd51938276e3af388cd4ad0f2750a "mmc: Delete bounce buffer handling" I just killed off the bounce buffers so they can't hurt anyone anymore. Can you check that you have this patch in your tree? Yours, Linus Walleij
Re: mainline 4.14-rc4: renesas_sdhi_internal_dmac ee100000.sd: swiotlb buffer is full?
CC LinusW, Ulf, linux-mmc
On Mon, Oct 16, 2017 at 10:36 AM, Yoshihiro Shimoda
wrote:
>> From: Dirk Behme, Sent: Thursday, October 12, 2017 6:13 PM
>> On 12.10.2017 10:20, Dirk Behme wrote:
>> > On 11.10.2017 15:20, Geert Uytterhoeven wrote:
>> >> On Wed, Oct 11, 2017 at 2:57 PM, Dirk Behme
>> >> wrote:
>> >>> On 11.10.2017 14:42, Geert Uytterhoeven wrote:
>> On Wed, Oct 11, 2017 at 2:23 PM, Dirk Behme
>> wrote:
>> > trying to boot recent mainline v4.14-rc4 on a custom H3 ES2.0 board
>> > with
>> > rootfs on SD card I'm getting [1].
>> >
>> > Last time I think I used v4.13 in the same environment and I think it
>> > worked
>> > fine, most probably because renesas_sdhi_internal_dmac wasn't
>> > there, yet
>> > ;)
>> >
>> > I checked renesas-drivers-2017-10-03-v4.14-rc3 if there is anything
>> > newer
>> > regarding drivers/mmc/host/renesas_sdhi_internal_dmac.c but doesn't
>> > look
>> > so.
>> >
>> > Any idea what I missed?
>> >
>> How much swiotlb memory do you have? The default is 64 MiB:
>>
>> software IO TLB [mem 0x77fff000-0x7bfff000] (64MB) mapped at
>> [ffc037fff000-ffc03bffefff]
>> >>>
>> >>> Same here:
>> >>
>> >> OK.
>> >>
>> Then, who else is consuming lots of swiotlb memory?
>> >>
>> >> Or, not freeing allocated memory.
>> >>
>> >>> Hmm, any idea how to find that?
>> >>>
>> >>> The dmesg doesn't seem to have more infos about that (?)
>> >>
>> >> I'm afraid you have to add some prints to swiotlb_tbl_map_single()
>> >> (and swiotlb_tbl_unmap_single())...
>> >
>> >
>> > With [1] I'm getting [2]. What doesn't look that bad, at least quite
>> > symmetric. Having a quick look I can't find any obvious not freed mem.
>> > Hmm ...
>>
>>
>> Ok, it seems to be somehow easy ;) It seems that the 524288 bytes them
>> self are the issue, independent on the usage of the swiotlb memory.
>>
>> Doing a hack like [3], i.e. trying to map 524288 bytes at the first
>> access, already, crashes, too [4].
>>
>> So seems we have two issues?
>>
>> a) swiotlb_tbl_map_single() can't handle 524288 bytes
>>
>> and
>>
>> b) failing swiotlb_tbl_map_single() results in a kernel crash
>>
>>
>> Now, I'm not sure whom to address both to? Is it an issue of
>> renesas_sdhi_internal_dmac() which shouldn't call swiotlb with 524288
>> bytes? Or is this an error of swiotlb? Same for the kernel crash.
>
> Our team also reproduced this issue if we did the following command on H3.
> # dd if=/dev/mmcblk0 of=/dev/null bs=512k count=200
>
> I investigate the issue and then if I modified the value of IO_TLB_SEGSIZE to
> 256,
> the issue disappeared. However, other issue happens if I dropped
> "count=200"...
> So, I will investigate the issues more...
>
> IIUC,
> - Since the MMC_BOUNCE disappeared in v4.14-rc1, the MMC core will request a
> command with 64kbyte+ size.
> - So, swiotlb will got request 64kbyte+ size. This is different with
> previous version.
>
> Best regards,
> Yoshihiro Shimoda
>
>>
>> Best regards
>>
>> Dirk
>>
>> [3]
>>
>> --- a/lib/swiotlb.c
>> +++ b/lib/swiotlb.c
>> @@ -510,6 +510,10 @@ phys_addr_t swiotlb_tbl_map_single(struct device
>> *hwdev,
>> if (sme_active())
>> pr_warn_once("SME is active and system is using DMA
>> bounce buffers\n");
>>
>> + dev_warn(hwdev, "swiotlb_tbl_map_single: trying to map %zd
>> bytes\n", size);
>> + size = (size_t)524288;
>> + dev_warn(hwdev, "swiotlb_tbl_map_single: test: size modified to
>> %zd bytes\n", size);
>> +
>> mask = dma_get_seg_boundary(hwdev);
>>
>> tbl_dma_addr &= mask;
>>
>> [4]
>>
>> renesas_sdhi_internal_dmac ee10.sd: swiotlb_tbl_map_single: trying
>> to map 8 bytes
>> renesas_sdhi_internal_dmac ee10.sd: swiotlb_tbl_map_single: test:
>> size modified to 524288 bytes
>> renesas_sdhi_internal_dmac ee10.sd: swiotlb buffer is full (sz:
>> 524288 bytes)
>> renesas_sdhi_internal_dmac ee10.sd: DMA: Out of SW-IOMMU space for 8
>> bytes
>> Unable to handle kernel paging request at virtual address c000
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
RE: mainline 4.14-rc4: renesas_sdhi_internal_dmac ee100000.sd: swiotlb buffer is full?
Hi Dirk-san,
> From: Dirk Behme, Sent: Thursday, October 12, 2017 6:13 PM
>
> On 12.10.2017 10:20, Dirk Behme wrote:
> > On 11.10.2017 15:20, Geert Uytterhoeven wrote:
> >> Hi Dirk,
> >>
> >> On Wed, Oct 11, 2017 at 2:57 PM, Dirk Behme
> >> wrote:
> >>> On 11.10.2017 14:42, Geert Uytterhoeven wrote:
> On Wed, Oct 11, 2017 at 2:23 PM, Dirk Behme
> wrote:
> > trying to boot recent mainline v4.14-rc4 on a custom H3 ES2.0 board
> > with
> > rootfs on SD card I'm getting [1].
> >
> > Last time I think I used v4.13 in the same environment and I think it
> > worked
> > fine, most probably because renesas_sdhi_internal_dmac wasn't
> > there, yet
> > ;)
> >
> > I checked renesas-drivers-2017-10-03-v4.14-rc3 if there is anything
> > newer
> > regarding drivers/mmc/host/renesas_sdhi_internal_dmac.c but doesn't
> > look
> > so.
> >
> > Any idea what I missed?
> >
> How much swiotlb memory do you have? The default is 64 MiB:
>
> software IO TLB [mem 0x77fff000-0x7bfff000] (64MB) mapped at
> [ffc037fff000-ffc03bffefff]
> >>>
> >>> Same here:
> >>
> >> OK.
> >>
> Then, who else is consuming lots of swiotlb memory?
> >>
> >> Or, not freeing allocated memory.
> >>
> >>> Hmm, any idea how to find that?
> >>>
> >>> The dmesg doesn't seem to have more infos about that (?)
> >>
> >> I'm afraid you have to add some prints to swiotlb_tbl_map_single()
> >> (and swiotlb_tbl_unmap_single())...
> >
> >
> > With [1] I'm getting [2]. What doesn't look that bad, at least quite
> > symmetric. Having a quick look I can't find any obvious not freed mem.
> > Hmm ...
>
>
> Ok, it seems to be somehow easy ;) It seems that the 524288 bytes them
> self are the issue, independent on the usage of the swiotlb memory.
>
> Doing a hack like [3], i.e. trying to map 524288 bytes at the first
> access, already, crashes, too [4].
>
> So seems we have two issues?
>
> a) swiotlb_tbl_map_single() can't handle 524288 bytes
>
> and
>
> b) failing swiotlb_tbl_map_single() results in a kernel crash
>
>
> Now, I'm not sure whom to address both to? Is it an issue of
> renesas_sdhi_internal_dmac() which shouldn't call swiotlb with 524288
> bytes? Or is this an error of swiotlb? Same for the kernel crash.
Our team also reproduced this issue if we did the following command on H3.
# dd if=/dev/mmcblk0 of=/dev/null bs=512k count=200
I investigate the issue and then if I modified the value of IO_TLB_SEGSIZE to
256,
the issue disappeared. However, other issue happens if I dropped "count=200"...
So, I will investigate the issues more...
IIUC,
- Since the MMC_BOUNCE disappeared in v4.14-rc1, the MMC core will request a
command with 64kbyte+ size.
- So, swiotlb will got request 64kbyte+ size. This is different with previous
version.
Best regards,
Yoshihiro Shimoda
>
> Best regards
>
> Dirk
>
> [3]
>
> --- a/lib/swiotlb.c
> +++ b/lib/swiotlb.c
> @@ -510,6 +510,10 @@ phys_addr_t swiotlb_tbl_map_single(struct device
> *hwdev,
> if (sme_active())
> pr_warn_once("SME is active and system is using DMA
> bounce buffers\n");
>
> + dev_warn(hwdev, "swiotlb_tbl_map_single: trying to map %zd
> bytes\n", size);
> + size = (size_t)524288;
> + dev_warn(hwdev, "swiotlb_tbl_map_single: test: size modified to
> %zd bytes\n", size);
> +
> mask = dma_get_seg_boundary(hwdev);
>
> tbl_dma_addr &= mask;
>
> [4]
>
> renesas_sdhi_internal_dmac ee10.sd: swiotlb_tbl_map_single: trying
> to map 8 bytes
> renesas_sdhi_internal_dmac ee10.sd: swiotlb_tbl_map_single: test:
> size modified to 524288 bytes
> renesas_sdhi_internal_dmac ee10.sd: swiotlb buffer is full (sz:
> 524288 bytes)
> renesas_sdhi_internal_dmac ee10.sd: DMA: Out of SW-IOMMU space for 8
> bytes
> Unable to handle kernel paging request at virtual address c000
Re: mainline 4.14-rc4: renesas_sdhi_internal_dmac ee100000.sd: swiotlb buffer is full?
On 12.10.2017 10:20, Dirk Behme wrote:
On 11.10.2017 15:20, Geert Uytterhoeven wrote:
Hi Dirk,
On Wed, Oct 11, 2017 at 2:57 PM, Dirk Behme
wrote:
On 11.10.2017 14:42, Geert Uytterhoeven wrote:
On Wed, Oct 11, 2017 at 2:23 PM, Dirk Behme
wrote:
trying to boot recent mainline v4.14-rc4 on a custom H3 ES2.0 board
with
rootfs on SD card I'm getting [1].
Last time I think I used v4.13 in the same environment and I think it
worked
fine, most probably because renesas_sdhi_internal_dmac wasn't
there, yet
;)
I checked renesas-drivers-2017-10-03-v4.14-rc3 if there is anything
newer
regarding drivers/mmc/host/renesas_sdhi_internal_dmac.c but doesn't
look
so.
Any idea what I missed?
How much swiotlb memory do you have? The default is 64 MiB:
software IO TLB [mem 0x77fff000-0x7bfff000] (64MB) mapped at
[ffc037fff000-ffc03bffefff]
Same here:
OK.
Then, who else is consuming lots of swiotlb memory?
Or, not freeing allocated memory.
Hmm, any idea how to find that?
The dmesg doesn't seem to have more infos about that (?)
I'm afraid you have to add some prints to swiotlb_tbl_map_single()
(and swiotlb_tbl_unmap_single())...
With [1] I'm getting [2]. What doesn't look that bad, at least quite
symmetric. Having a quick look I can't find any obvious not freed mem.
Hmm ...
Ok, it seems to be somehow easy ;) It seems that the 524288 bytes them
self are the issue, independent on the usage of the swiotlb memory.
Doing a hack like [3], i.e. trying to map 524288 bytes at the first
access, already, crashes, too [4].
So seems we have two issues?
a) swiotlb_tbl_map_single() can't handle 524288 bytes
and
b) failing swiotlb_tbl_map_single() results in a kernel crash
Now, I'm not sure whom to address both to? Is it an issue of
renesas_sdhi_internal_dmac() which shouldn't call swiotlb with 524288
bytes? Or is this an error of swiotlb? Same for the kernel crash.
Best regards
Dirk
[3]
--- a/lib/swiotlb.c
+++ b/lib/swiotlb.c
@@ -510,6 +510,10 @@ phys_addr_t swiotlb_tbl_map_single(struct device
*hwdev,
if (sme_active())
pr_warn_once("SME is active and system is using DMA
bounce buffers\n");
+ dev_warn(hwdev, "swiotlb_tbl_map_single: trying to map %zd
bytes\n", size);
+ size = (size_t)524288;
+ dev_warn(hwdev, "swiotlb_tbl_map_single: test: size modified to
%zd bytes\n", size);
+
mask = dma_get_seg_boundary(hwdev);
tbl_dma_addr &= mask;
[4]
renesas_sdhi_internal_dmac ee10.sd: swiotlb_tbl_map_single: trying
to map 8 bytes
renesas_sdhi_internal_dmac ee10.sd: swiotlb_tbl_map_single: test:
size modified to 524288 bytes
renesas_sdhi_internal_dmac ee10.sd: swiotlb buffer is full (sz:
524288 bytes)
renesas_sdhi_internal_dmac ee10.sd: DMA: Out of SW-IOMMU space for 8
bytes
Unable to handle kernel paging request at virtual address c000
Re: mainline 4.14-rc4: renesas_sdhi_internal_dmac ee100000.sd: swiotlb buffer is full?
Hi Dirk,
On Wed, Oct 11, 2017 at 2:57 PM, Dirk Behme wrote:
> On 11.10.2017 14:42, Geert Uytterhoeven wrote:
>> On Wed, Oct 11, 2017 at 2:23 PM, Dirk Behme
>> wrote:
>>> trying to boot recent mainline v4.14-rc4 on a custom H3 ES2.0 board with
>>> rootfs on SD card I'm getting [1].
>>>
>>> Last time I think I used v4.13 in the same environment and I think it
>>> worked
>>> fine, most probably because renesas_sdhi_internal_dmac wasn't there, yet
>>> ;)
>>>
>>> I checked renesas-drivers-2017-10-03-v4.14-rc3 if there is anything newer
>>> regarding drivers/mmc/host/renesas_sdhi_internal_dmac.c but doesn't look
>>> so.
>>>
>>> Any idea what I missed?
>>>
>> How much swiotlb memory do you have? The default is 64 MiB:
>>
>> software IO TLB [mem 0x77fff000-0x7bfff000] (64MB) mapped at
>> [ffc037fff000-ffc03bffefff]
>
> Same here:
OK.
>> Then, who else is consuming lots of swiotlb memory?
Or, not freeing allocated memory.
> Hmm, any idea how to find that?
>
> The dmesg doesn't seem to have more infos about that (?)
I'm afraid you have to add some prints to swiotlb_tbl_map_single()
(and swiotlb_tbl_unmap_single())...
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
Re: mainline 4.14-rc4: renesas_sdhi_internal_dmac ee100000.sd: swiotlb buffer is full?
Hi Geert, On 11.10.2017 14:42, Geert Uytterhoeven wrote: Hi Dirk, On Wed, Oct 11, 2017 at 2:23 PM, Dirk Behme wrote: trying to boot recent mainline v4.14-rc4 on a custom H3 ES2.0 board with rootfs on SD card I'm getting [1]. Last time I think I used v4.13 in the same environment and I think it worked fine, most probably because renesas_sdhi_internal_dmac wasn't there, yet ;) I checked renesas-drivers-2017-10-03-v4.14-rc3 if there is anything newer regarding drivers/mmc/host/renesas_sdhi_internal_dmac.c but doesn't look so. Any idea what I missed? How much swiotlb memory do you have? The default is 64 MiB: software IO TLB [mem 0x77fff000-0x7bfff000] (64MB) mapped at [ffc037fff000-ffc03bffefff] Same here: software IO TLB [mem 0x4000-0x53fff000] (64MB) mapped at [8000-800013ffefff] Then, who else is consuming lots of swiotlb memory? Hmm, any idea how to find that? The dmesg doesn't seem to have more infos about that (?) Freeing unused kernel memory: 512K usb 3-1: new high-speed USB device number 2 using ehci-platform usb 3-1: New USB device found, idVendor=0424, idProduct=2514 usb 3-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 hub 3-1:1.0: USB hub found hub 3-1:1.0: 4 ports detected systemd[1]: System time before build time, advancing clock. renesas_sdhi_internal_dmac ee10.sd: swiotlb buffer is full (sz: 524288 bytes) renesas_sdhi_internal_dmac ee10.sd: DMA: Out of SW-IOMMU space for 524288 bytes Unable to handle kernel paging request at virtual address c000 Ugh, and the mapping failure isn't even handled correctly... :( Best regards Dirk
Re: mainline 4.14-rc4: renesas_sdhi_internal_dmac ee100000.sd: swiotlb buffer is full?
Hi Dirk,
On Wed, Oct 11, 2017 at 2:23 PM, Dirk Behme wrote:
> trying to boot recent mainline v4.14-rc4 on a custom H3 ES2.0 board with
> rootfs on SD card I'm getting [1].
>
> Last time I think I used v4.13 in the same environment and I think it worked
> fine, most probably because renesas_sdhi_internal_dmac wasn't there, yet ;)
>
> I checked renesas-drivers-2017-10-03-v4.14-rc3 if there is anything newer
> regarding drivers/mmc/host/renesas_sdhi_internal_dmac.c but doesn't look so.
>
> Any idea what I missed?
How much swiotlb memory do you have? The default is 64 MiB:
software IO TLB [mem 0x77fff000-0x7bfff000] (64MB) mapped at
[ffc037fff000-ffc03bffefff]
Then, who else is consuming lots of swiotlb memory?
> Freeing unused kernel memory: 512K
> usb 3-1: new high-speed USB device number 2 using ehci-platform
> usb 3-1: New USB device found, idVendor=0424, idProduct=2514
> usb 3-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0
> hub 3-1:1.0: USB hub found
> hub 3-1:1.0: 4 ports detected
> systemd[1]: System time before build time, advancing clock.
> renesas_sdhi_internal_dmac ee10.sd: swiotlb buffer is full (sz: 524288
> bytes)
> renesas_sdhi_internal_dmac ee10.sd: DMA: Out of SW-IOMMU space for
> 524288 bytes
> Unable to handle kernel paging request at virtual address c000
Ugh, and the mapping failure isn't even handled correctly...
Gr{oetje,eeting}s,
Geert
--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- ge...@linux-m68k.org
In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
-- Linus Torvalds
mainline 4.14-rc4: renesas_sdhi_internal_dmac ee100000.sd: swiotlb buffer is full?
Hi, trying to boot recent mainline v4.14-rc4 on a custom H3 ES2.0 board with rootfs on SD card I'm getting [1]. Last time I think I used v4.13 in the same environment and I think it worked fine, most probably because renesas_sdhi_internal_dmac wasn't there, yet ;) I checked renesas-drivers-2017-10-03-v4.14-rc3 if there is anything newer regarding drivers/mmc/host/renesas_sdhi_internal_dmac.c but doesn't look so. Any idea what I missed? Best regards Dirk [1] ... Freeing unused kernel memory: 512K usb 3-1: new high-speed USB device number 2 using ehci-platform usb 3-1: New USB device found, idVendor=0424, idProduct=2514 usb 3-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 hub 3-1:1.0: USB hub found hub 3-1:1.0: 4 ports detected systemd[1]: System time before build time, advancing clock. renesas_sdhi_internal_dmac ee10.sd: swiotlb buffer is full (sz: 524288 bytes) renesas_sdhi_internal_dmac ee10.sd: DMA: Out of SW-IOMMU space for 524288 bytes Unable to handle kernel paging request at virtual address c000 Mem abort info: Exception class = DABT (current EL), IL = 32 bits SET = 0, FnV = 0 EA = 0, S1PTW = 0 Data abort info: ISV = 0, ISS = 0x0144 CM = 1, WnR = 1 swapper pgtable: 4k pages, 48-bit VAs, pgd = 08aa5000 [c000] *pgd= Internal error: Oops: 96000144 [#1] PREEMPT SMP Modules linked in: CPU: 0 PID: 0 Comm: swapper/0 Not tainted 4.14.0-rc4-dirty #1 Hardware name: Bosch board based on r8a7795 (DT) task: 0890e900 task.stack: 0890 PC is at __dma_inv_area+0x40/0x58 LR is at __swiotlb_unmap_sg_attrs+0xa8/0xb8 pc : [] lr : [] pstate: 41c5 sp : 08003df0 x29: 08003df0 x28: 088e6000 x27: 000a x26: 08811000 x25: x24: 0002 x23: 0001 x22: 8006f991f280 x21: 0001 x20: 8006faa56010 x19: 8006f991f280 x18: 000a x17: 000e x16: 0007 x15: 0009d4ff x14: 3d08e7ffed39 x13: 8235 x12: 0002 x11: 7fff x10: 0002 x9 : 08926640 x8 : 006d x7 : 08926638 x6 : x5 : 08092948 x4 : x3 : 003f x2 : 0040 x1 : c008 x0 : c000 Process swapper/0 (pid: 0, stack limit = 0x0890) Call trace: Exception stack(0x08003cb0 to 0x08003df0) 3ca0: c000 c008 3cc0: 0040 003f 08092948 3ce0: 08926638 006d 08926640 3d00: 0002 7fff 0002 8235 3d20: 3d08e7ffed39 0009d4ff 0007 000e 3d40: 000a 8006f991f280 8006faa56010 0001 3d60: 8006f991f280 0001 0002 3d80: 08811000 000a 088e6000 08003df0 3da0: 080929f0 08003df0 08094414 41c5 3dc0: 0001 3de0: 08003df0 08094414 [] __dma_inv_area+0x40/0x58 [] renesas_sdhi_internal_dmac_complete_tasklet_fn+0x7c/0xa8 [] tasklet_action+0xd8/0x144 [] __do_softirq+0x2d4/0x394 [] irq_exit+0x60/0xac [] __handle_domain_irq+0xa0/0xec [] gic_handle_irq+0x80/0xac Exception stack(0x08903dd0 to 0x08903f10) 3dc0: 087a9ee0 3de0: 0001 088e6000 08a57b70 3e00: 0446 00c0 0890f3e0 08903e90 3e20: 0a80 8006fff16fc0 0001 3e40: 6620656361707320 0009d4ff 0007 000e 3e60: 000a 088eee10 3e80: 08908aa0 8006fa00 088d2028 3ea0: 49880018 08903f10 3ec0: 080849b0 08903f10 080849b4 0045 3ee0: 088eee10 08908aa0 088d2028 3f00: 08903f10 080849b4 [] el1_irq+0xb4/0x12c [] arch_cpu_idle+0x140/0x19c [] default_idle_call+0x2c/0x34 [] do_idle+0xfc/0x19c [] cpu_startup_entry+0x20/0x24 [] rest_init+0xc8/0xd4 [] start_kernel+0x38c/0x3a0 Code: 8a23 5460 d50b7e20 1402 (d5087620) ---[ end trace 6a04242e7e1f35dd ]--- Kernel panic - not syncing: Fatal exception in interrupt SMP: stopping secondary CPUs Kernel Offset: disabled CPU features: 0x002004 Memory Limit: 4096 MB ---[ end Kernel panic - not syncing: Fatal exception in interrupt
