Hi Ben,
The second issue. you told me "some hosts" has problem,
and some hosts worked well.
what is the problem-hosts?
The issue seems like the hosts did set the NO SNOOP attribute
bit at TLP.
The PEX_DEVICE_CONTROL is standard PCI configuration space
register, it controls the behavior of the initiator's transaction.
For 8315, it is outbound, not inbound transaction.
Thanks, Dave
________________________________
From: Ben Menchaca [mailto:ben.mench...@gmail.com]
Sent: Saturday, March 07, 2009 12:30 AM
To: Liu Dave-R63238
Cc: linuxppc-dev@ozlabs.org
Subject: Re: 83xx: Marking or Allocating Pages as
Cache-Inhibited
Thank you for your help! That bit resolved all of the RDMA/WDMA
coherency issues on the CSB side...except:
We expose a 1MB region of memory from CSB via a BAR (BAR1, if it
matters) to the Host. This region is also not behaving correctly with
respect to coherency on SOME hosts; again, disabling our cache makes it
work correctly on all hosts. We have set PEX_DEVICE_CONTROL in PCI-E
Config Space (0x54) to 0x2010 (sorry about the endianness below). We
thought that CLEARING the no-snoop bit here would indicate that snooping
was required for this region...is this a similar issue?
- Ben
On Fri, Mar 6, 2009 at 10:12 AM, Ben Menchaca
<ben.mench...@gmail.com> wrote:
Testing now...it looks like it (almost) works, though!
Why does setting no-snoop cause snooping to work? More on the effect on
setting that bit in a few minutes...need more testing.
ACR is 0x00030300.
- Ben
On Fri, Mar 6, 2009 at 12:30 AM, Liu Dave-R63238
<dave...@freescale.com> wrote:
Did you enable the descriptor bit 3 to have a
try?
________________________________
From: Ben Menchaca
[mailto:ben.mench...@gmail.com]
Sent: Friday, March 06, 2009 2:10 PM
To: Liu Dave-R63238
Cc: linuxppc-dev@ozlabs.org
Subject: Re: 83xx: Marking or Allocating
Pages as Cache-Inhibited
I can look at ACR morning...although I
can say with a fair amount of certainty that I have not changed it from
the POR value.
I will try enabling No Snoop for CSB in
the descriptor (bit 3, yes?)...this seems a bit counterintuitive to me.
What is the hope regarding these two?
Some combination I am not seeing?
On Thu, Mar 5, 2009 at 11:40 PM, Liu
Dave-R63238 <dave...@freescale.com> wrote:
what is the value of ACR register?
________________________________
From: Ben Menchaca
[mailto:ben.mench...@gmail.com]
Sent: Friday, March 06, 2009 1:38 PM
To: Liu Dave-R63238
Cc: linuxppc-dev@ozlabs.org
Subject: Re: 83xx: Marking or Allocating
Pages as Cache-Inhibited
1. BAT2 in linux is set to WIMG=0010,
and covers all 64M
2. PEX_DEVICE_CONTROL in PCI-E Config
Space (0x54): 0x1020
3. PEX_xDMA_CTRL is set to 0x00000401
at the initiation of the DMA.
4. OWAR0 is set to 0xFFFFF005, so NSNP
is 0.
5. The DMA descriptor (randomly chosen
when I hit a trigger...just ignore the size...) contains 0002AFF3 at
offset 0, so nosnoops are cleared.
Core is 400MHz, and CSB is 133MHz.
- Ben
On Thu, Mar 5, 2009 at 11:27 PM, Liu
Dave-R63238 <dave...@freescale.com> wrote:
and what settings is DMA description
bit 3?
> -----Original Message-----
> From:
linuxppc-dev-bounces+daveliu=freescale....@ozlabs.org
> [mailto:linuxppc-dev-bounces+daveliu
<mailto:linuxppc-dev-bounces%2Bdaveliu> =freescale....@ozlabs.org]
> On Behalf Of Liu Dave-R63238
> Sent: Friday, March 06, 2009 1:22 PM
> To: Ben Menchaca;
linuxppc-dev@ozlabs.org
> Subject: RE: 83xx: Marking or
Allocating Pages as Cache-Inhibited
>
> Did you enable the snoop bit at
PEX_WDMA_CTRL[SNOOP] and
> PEX_RDMA_CTRL[SNOOP]?
>
> What is the freq settings? CORE/CSB
bus.
>
> Thanks, Dave
>
> ________________________________
>
> From:
linuxppc-dev-bounces+daveliu=freescale....@ozlabs.org
> [mailto:linuxppc-dev-bounces+daveliu
<mailto:linuxppc-dev-bounces%2Bdaveliu> =freescale....@ozlabs.org]
> On Behalf Of Ben Menchaca
> Sent: Friday, March 06, 2009
12:33 PM
> To: linuxppc-dev@ozlabs.org
> Subject: 83xx: Marking or
Allocating Pages as Cache-Inhibited
>
>
> I am working on a Freescale
8314e design, and the
> embedded device is configured as a
PCI-e endpoint running a
> 2.6.27-5 kernel. For context, we have
written a kernel
> module which, among other things, uses
the RDMA/WDMA engine
> in the PCI-e IP block. On the host
side, these DMAs are
> coherent. However, on the embedded
side, things are quite a
> bit less rosy; we must manually
flush/invalidate cache lines
> for WDMA/RDMAs to occur successfully.
After speaking with
> (several) FAEs at Freescale, we
believe there is a
> configuration issue that is the cause,
but we have yet to
> have anyone successfully point to it.
>
> Disabling the data cache
altogether resolves the issue
> entirely, but of course, also
completely tanks performance.
> As a temporary workaround, I would
like to simply mark the
> pages (obtained currently via
dma_alloc_coherent) involved as
> cache-inhibited. I have attempted to
do this via some
> snippets remaining in fec.c
(va_to_pte, uncache_pte to set
> _PAGE_NO_CACHE, flush_tlb_page, then
unmap_pte), but this is
> almost certainly braindead; va_to_pte
is not a part of the
> 83xx source, as far as I can tell; 8xx
only.
>
> A quick pointer in the correct
direction for marking
> pages as cache-inhibited on a 2.6.27-5
kernel would be
> appreciated, or if my approach to a
workaround is flawed, a
> pointer to the correct way would be
great.
>
> Ben Menchaca
>
>
>
_______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@ozlabs.org
>
https://ozlabs.org/mailman/listinfo/linuxppc-dev
>
>
_______________________________________________
Linuxppc-dev mailing list
Linuxppc-dev@ozlabs.org
https://ozlabs.org/mailman/listinfo/linuxppc-dev
