Rather than flushing cache you can create a non-cached section of RAM in
the linker script and put buffers there.
.ram_nocache (NOLOAD):
{
. = ALIGN(4);
_s_ram_nocache = .;
*(.ram_nocache)
. = ALIGN(4);
_e_ram_nocache = .;
} > ram_nocache
.ram_nocache_data : AT (_etext + SIZEOF(.relocate))
{
. = ALIGN(4);
_s_ram_nocache_vma = .;
_s_ram_nocache_lma = LOADADDR(.ram_nocache_data);
*(.ram_nocache_data)
. = ALIGN(4);
_e_ram_nocache_vma = .;
} > ram_nocache
Then in your start up code you configure the MPU to disable the cache for
this area of code:
// START THE NO CACHE DATA SEGMENT
/* Initialize the no cache data segment */
pSrc = &_s_ram_nocache_lma;
pDest = &_s_ram_nocache_vma;
if (pSrc != pDest) {
for (; pDest < &_e_ram_nocache_vma;) {
*pDest++ = *pSrc++;
}
}
/* Clear the no cache zero segment */
for (pDest = &_s_ram_nocache; pDest < &_e_ram_nocache;) {
*pDest++ = 0;
}
// END THE NO CACHE DATA SEGMENT
#define SRAM_NOCACHE_START_ADDRESS (uint32_t)(&_s_ram_nocache)
#define NOCACHE_SRAM_REGION_SIZE ((uint32_t)()&_e_ram_nocache) -
(uint32_t)(&_s_ram_nocache))
#define MPU_NOCACHE_SRAM_REGION (11)
#define INNER_OUTER_NORMAL_NOCACHE_TYPE(x) ((0x01 << MPU_RASR_TEX_Pos ) | (
0 <<MPU_RASR_C_Pos ) | \
( 0 << MPU_RASR_B_Pos ) | ( x << MPU_RASR_S_Pos))
/* Disable the MPU region */
MPU->CTRL = 0; //MPU_CTRL_ENABLE_Msk;
uint32_t dw_region_base_addr = SRAM_NOCACHE_START_ADDRESS |
MPU_RBAR_VALID_Msk |
MPU_NOCACHE_SRAM_REGION;
uint32_t dw_region_attr = 0x03<< MPU_RASR_AP_Pos|
INNER_OUTER_NORMAL_NOCACHE_TYPE( 1 ) |
(15<<1) | //set size to 2^16 as we are at end of SRAM and memory above not
used
//mpu_cal_mpu_region_size(NOCACHE_SRAM_REGION_SIZE) |
MPU_RASR_ENABLE_Msk;
MPU->RBAR = dw_region_base_addr;
MPU->RASR = dw_region_attr;
/* Enable the MPU region */
MPU->CTRL = (MPU_CTRL_ENABLE_Msk | MPU_CTRL_PRIVDEFENA_Msk);
__DSB();
__ISB();
Then you can put your buffers in this section of RAM and it will not be
cached.
__attribute__ ((section (".ram_nocache"), aligned (8))) uint8_t
mydata[...];
For the ATSAM part the GMAC driver used a RAM buffer and internal DMA. So
the driver checked a byte in the RAM buffer to see when the DMA was done,
basically polled that byte. Since the buffer was cached the byte never
changed as the core had no idea the DMA was happening.
This same problem occurred anytime DMA was used, be it using DMA peripheral
or with other peripherals (USB and GMAC) that had internal DMA engines.
One option is to invalidate cache before reading these buffers, the other
is to use a non-cached region of RAM for the buffers.
Like you I questioned the timing, so I put buffers in the non-cached region
and verified that this did fix the problem. Now I was confident that this
was indeed the problem and not a timing problem.
Trampas
On Thu, Mar 25, 2021 at 4:06 AM Osborne, David <
david.osbo...@leicabiosystems.com> wrote:
> Damn you L1 cache! I would disable you permanently if you weren’t so damn
> fast. :)
>
>
>
> It appears cleaning the cache over the tx buffers address space has fixed
> the issue. My only concern is, it shouldn’t have. The tx buffers are set
> up as Normal Write through memory (T=0, C=1, B=0, S=0). I have a sneaky
> suspicion cleaning the cache is simply providing the delay that the debug
> was (but hey a fix is a fix)
>
>
>
> @Alister Fisher <alis...@advateklights.com> I am using your Ethernet
> patches. I was hoping to switch back to the mainstream by now, but I’m not
> confident ST has fixed all the issues yet.
>
>
>
> Thanks for your help everyone.
>
>
>
> Cheers
>
> David
>
>
>
> *From:* lwip-users <lwip-users-bounces+david.osborne=
> leicabiosystems....@nongnu.org> *On Behalf Of *Trampas Stern
> *Sent:* Thursday, 25 March 2021 4:10 AM
> *To:* Mailing list for lwIP users <lwip-users@nongnu.org>
> *Subject:* Re: [lwip-users] Why does enabling Debug fix this tcp_sndbuf()
>
>
>
> I had similar problem with an ATSAME70 which was an M7 core. Turns out
> that the ethernet driver from Microchip did not invalidate cache for the
> ethernet MAC driver. Hence it would work some of the time and not others,
> like when debugging was on. Disabling the data cache on the processor
> would allow code to run with no errors.
>
>
>
> Microchip told me that none of their drivers were designed to work with
> cache on and to run ANY sample code or drivers required instruction and
> data cache to be turned off. Needless to say I rewrote their drivers.
>
>
>
> Also note the need for barrier instructions (__DSB(), __ISB()) these are
> often needed and vendor's drivers are missing them. For example barriers
> are often needed at end of interrupt handlers. Note even then some
> peripherals have caches in the peripherals that can not be flushed with
> barrier instructions, for example Flash memory.
>
> I also ran into this as that right after writing to flash memory I would
> read back the wrong values due to flash peripheral caching. I had to
> create my own cache flush by reading large block of flash no near where I
> wrote, then returning to where I wrote and reading again. The peripheral
> driver had a cache flush that did not work, so the reading different region
> was the only way to ensure the flash cache was flushed.
>
>
>
>
>
> The function below disables the cache in the core, part of core_cm7.h
>
>
>
>
>
> /**
> \brief Disable D-Cache
> \details Turns off D-Cache
> */
> __STATIC_INLINE void SCB_DisableDCache (void)
> {
> #if defined (__DCACHE_PRESENT) && (__DCACHE_PRESENT == 1U)
> register uint32_t ccsidr;
> register uint32_t sets;
> register uint32_t ways;
>
> SCB->CSSELR = 0U; /*(0U << 1U) | 0U;*/ /* Level 1 data cache */
> __DSB();
>
> SCB->CCR &= ~(uint32_t)SCB_CCR_DC_Msk; /* disable D-Cache */
> __DSB();
>
> ccsidr = SCB->CCSIDR;
>
> /* clean & invalidate D-Cache
> */
> sets = (uint32_t)(CCSIDR_SETS(ccsidr));
> do {
> ways = (uint32_t)(CCSIDR_WAYS(ccsidr));
> do {
> SCB->DCCISW = (((sets << SCB_DCCISW_SET_Pos) & SCB_DCCISW_SET_Msk)
> |
> ((ways << SCB_DCCISW_WAY_Pos) & SCB_DCCISW_WAY_Msk)
> );
> #if defined ( __CC_ARM )
> __schedule_barrier();
> #endif
> } while (ways-- != 0U);
> } while(sets-- != 0U);
>
> __DSB();
> __ISB();
> #endif
> }
>
>
>
>
>
> On Wed, Mar 24, 2021 at 3:11 PM goldsi...@gmx.de <goldsi...@gmx.de> wrote:
>
> Am 24.03.2021 um 16:32 schrieb Osborne, David:
> > Hi All,
> >
> >
> >
> > I’m using LwIP + FreeRTOS + Sockets on STM32H7 with external SDRAM and
> > FLASH (Ethernet buffers are internal). It works well until the server
> > has to send a very large message (> 18Kbytes). One in five attempts
> > ends in failure. During the transmission tcp_sndbuf() (in api_msg.c)
> > returns zero and the connection falls over. From what I’ve read, a
> > common cause is that it’s waiting for an outstanding ACK. I’m no
> > expert, but it looks like the ACK has been sent (see Wireshark
> attachment).
> >
> >
> >
> > When I enable certain debug settings it works every time (don’t you hate
> > that).
>
> That's an indication that there's something wrong that's hidden by
> either longer runtime (more debug code enabled) or by heavier cache
> usage (more code/data in the caches).
>
> In any case, I'd suggest thoroughly review the OS port and the netif
> driver regarding caching issues and/or threading issues (where interrupt
> vs. thread/main loop also counts in).
>
> Regards,
> Simon
>
> _______________________________________________
> lwip-users mailing list
> lwip-users@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/lwip-users
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.nongnu.org_mailman_listinfo_lwip-2Dusers&d=DwMFaQ&c=9mghv0deYPYDGP-W745IEdQLV1kHpn4XJRvR6xMRXtA&r=QVeuakQCDAOAs7rR-ApJqAYA3YIWaYZKYG5tVksGK7pzGH-yC2t6CxcUbEr-zvEH&m=FmLt5cJKfAKGxuFfmhWGCfR7IrPAeUTRfwUjJkNhqyc&s=3yHQAB14cwwWkkhj2FjbpZ0JTE8SqNhcPp5bh01hFYw&e=>
>
> Please be advised that this email may contain confidential information. If
> you are not the intended recipient, please notify us by email by replying
> to the sender and delete this message. The sender disclaims that the
> content of this email constitutes an offer to enter into, or the acceptance
> of, any agreement; provided that the foregoing does not invalidate the
> binding effect of any digital or other electronic reproduction of a manual
> signature that is included in any attachment.
> _______________________________________________
> lwip-users mailing list
> lwip-users@nongnu.org
> https://lists.nongnu.org/mailman/listinfo/lwip-users
_______________________________________________
lwip-users mailing list
lwip-users@nongnu.org
https://lists.nongnu.org/mailman/listinfo/lwip-users
